2024-05-08T23:17:20.709 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T23:17:20.709 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/pool-perm} --name pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698173 --owner scheduled_pdonnell@teuthology --verbose -- /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698173/orig.config.yaml 2024-05-08T23:17:20.752 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T23:17:20.844 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698173 branch: wip-pdonnell-testing-20240508.150423-reef description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/pool-perm} email: pdonnell@redhat.com first_in_suite: false job_id: '7698173' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-pdonnell-testing-20240508.150423-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.150423-reef sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 2738 sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 sleep_before_teardown: 0 subset: 17/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.150423-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi081.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_pool_perm teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_19:22:21 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T23:17:20.845 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa; will attempt to use it 2024-05-08T23:17:20.845 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks 2024-05-08T23:17:20.845 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T23:17:20.847 INFO:teuthology.task.internal:Checking packages... 2024-05-08T23:17:20.872 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd21e4f961318b56f0d915caa2490c3711bf13c34' 2024-05-08T23:17:20.872 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T23:17:20.872 INFO:teuthology.packaging:ref: None 2024-05-08T23:17:20.872 INFO:teuthology.packaging:tag: None 2024-05-08T23:17:20.872 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.150423-reef 2024-05-08T23:17:20.872 INFO:teuthology.packaging:sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:17:20.873 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-pdonnell-testing-20240508.150423-reef 2024-05-08T23:17:21.011 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1322-gd21e4f96-1focal 2024-05-08T23:17:21.012 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T23:17:21.018 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T23:17:21.018 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T23:17:21.067 INFO:teuthology.task.internal:Saving configuration 2024-05-08T23:17:21.083 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T23:17:21.111 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T23:17:21.136 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698173', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 23:11:12.241683', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-05-08T23:17:21.157 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698173', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 23:11:12.240131', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-05-08T23:17:21.157 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T23:17:21.163 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-08T23:17:21.164 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-08T23:17:21.164 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T23:17:21.321 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f551472f4c0>, signals=[15]) 2024-05-08T23:17:21.322 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T23:17:21.328 INFO:teuthology.task.internal:Opening connections... 2024-05-08T23:17:21.328 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:17:21.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:17:21.440 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:17:21.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:17:21.529 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T23:17:21.535 DEBUG:teuthology.orchestra.run.smithi081:> uname -m 2024-05-08T23:17:21.542 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2024-05-08T23:17:21.542 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2024-05-08T23:17:21.590 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="Ubuntu" 2024-05-08T23:17:21.590 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-05-08T23:17:21.590 INFO:teuthology.orchestra.run.smithi081.stdout:ID=ubuntu 2024-05-08T23:17:21.590 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE=debian 2024-05-08T23:17:21.590 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-05-08T23:17:21.590 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="20.04" 2024-05-08T23:17:21.590 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-08T23:17:21.590 INFO:teuthology.orchestra.run.smithi081.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-08T23:17:21.590 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-08T23:17:21.590 INFO:teuthology.orchestra.run.smithi081.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-08T23:17:21.590 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_CODENAME=focal 2024-05-08T23:17:21.591 INFO:teuthology.orchestra.run.smithi081.stdout:UBUNTU_CODENAME=focal 2024-05-08T23:17:21.591 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2024-05-08T23:17:21.618 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-05-08T23:17:21.624 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-05-08T23:17:21.625 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-05-08T23:17:21.672 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="Ubuntu" 2024-05-08T23:17:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-05-08T23:17:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:ID=ubuntu 2024-05-08T23:17:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE=debian 2024-05-08T23:17:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-05-08T23:17:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="20.04" 2024-05-08T23:17:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-08T23:17:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-08T23:17:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-08T23:17:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-08T23:17:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_CODENAME=focal 2024-05-08T23:17:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:UBUNTU_CODENAME=focal 2024-05-08T23:17:21.674 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-05-08T23:17:21.692 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T23:17:21.701 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T23:17:21.717 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T23:17:21.717 DEBUG:teuthology.orchestra.run.smithi081:> test '!' -e /home/ubuntu/cephtest 2024-05-08T23:17:21.719 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-05-08T23:17:21.725 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T23:17:21.741 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T23:17:21.742 DEBUG:teuthology.orchestra.run.smithi081:> test -z $(ls -A /var/lib/ceph) 2024-05-08T23:17:21.744 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-05-08T23:17:21.762 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T23:17:21.819 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T23:17:21.834 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-08T23:17:21.834 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-08T23:17:21.835 DEBUG:teuthology.orchestra.run.smithi081:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T23:17:21.835 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T23:17:21.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:17:21.841 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-05-08T23:17:21.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:17:21.842 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-05-08T23:17:21.848 INFO:teuthology.orchestra.run.smithi117.stdout:5.4.0-124-generic 2024-05-08T23:17:21.848 INFO:teuthology.task.kernel:Running kernel on smithi117: 5.4.0-124-generic 2024-05-08T23:17:21.848 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get clean 2024-05-08T23:17:21.849 INFO:teuthology.orchestra.run.smithi081.stdout:5.4.0-124-generic 2024-05-08T23:17:21.849 INFO:teuthology.task.kernel:Running kernel on smithi081: 5.4.0-124-generic 2024-05-08T23:17:21.850 DEBUG:teuthology.orchestra.run.smithi081:> sudo apt-get clean 2024-05-08T23:17:21.954 DEBUG:teuthology.orchestra.run.smithi081:> sudo apt-get update 2024-05-08T23:17:21.992 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get update 2024-05-08T23:17:22.237 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-05-08T23:17:22.238 INFO:teuthology.orchestra.run.smithi117.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2024-05-08T23:17:22.238 INFO:teuthology.orchestra.run.smithi117.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2024-05-08T23:17:22.248 INFO:teuthology.orchestra.run.smithi081.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-05-08T23:17:22.253 INFO:teuthology.orchestra.run.smithi081.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2024-05-08T23:17:22.298 INFO:teuthology.orchestra.run.smithi081.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2024-05-08T23:17:22.317 INFO:teuthology.orchestra.run.smithi117.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2024-05-08T23:17:22.628 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,920 kB] 2024-05-08T23:17:22.652 INFO:teuthology.orchestra.run.smithi081.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2024-05-08T23:17:22.722 INFO:teuthology.orchestra.run.smithi081.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [748 kB] 2024-05-08T23:17:22.782 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,296 kB] 2024-05-08T23:17:22.782 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [748 kB] 2024-05-08T23:17:22.792 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [438 kB] 2024-05-08T23:17:22.798 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2024-05-08T23:17:22.798 INFO:teuthology.orchestra.run.smithi117.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,799 kB] 2024-05-08T23:17:22.820 INFO:teuthology.orchestra.run.smithi117.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [36.4 kB] 2024-05-08T23:17:22.820 INFO:teuthology.orchestra.run.smithi117.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [391 kB] 2024-05-08T23:17:22.824 INFO:teuthology.orchestra.run.smithi117.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [552 B] 2024-05-08T23:17:22.824 INFO:teuthology.orchestra.run.smithi117.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [959 kB] 2024-05-08T23:17:22.831 INFO:teuthology.orchestra.run.smithi117.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [657 kB] 2024-05-08T23:17:22.837 INFO:teuthology.orchestra.run.smithi117.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [202 kB] 2024-05-08T23:17:22.839 INFO:teuthology.orchestra.run.smithi117.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-05-08T23:17:22.840 INFO:teuthology.orchestra.run.smithi117.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,200 B] 2024-05-08T23:17:22.841 INFO:teuthology.orchestra.run.smithi117.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.0 kB] 2024-05-08T23:17:22.860 INFO:teuthology.orchestra.run.smithi117.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,904 B] 2024-05-08T23:17:22.861 INFO:teuthology.orchestra.run.smithi117.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2024-05-08T23:17:22.862 INFO:teuthology.orchestra.run.smithi081.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,296 kB] 2024-05-08T23:17:22.927 INFO:teuthology.orchestra.run.smithi117.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [972 kB] 2024-05-08T23:17:22.939 INFO:teuthology.orchestra.run.smithi117.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [520 kB] 2024-05-08T23:17:22.943 INFO:teuthology.orchestra.run.smithi117.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.2 kB] 2024-05-08T23:17:22.943 INFO:teuthology.orchestra.run.smithi117.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,916 kB] 2024-05-08T23:17:22.966 INFO:teuthology.orchestra.run.smithi117.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [37.7 kB] 2024-05-08T23:17:22.966 INFO:teuthology.orchestra.run.smithi117.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [407 kB] 2024-05-08T23:17:22.970 INFO:teuthology.orchestra.run.smithi117.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [552 B] 2024-05-08T23:17:22.970 INFO:teuthology.orchestra.run.smithi117.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [783 kB] 2024-05-08T23:17:22.976 INFO:teuthology.orchestra.run.smithi117.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,185 kB] 2024-05-08T23:17:22.988 INFO:teuthology.orchestra.run.smithi117.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [284 kB] 2024-05-08T23:17:22.991 INFO:teuthology.orchestra.run.smithi117.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.7 kB] 2024-05-08T23:17:22.991 INFO:teuthology.orchestra.run.smithi117.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,444 B] 2024-05-08T23:17:22.991 INFO:teuthology.orchestra.run.smithi117.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [26.2 kB] 2024-05-08T23:17:23.009 INFO:teuthology.orchestra.run.smithi117.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,880 B] 2024-05-08T23:17:23.010 INFO:teuthology.orchestra.run.smithi117.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2024-05-08T23:17:23.010 INFO:teuthology.orchestra.run.smithi117.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-05-08T23:17:23.010 INFO:teuthology.orchestra.run.smithi117.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-05-08T23:17:23.010 INFO:teuthology.orchestra.run.smithi117.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-05-08T23:17:23.011 INFO:teuthology.orchestra.run.smithi117.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-05-08T23:17:23.030 INFO:teuthology.orchestra.run.smithi117.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-05-08T23:17:23.031 INFO:teuthology.orchestra.run.smithi117.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-05-08T23:17:23.087 INFO:teuthology.orchestra.run.smithi081.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,920 kB] 2024-05-08T23:17:23.280 INFO:teuthology.orchestra.run.smithi081.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [438 kB] 2024-05-08T23:17:23.295 INFO:teuthology.orchestra.run.smithi081.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2024-05-08T23:17:23.295 INFO:teuthology.orchestra.run.smithi081.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,799 kB] 2024-05-08T23:17:23.344 INFO:teuthology.orchestra.run.smithi081.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [972 kB] 2024-05-08T23:17:23.361 INFO:teuthology.orchestra.run.smithi081.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [520 kB] 2024-05-08T23:17:23.370 INFO:teuthology.orchestra.run.smithi081.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [36.4 kB] 2024-05-08T23:17:23.370 INFO:teuthology.orchestra.run.smithi081.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.2 kB] 2024-05-08T23:17:23.370 INFO:teuthology.orchestra.run.smithi081.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,916 kB] 2024-05-08T23:17:23.371 INFO:teuthology.orchestra.run.smithi081.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [391 kB] 2024-05-08T23:17:23.392 INFO:teuthology.orchestra.run.smithi081.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [552 B] 2024-05-08T23:17:23.395 INFO:teuthology.orchestra.run.smithi081.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [657 kB] 2024-05-08T23:17:23.409 INFO:teuthology.orchestra.run.smithi081.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [959 kB] 2024-05-08T23:17:23.504 INFO:teuthology.orchestra.run.smithi081.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [37.7 kB] 2024-05-08T23:17:23.510 INFO:teuthology.orchestra.run.smithi081.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [407 kB] 2024-05-08T23:17:23.515 INFO:teuthology.orchestra.run.smithi081.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [552 B] 2024-05-08T23:17:23.591 INFO:teuthology.orchestra.run.smithi081.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [202 kB] 2024-05-08T23:17:23.679 INFO:teuthology.orchestra.run.smithi081.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [783 kB] 2024-05-08T23:17:24.068 INFO:teuthology.orchestra.run.smithi081.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-05-08T23:17:24.077 INFO:teuthology.orchestra.run.smithi081.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,200 B] 2024-05-08T23:17:24.080 INFO:teuthology.orchestra.run.smithi081.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.0 kB] 2024-05-08T23:17:24.103 INFO:teuthology.orchestra.run.smithi081.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,904 B] 2024-05-08T23:17:24.121 INFO:teuthology.orchestra.run.smithi081.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2024-05-08T23:17:24.276 INFO:teuthology.orchestra.run.smithi081.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,185 kB] 2024-05-08T23:17:24.386 INFO:teuthology.orchestra.run.smithi081.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [284 kB] 2024-05-08T23:17:24.407 INFO:teuthology.orchestra.run.smithi081.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.7 kB] 2024-05-08T23:17:24.408 INFO:teuthology.orchestra.run.smithi081.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [26.2 kB] 2024-05-08T23:17:24.410 INFO:teuthology.orchestra.run.smithi081.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,444 B] 2024-05-08T23:17:24.411 INFO:teuthology.orchestra.run.smithi081.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,880 B] 2024-05-08T23:17:24.411 INFO:teuthology.orchestra.run.smithi081.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2024-05-08T23:17:24.412 INFO:teuthology.orchestra.run.smithi081.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-05-08T23:17:24.414 INFO:teuthology.orchestra.run.smithi081.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-05-08T23:17:24.416 INFO:teuthology.orchestra.run.smithi081.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-05-08T23:17:24.580 INFO:teuthology.orchestra.run.smithi081.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-05-08T23:17:24.744 INFO:teuthology.orchestra.run.smithi081.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-05-08T23:17:24.795 INFO:teuthology.orchestra.run.smithi081.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-05-08T23:17:27.041 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 20.2 MB in 3s (6,854 kB/s) 2024-05-08T23:17:27.324 INFO:teuthology.orchestra.run.smithi081.stdout:Fetched 20.2 MB in 3s (5,880 kB/s) 2024-05-08T23:17:28.570 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-05-08T23:17:28.596 DEBUG:teuthology.orchestra.run.smithi117:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-08T23:17:28.680 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-05-08T23:17:28.824 INFO:teuthology.orchestra.run.smithi081.stdout:Reading package lists... 2024-05-08T23:17:28.851 DEBUG:teuthology.orchestra.run.smithi081:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-08T23:17:28.895 INFO:teuthology.orchestra.run.smithi117.stdout:Building dependency tree... 2024-05-08T23:17:28.895 INFO:teuthology.orchestra.run.smithi117.stdout:Reading state information... 2024-05-08T23:17:28.920 INFO:teuthology.orchestra.run.smithi081.stdout:Reading package lists... 2024-05-08T23:17:29.092 INFO:teuthology.orchestra.run.smithi117.stdout:The following additional packages will be installed: 2024-05-08T23:17:29.092 INFO:teuthology.orchestra.run.smithi117.stdout: linux-generic linux-headers-5.4.0-181 linux-headers-5.4.0-181-generic 2024-05-08T23:17:29.092 INFO:teuthology.orchestra.run.smithi117.stdout: linux-headers-generic linux-image-5.4.0-181-generic 2024-05-08T23:17:29.092 INFO:teuthology.orchestra.run.smithi117.stdout: linux-modules-5.4.0-181-generic linux-modules-extra-5.4.0-181-generic 2024-05-08T23:17:29.093 INFO:teuthology.orchestra.run.smithi117.stdout:Suggested packages: 2024-05-08T23:17:29.093 INFO:teuthology.orchestra.run.smithi117.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-05-08T23:17:29.094 INFO:teuthology.orchestra.run.smithi117.stdout:Recommended packages: 2024-05-08T23:17:29.094 INFO:teuthology.orchestra.run.smithi117.stdout: thermald 2024-05-08T23:17:29.131 INFO:teuthology.orchestra.run.smithi081.stdout:Building dependency tree... 2024-05-08T23:17:29.131 INFO:teuthology.orchestra.run.smithi081.stdout:Reading state information... 2024-05-08T23:17:29.161 INFO:teuthology.orchestra.run.smithi117.stdout:The following NEW packages will be installed: 2024-05-08T23:17:29.161 INFO:teuthology.orchestra.run.smithi117.stdout: linux-headers-5.4.0-181 linux-headers-5.4.0-181-generic 2024-05-08T23:17:29.161 INFO:teuthology.orchestra.run.smithi117.stdout: linux-image-5.4.0-181-generic linux-modules-5.4.0-181-generic 2024-05-08T23:17:29.161 INFO:teuthology.orchestra.run.smithi117.stdout: linux-modules-extra-5.4.0-181-generic 2024-05-08T23:17:29.163 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages will be upgraded: 2024-05-08T23:17:29.163 INFO:teuthology.orchestra.run.smithi117.stdout: linux-generic linux-headers-generic linux-image-generic 2024-05-08T23:17:29.329 INFO:teuthology.orchestra.run.smithi081.stdout:The following additional packages will be installed: 2024-05-08T23:17:29.329 INFO:teuthology.orchestra.run.smithi081.stdout: linux-generic linux-headers-5.4.0-181 linux-headers-5.4.0-181-generic 2024-05-08T23:17:29.329 INFO:teuthology.orchestra.run.smithi081.stdout: linux-headers-generic linux-image-5.4.0-181-generic 2024-05-08T23:17:29.330 INFO:teuthology.orchestra.run.smithi081.stdout: linux-modules-5.4.0-181-generic linux-modules-extra-5.4.0-181-generic 2024-05-08T23:17:29.330 INFO:teuthology.orchestra.run.smithi081.stdout:Suggested packages: 2024-05-08T23:17:29.330 INFO:teuthology.orchestra.run.smithi081.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-05-08T23:17:29.330 INFO:teuthology.orchestra.run.smithi081.stdout:Recommended packages: 2024-05-08T23:17:29.330 INFO:teuthology.orchestra.run.smithi081.stdout: thermald 2024-05-08T23:17:29.344 INFO:teuthology.orchestra.run.smithi117.stdout:3 upgraded, 5 newly installed, 0 to remove and 299 not upgraded. 2024-05-08T23:17:29.344 INFO:teuthology.orchestra.run.smithi117.stdout:Need to get 77.1 MB of archives. 2024-05-08T23:17:29.344 INFO:teuthology.orchestra.run.smithi117.stdout:After this operation, 380 MB of additional disk space will be used. 2024-05-08T23:17:29.344 INFO:teuthology.orchestra.run.smithi117.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-181-generic amd64 5.4.0-181.201 [15.0 MB] 2024-05-08T23:17:29.384 INFO:teuthology.orchestra.run.smithi081.stdout:The following NEW packages will be installed: 2024-05-08T23:17:29.384 INFO:teuthology.orchestra.run.smithi081.stdout: linux-headers-5.4.0-181 linux-headers-5.4.0-181-generic 2024-05-08T23:17:29.384 INFO:teuthology.orchestra.run.smithi081.stdout: linux-image-5.4.0-181-generic linux-modules-5.4.0-181-generic 2024-05-08T23:17:29.384 INFO:teuthology.orchestra.run.smithi081.stdout: linux-modules-extra-5.4.0-181-generic 2024-05-08T23:17:29.385 INFO:teuthology.orchestra.run.smithi081.stdout:The following packages will be upgraded: 2024-05-08T23:17:29.386 INFO:teuthology.orchestra.run.smithi081.stdout: linux-generic linux-headers-generic linux-image-generic 2024-05-08T23:17:29.568 INFO:teuthology.orchestra.run.smithi081.stdout:3 upgraded, 5 newly installed, 0 to remove and 299 not upgraded. 2024-05-08T23:17:29.568 INFO:teuthology.orchestra.run.smithi081.stdout:Need to get 77.1 MB of archives. 2024-05-08T23:17:29.568 INFO:teuthology.orchestra.run.smithi081.stdout:After this operation, 380 MB of additional disk space will be used. 2024-05-08T23:17:29.568 INFO:teuthology.orchestra.run.smithi081.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-181-generic amd64 5.4.0-181.201 [15.0 MB] 2024-05-08T23:17:30.526 INFO:teuthology.orchestra.run.smithi117.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-181-generic amd64 5.4.0-181.201 [10.5 MB] 2024-05-08T23:17:30.748 INFO:teuthology.orchestra.run.smithi081.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-181-generic amd64 5.4.0-181.201 [10.5 MB] 2024-05-08T23:17:30.798 INFO:teuthology.orchestra.run.smithi117.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-181-generic amd64 5.4.0-181.201 [39.2 MB] 2024-05-08T23:17:31.078 INFO:teuthology.orchestra.run.smithi081.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-181-generic amd64 5.4.0-181.201 [39.2 MB] 2024-05-08T23:17:32.061 INFO:teuthology.orchestra.run.smithi117.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.181.179 [1,896 B] 2024-05-08T23:17:32.063 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.181.179 [2,596 B] 2024-05-08T23:17:32.063 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-181 all 5.4.0-181.201 [11.0 MB] 2024-05-08T23:17:32.352 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-181-generic amd64 5.4.0-181.201 [1,383 kB] 2024-05-08T23:17:32.389 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.181.179 [2,464 B] 2024-05-08T23:17:32.395 INFO:teuthology.orchestra.run.smithi081.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.181.179 [1,896 B] 2024-05-08T23:17:32.395 INFO:teuthology.orchestra.run.smithi081.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.181.179 [2,596 B] 2024-05-08T23:17:32.395 INFO:teuthology.orchestra.run.smithi081.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-181 all 5.4.0-181.201 [11.0 MB] 2024-05-08T23:17:32.675 INFO:teuthology.orchestra.run.smithi081.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-181-generic amd64 5.4.0-181.201 [1,383 kB] 2024-05-08T23:17:32.718 INFO:teuthology.orchestra.run.smithi081.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.181.179 [2,464 B] 2024-05-08T23:17:33.111 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 77.1 MB in 3s (24.0 MB/s) 2024-05-08T23:17:33.290 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-modules-5.4.0-181-generic. 2024-05-08T23:17:33.395 INFO:teuthology.orchestra.run.smithi081.stdout:Fetched 77.1 MB in 3s (23.2 MB/s) 2024-05-08T23:17:33.597 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package linux-modules-5.4.0-181-generic. 2024-05-08T23:17:35.009 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-05-08T23:17:35.014 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../0-linux-modules-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T23:17:35.120 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-modules-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:35.363 INFO:teuthology.orchestra.run.smithi081.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-05-08T23:17:35.368 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../0-linux-modules-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T23:17:35.478 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking linux-modules-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:37.019 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-image-5.4.0-181-generic. 2024-05-08T23:17:37.032 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../1-linux-image-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T23:17:37.127 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-image-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:37.412 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package linux-image-5.4.0-181-generic. 2024-05-08T23:17:37.429 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../1-linux-image-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T23:17:37.529 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking linux-image-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:38.180 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-181-generic. 2024-05-08T23:17:38.197 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T23:17:38.238 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-modules-extra-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:38.640 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-181-generic. 2024-05-08T23:17:38.653 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T23:17:38.690 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking linux-modules-extra-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:43.182 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../3-linux-generic_5.4.0.181.179_amd64.deb ... 2024-05-08T23:17:43.308 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-generic (5.4.0.181.179) over (5.4.0.124.125) ... 2024-05-08T23:17:43.578 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../3-linux-generic_5.4.0.181.179_amd64.deb ... 2024-05-08T23:17:43.644 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.181.179_amd64.deb ... 2024-05-08T23:17:43.810 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking linux-generic (5.4.0.181.179) over (5.4.0.124.125) ... 2024-05-08T23:17:43.810 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-image-generic (5.4.0.181.179) over (5.4.0.124.125) ... 2024-05-08T23:17:44.054 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.181.179_amd64.deb ... 2024-05-08T23:17:44.114 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-headers-5.4.0-181. 2024-05-08T23:17:44.127 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../5-linux-headers-5.4.0-181_5.4.0-181.201_all.deb ... 2024-05-08T23:17:44.172 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-headers-5.4.0-181 (5.4.0-181.201) ... 2024-05-08T23:17:44.506 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking linux-image-generic (5.4.0.181.179) over (5.4.0.124.125) ... 2024-05-08T23:17:44.853 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package linux-headers-5.4.0-181. 2024-05-08T23:17:44.868 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../5-linux-headers-5.4.0-181_5.4.0-181.201_all.deb ... 2024-05-08T23:17:44.911 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking linux-headers-5.4.0-181 (5.4.0-181.201) ... 2024-05-08T23:17:49.856 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-headers-5.4.0-181-generic. 2024-05-08T23:17:49.877 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../6-linux-headers-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T23:17:49.914 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-headers-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:50.512 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package linux-headers-5.4.0-181-generic. 2024-05-08T23:17:50.534 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../6-linux-headers-5.4.0-181-generic_5.4.0-181.201_amd64.deb ... 2024-05-08T23:17:50.603 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking linux-headers-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:52.740 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.181.179_amd64.deb ... 2024-05-08T23:17:52.868 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-headers-generic (5.4.0.181.179) over (5.4.0.124.125) ... 2024-05-08T23:17:53.217 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-modules-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:53.328 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.181.179_amd64.deb ... 2024-05-08T23:17:53.475 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking linux-headers-generic (5.4.0.181.179) over (5.4.0.124.125) ... 2024-05-08T23:17:53.798 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up linux-modules-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:54.424 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-image-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:55.576 INFO:teuthology.orchestra.run.smithi117.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-181-generic 2024-05-08T23:17:55.576 INFO:teuthology.orchestra.run.smithi117.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-181-generic 2024-05-08T23:17:55.734 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-headers-5.4.0-181 (5.4.0-181.201) ... 2024-05-08T23:17:55.803 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up linux-image-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:55.860 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-headers-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:55.986 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-modules-extra-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:57.007 INFO:teuthology.orchestra.run.smithi081.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-181-generic 2024-05-08T23:17:57.007 INFO:teuthology.orchestra.run.smithi081.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-181-generic 2024-05-08T23:17:57.151 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up linux-headers-5.4.0-181 (5.4.0-181.201) ... 2024-05-08T23:17:57.196 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-headers-generic (5.4.0.181.179) ... 2024-05-08T23:17:57.302 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up linux-headers-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:57.428 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up linux-modules-extra-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:58.269 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-image-generic (5.4.0.181.179) ... 2024-05-08T23:17:58.629 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up linux-headers-generic (5.4.0.181.179) ... 2024-05-08T23:17:58.764 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up linux-image-generic (5.4.0.181.179) ... 2024-05-08T23:17:58.915 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up linux-generic (5.4.0.181.179) ... 2024-05-08T23:17:59.067 INFO:teuthology.orchestra.run.smithi081.stdout:Processing triggers for linux-image-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:17:59.136 INFO:teuthology.orchestra.run.smithi081.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-05-08T23:17:59.136 INFO:teuthology.orchestra.run.smithi081.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-181-generic 2024-05-08T23:17:59.732 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-generic (5.4.0.181.179) ... 2024-05-08T23:18:00.100 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for linux-image-5.4.0-181-generic (5.4.0-181.201) ... 2024-05-08T23:18:00.186 INFO:teuthology.orchestra.run.smithi117.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-05-08T23:18:00.186 INFO:teuthology.orchestra.run.smithi117.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-181-generic 2024-05-08T23:18:21.077 INFO:teuthology.orchestra.run.smithi081.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-05-08T23:18:21.077 INFO:teuthology.orchestra.run.smithi081.stdout:Sourcing file `/etc/default/grub' 2024-05-08T23:18:21.106 INFO:teuthology.orchestra.run.smithi081.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-08T23:18:21.125 INFO:teuthology.orchestra.run.smithi081.stdout:Generating grub configuration file ... 2024-05-08T23:18:21.496 INFO:teuthology.orchestra.run.smithi081.stdout:Found linux image: /boot/vmlinuz-5.4.0-181-generic 2024-05-08T23:18:21.519 INFO:teuthology.orchestra.run.smithi081.stdout:Found initrd image: /boot/initrd.img-5.4.0-181-generic 2024-05-08T23:18:21.820 INFO:teuthology.orchestra.run.smithi117.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-05-08T23:18:21.820 INFO:teuthology.orchestra.run.smithi117.stdout:Sourcing file `/etc/default/grub' 2024-05-08T23:18:21.838 INFO:teuthology.orchestra.run.smithi117.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-08T23:18:21.862 INFO:teuthology.orchestra.run.smithi117.stdout:Generating grub configuration file ... 2024-05-08T23:18:21.971 INFO:teuthology.orchestra.run.smithi081.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-05-08T23:18:21.982 INFO:teuthology.orchestra.run.smithi081.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-05-08T23:18:22.190 INFO:teuthology.orchestra.run.smithi117.stdout:Found linux image: /boot/vmlinuz-5.4.0-181-generic 2024-05-08T23:18:22.219 INFO:teuthology.orchestra.run.smithi117.stdout:Found initrd image: /boot/initrd.img-5.4.0-181-generic 2024-05-08T23:18:22.615 INFO:teuthology.orchestra.run.smithi117.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-05-08T23:18:22.622 INFO:teuthology.orchestra.run.smithi117.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-05-08T23:18:24.712 INFO:teuthology.orchestra.run.smithi081.stdout:done 2024-05-08T23:18:25.119 DEBUG:teuthology.orchestra.run.smithi081:> dpkg -s linux-image-generic 2024-05-08T23:18:25.173 INFO:teuthology.orchestra.run.smithi081.stdout:Package: linux-image-generic 2024-05-08T23:18:25.173 INFO:teuthology.orchestra.run.smithi081.stdout:Status: install ok installed 2024-05-08T23:18:25.173 INFO:teuthology.orchestra.run.smithi081.stdout:Priority: optional 2024-05-08T23:18:25.173 INFO:teuthology.orchestra.run.smithi081.stdout:Section: kernel 2024-05-08T23:18:25.173 INFO:teuthology.orchestra.run.smithi081.stdout:Installed-Size: 20 2024-05-08T23:18:25.173 INFO:teuthology.orchestra.run.smithi081.stdout:Maintainer: Ubuntu Kernel Team 2024-05-08T23:18:25.174 INFO:teuthology.orchestra.run.smithi081.stdout:Architecture: amd64 2024-05-08T23:18:25.174 INFO:teuthology.orchestra.run.smithi081.stdout:Source: linux-meta 2024-05-08T23:18:25.174 INFO:teuthology.orchestra.run.smithi081.stdout:Version: 5.4.0.181.179 2024-05-08T23:18:25.174 INFO:teuthology.orchestra.run.smithi081.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-05-08T23:18:25.174 INFO:teuthology.orchestra.run.smithi081.stdout:Depends: linux-image-5.4.0-181-generic, linux-modules-extra-5.4.0-181-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-08T23:18:25.174 INFO:teuthology.orchestra.run.smithi081.stdout:Recommends: thermald 2024-05-08T23:18:25.174 INFO:teuthology.orchestra.run.smithi081.stdout:Description: Generic Linux kernel image 2024-05-08T23:18:25.174 INFO:teuthology.orchestra.run.smithi081.stdout: This package will always depend on the latest generic kernel image 2024-05-08T23:18:25.174 INFO:teuthology.orchestra.run.smithi081.stdout: available. 2024-05-08T23:18:25.175 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-181-generic 2024-05-08T23:18:25.175 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T23:18:25.175 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T23:18:25.175 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-08T23:18:25.175 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi081.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T23:18:25.175 DEBUG:teuthology.orchestra.run.smithi081:> sudo apt-get clean 2024-05-08T23:18:25.251 DEBUG:teuthology.orchestra.run.smithi081:> sudo apt-get update 2024-05-08T23:18:25.322 INFO:teuthology.orchestra.run.smithi117.stdout:done 2024-05-08T23:18:25.387 INFO:teuthology.orchestra.run.smithi081.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-05-08T23:18:25.500 INFO:teuthology.orchestra.run.smithi081.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-05-08T23:18:25.582 INFO:teuthology.orchestra.run.smithi081.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-05-08T23:18:25.664 INFO:teuthology.orchestra.run.smithi081.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-05-08T23:18:25.746 DEBUG:teuthology.orchestra.run.smithi117:> dpkg -s linux-image-generic 2024-05-08T23:18:25.794 INFO:teuthology.orchestra.run.smithi117.stdout:Package: linux-image-generic 2024-05-08T23:18:25.794 INFO:teuthology.orchestra.run.smithi117.stdout:Status: install ok installed 2024-05-08T23:18:25.794 INFO:teuthology.orchestra.run.smithi117.stdout:Priority: optional 2024-05-08T23:18:25.794 INFO:teuthology.orchestra.run.smithi117.stdout:Section: kernel 2024-05-08T23:18:25.795 INFO:teuthology.orchestra.run.smithi117.stdout:Installed-Size: 20 2024-05-08T23:18:25.795 INFO:teuthology.orchestra.run.smithi117.stdout:Maintainer: Ubuntu Kernel Team 2024-05-08T23:18:25.795 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: amd64 2024-05-08T23:18:25.795 INFO:teuthology.orchestra.run.smithi117.stdout:Source: linux-meta 2024-05-08T23:18:25.795 INFO:teuthology.orchestra.run.smithi117.stdout:Version: 5.4.0.181.179 2024-05-08T23:18:25.795 INFO:teuthology.orchestra.run.smithi117.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-05-08T23:18:25.795 INFO:teuthology.orchestra.run.smithi117.stdout:Depends: linux-image-5.4.0-181-generic, linux-modules-extra-5.4.0-181-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-08T23:18:25.795 INFO:teuthology.orchestra.run.smithi117.stdout:Recommends: thermald 2024-05-08T23:18:25.795 INFO:teuthology.orchestra.run.smithi117.stdout:Description: Generic Linux kernel image 2024-05-08T23:18:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: This package will always depend on the latest generic kernel image 2024-05-08T23:18:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: available. 2024-05-08T23:18:25.795 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-181-generic 2024-05-08T23:18:25.796 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T23:18:25.796 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T23:18:25.796 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-08T23:18:25.796 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi117.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T23:18:25.796 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get clean 2024-05-08T23:18:25.887 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get update 2024-05-08T23:18:26.026 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-05-08T23:18:26.035 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-05-08T23:18:26.054 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-05-08T23:18:26.134 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-05-08T23:18:27.174 INFO:teuthology.orchestra.run.smithi081.stdout:Reading package lists... 2024-05-08T23:18:27.201 DEBUG:teuthology.orchestra.run.smithi081:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-08T23:18:27.270 INFO:teuthology.orchestra.run.smithi081.stdout:Reading package lists... 2024-05-08T23:18:27.481 INFO:teuthology.orchestra.run.smithi081.stdout:Building dependency tree... 2024-05-08T23:18:27.481 INFO:teuthology.orchestra.run.smithi081.stdout:Reading state information... 2024-05-08T23:18:27.712 INFO:teuthology.orchestra.run.smithi081.stdout:linux-image-generic is already the newest version (5.4.0.181.179). 2024-05-08T23:18:27.712 INFO:teuthology.orchestra.run.smithi081.stdout:0 upgraded, 0 newly installed, 0 to remove and 299 not upgraded. 2024-05-08T23:18:27.714 DEBUG:teuthology.orchestra.run.smithi081:> dpkg -s linux-image-generic 2024-05-08T23:18:27.735 INFO:teuthology.orchestra.run.smithi081.stdout:Package: linux-image-generic 2024-05-08T23:18:27.735 INFO:teuthology.orchestra.run.smithi081.stdout:Status: install ok installed 2024-05-08T23:18:27.735 INFO:teuthology.orchestra.run.smithi081.stdout:Priority: optional 2024-05-08T23:18:27.735 INFO:teuthology.orchestra.run.smithi081.stdout:Section: kernel 2024-05-08T23:18:27.735 INFO:teuthology.orchestra.run.smithi081.stdout:Installed-Size: 20 2024-05-08T23:18:27.735 INFO:teuthology.orchestra.run.smithi081.stdout:Maintainer: Ubuntu Kernel Team 2024-05-08T23:18:27.735 INFO:teuthology.orchestra.run.smithi081.stdout:Architecture: amd64 2024-05-08T23:18:27.735 INFO:teuthology.orchestra.run.smithi081.stdout:Source: linux-meta 2024-05-08T23:18:27.735 INFO:teuthology.orchestra.run.smithi081.stdout:Version: 5.4.0.181.179 2024-05-08T23:18:27.735 INFO:teuthology.orchestra.run.smithi081.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-05-08T23:18:27.736 INFO:teuthology.orchestra.run.smithi081.stdout:Depends: linux-image-5.4.0-181-generic, linux-modules-extra-5.4.0-181-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-08T23:18:27.736 INFO:teuthology.orchestra.run.smithi081.stdout:Recommends: thermald 2024-05-08T23:18:27.736 INFO:teuthology.orchestra.run.smithi081.stdout:Description: Generic Linux kernel image 2024-05-08T23:18:27.736 INFO:teuthology.orchestra.run.smithi081.stdout: This package will always depend on the latest generic kernel image 2024-05-08T23:18:27.736 INFO:teuthology.orchestra.run.smithi081.stdout: available. 2024-05-08T23:18:27.736 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-05-08T23:18:27.740 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.7oWhBVUrD1 2024-05-08T23:18:27.741 DEBUG:teuthology.orchestra.run.smithi081:> sudo cp /boot/grub/grub.cfg /tmp/tmp.7oWhBVUrD1 2024-05-08T23:18:27.763 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.7oWhBVUrD1 2024-05-08T23:18:27.839 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-05-08T23:18:27.848 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.7oWhBVUrD1 is 9KB 2024-05-08T23:18:27.861 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.7oWhBVUrD1 2024-05-08T23:18:27.865 DEBUG:teuthology.orchestra.run.smithi117:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-08T23:18:27.868 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-05-08T23:18:27.924 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:18:27.924 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-05-08T23:18:27.924 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-05-08T23:18:27.932 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-05-08T23:18:27.956 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-181-generic 2024-05-08T23:18:27.956 DEBUG:teuthology.orchestra.run.smithi081:> sudo update-grub 2024-05-08T23:18:28.144 INFO:teuthology.orchestra.run.smithi117.stdout:Building dependency tree... 2024-05-08T23:18:28.145 INFO:teuthology.orchestra.run.smithi117.stdout:Reading state information... 2024-05-08T23:18:28.373 INFO:teuthology.orchestra.run.smithi117.stdout:linux-image-generic is already the newest version (5.4.0.181.179). 2024-05-08T23:18:28.373 INFO:teuthology.orchestra.run.smithi117.stdout:0 upgraded, 0 newly installed, 0 to remove and 299 not upgraded. 2024-05-08T23:18:28.375 DEBUG:teuthology.orchestra.run.smithi117:> dpkg -s linux-image-generic 2024-05-08T23:18:28.396 INFO:teuthology.orchestra.run.smithi117.stdout:Package: linux-image-generic 2024-05-08T23:18:28.396 INFO:teuthology.orchestra.run.smithi117.stdout:Status: install ok installed 2024-05-08T23:18:28.397 INFO:teuthology.orchestra.run.smithi117.stdout:Priority: optional 2024-05-08T23:18:28.397 INFO:teuthology.orchestra.run.smithi117.stdout:Section: kernel 2024-05-08T23:18:28.397 INFO:teuthology.orchestra.run.smithi117.stdout:Installed-Size: 20 2024-05-08T23:18:28.397 INFO:teuthology.orchestra.run.smithi117.stdout:Maintainer: Ubuntu Kernel Team 2024-05-08T23:18:28.397 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: amd64 2024-05-08T23:18:28.397 INFO:teuthology.orchestra.run.smithi117.stdout:Source: linux-meta 2024-05-08T23:18:28.397 INFO:teuthology.orchestra.run.smithi117.stdout:Version: 5.4.0.181.179 2024-05-08T23:18:28.397 INFO:teuthology.orchestra.run.smithi117.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-05-08T23:18:28.397 INFO:teuthology.orchestra.run.smithi117.stdout:Depends: linux-image-5.4.0-181-generic, linux-modules-extra-5.4.0-181-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-08T23:18:28.397 INFO:teuthology.orchestra.run.smithi117.stdout:Recommends: thermald 2024-05-08T23:18:28.397 INFO:teuthology.orchestra.run.smithi117.stdout:Description: Generic Linux kernel image 2024-05-08T23:18:28.397 INFO:teuthology.orchestra.run.smithi117.stdout: This package will always depend on the latest generic kernel image 2024-05-08T23:18:28.397 INFO:teuthology.orchestra.run.smithi117.stdout: available. 2024-05-08T23:18:28.398 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-05-08T23:18:28.401 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.madJLTCqTh 2024-05-08T23:18:28.402 DEBUG:teuthology.orchestra.run.smithi117:> sudo cp /boot/grub/grub.cfg /tmp/tmp.madJLTCqTh 2024-05-08T23:18:28.458 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.madJLTCqTh 2024-05-08T23:18:28.496 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.madJLTCqTh is 9KB 2024-05-08T23:18:28.510 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.madJLTCqTh 2024-05-08T23:18:28.516 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-05-08T23:18:28.574 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:18:28.574 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-05-08T23:18:28.574 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-05-08T23:18:28.619 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-181-generic 2024-05-08T23:18:28.620 DEBUG:teuthology.orchestra.run.smithi117:> sudo update-grub 2024-05-08T23:18:29.274 INFO:teuthology.orchestra.run.smithi081.stderr:Sourcing file `/etc/default/grub' 2024-05-08T23:18:29.279 INFO:teuthology.orchestra.run.smithi081.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-08T23:18:29.290 INFO:teuthology.orchestra.run.smithi081.stderr:Generating grub configuration file ... 2024-05-08T23:18:29.586 INFO:teuthology.orchestra.run.smithi081.stderr:Found linux image: /boot/vmlinuz-5.4.0-181-generic 2024-05-08T23:18:29.598 INFO:teuthology.orchestra.run.smithi081.stderr:Found initrd image: /boot/initrd.img-5.4.0-181-generic 2024-05-08T23:18:29.996 INFO:teuthology.orchestra.run.smithi117.stderr:Sourcing file `/etc/default/grub' 2024-05-08T23:18:29.997 INFO:teuthology.orchestra.run.smithi081.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-05-08T23:18:30.003 INFO:teuthology.orchestra.run.smithi117.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-08T23:18:30.008 INFO:teuthology.orchestra.run.smithi081.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-05-08T23:18:30.021 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2024-05-08T23:18:30.298 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-5.4.0-181-generic 2024-05-08T23:18:30.309 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-5.4.0-181-generic 2024-05-08T23:18:30.704 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-05-08T23:18:30.716 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-05-08T23:18:31.658 INFO:teuthology.orchestra.run.smithi081.stderr:done 2024-05-08T23:18:31.659 DEBUG:teuthology.orchestra.run.smithi081:> sudo shutdown -r now 2024-05-08T23:18:32.370 INFO:teuthology.orchestra.run.smithi117.stderr:done 2024-05-08T23:18:32.371 DEBUG:teuthology.orchestra.run.smithi117:> sudo shutdown -r now 2024-05-08T23:19:01.677 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-08T23:19:01.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-08T23:19:01.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:19:02.375 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-08T23:19:02.375 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-05-08T23:19:02.376 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:19:36.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-05-08T23:19:45.238 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-08T23:19:45.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:20:02.390 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T23:20:03.614 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-05-08T23:20:11.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-05-08T23:20:11.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:20:11.788 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-05-08T23:20:12.693 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-05-08T23:20:12.693 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-181-generic"... 2024-05-08T23:20:12.694 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-05-08T23:20:12.700 INFO:teuthology.orchestra.run.smithi117.stdout:5.4.0-181-generic 2024-05-08T23:20:12.700 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-181-generic vs 5.4.0-181-generic 2024-05-08T23:20:12.700 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T23:20:12.700 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-08T23:20:13.701 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-08T23:20:13.701 DEBUG:teuthology.orchestra.run.smithi117:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T23:20:13.829 INFO:teuthology.orchestra.run.smithi117.stdout:ttyS1 2024-05-08T23:20:13.865 DEBUG:teuthology.parallel:result is None 2024-05-08T23:20:15.615 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-08T23:20:15.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:20:15.617 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-05-08T23:20:30.618 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-08T23:20:30.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:20:30.870 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-05-08T23:20:31.454 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-08T23:20:31.454 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-181-generic"... 2024-05-08T23:20:31.455 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-05-08T23:20:31.461 INFO:teuthology.orchestra.run.smithi081.stdout:5.4.0-181-generic 2024-05-08T23:20:31.461 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-181-generic vs 5.4.0-181-generic 2024-05-08T23:20:31.461 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T23:20:31.461 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-08T23:20:32.463 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-08T23:20:32.463 DEBUG:teuthology.orchestra.run.smithi081:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T23:20:32.564 INFO:teuthology.orchestra.run.smithi081.stdout:ttyS1 2024-05-08T23:20:32.583 DEBUG:teuthology.parallel:result is None 2024-05-08T23:20:32.583 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T23:20:32.590 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T23:20:32.590 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T23:20:32.593 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T23:20:32.600 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T23:20:32.649 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T23:20:32.716 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T23:20:32.717 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T23:20:32.719 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T23:20:32.745 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T23:20:32.792 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T23:20:32.793 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T23:20:32.795 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T23:20:32.810 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:20:32.816 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:20:32.822 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:20:32.827 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:20:32.829 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T23:20:32.866 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T23:20:32.866 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T23:20:32.868 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T23:20:32.884 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T23:20:32.943 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T23:20:32.943 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T23:20:32.946 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T23:20:32.953 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:20:32.959 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:20:33.007 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:20:33.008 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T23:20:33.039 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:20:33.047 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:20:33.094 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:20:33.094 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T23:20:33.115 DEBUG:teuthology.orchestra.run.smithi081:> sudo service rsyslog restart 2024-05-08T23:20:33.118 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-05-08T23:20:33.165 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T23:20:33.173 INFO:teuthology.task.internal:Starting timer... 2024-05-08T23:20:33.173 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T23:20:33.204 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T23:20:33.234 DEBUG:teuthology.task.selinux:Excluding smithi081: OS 'ubuntu' does not support SELinux 2024-05-08T23:20:33.234 DEBUG:teuthology.task.selinux:Excluding smithi117: OS 'ubuntu' does not support SELinux 2024-05-08T23:20:33.234 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T23:20:33.235 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-05-08T23:20:33.235 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T23:20:33.235 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T23:20:33.268 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-08T23:20:33.273 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-08T23:20:33.482 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T23:20:33.498 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T23:20:33.500 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi081.front.sepia.ceph.com,smithi117.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T23:26:04.358 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi081.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com')] 2024-05-08T23:26:04.360 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-08T23:26:04.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:26:04.448 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-05-08T23:26:04.579 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-08T23:26:04.580 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-05-08T23:26:04.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:26:04.666 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-05-08T23:26:04.787 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-05-08T23:26:04.788 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T23:26:04.801 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T23:26:04.802 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T23:26:04.802 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:26:04.804 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T23:26:04.805 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:26:04.839 INFO:teuthology.orchestra.run.smithi081.stdout: 8 May 23:26:04 ntpd[13005]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-05-08T23:26:04.839 INFO:teuthology.orchestra.run.smithi081.stdout: 8 May 23:26:04 ntpd[13005]: Command line: ntpd -gq 2024-05-08T23:26:04.840 INFO:teuthology.orchestra.run.smithi081.stdout: 8 May 23:26:04 ntpd[13005]: proto: precision = 0.108 usec (-23) 2024-05-08T23:26:04.841 INFO:teuthology.orchestra.run.smithi081.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-08T23:26:04.841 INFO:teuthology.orchestra.run.smithi117.stdout: 8 May 23:26:04 ntpd[13048]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-05-08T23:26:04.841 INFO:teuthology.orchestra.run.smithi117.stdout: 8 May 23:26:04 ntpd[13048]: Command line: ntpd -gq 2024-05-08T23:26:04.841 INFO:teuthology.orchestra.run.smithi081.stdout: 8 May 23:26:04 ntpd[13005]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-08T23:26:04.842 INFO:teuthology.orchestra.run.smithi081.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-08T23:26:04.842 INFO:teuthology.orchestra.run.smithi081.stdout: 8 May 23:26:04 ntpd[13005]: restrict ::: KOD does nothing without LIMITED. 2024-05-08T23:26:04.843 INFO:teuthology.orchestra.run.smithi081.stdout: 8 May 23:26:04 ntpd[13005]: Listen and drop on 0 v6wildcard [::]:123 2024-05-08T23:26:04.843 INFO:teuthology.orchestra.run.smithi081.stdout: 8 May 23:26:04 ntpd[13005]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-08T23:26:04.843 INFO:teuthology.orchestra.run.smithi081.stdout: 8 May 23:26:04 ntpd[13005]: Listen normally on 2 lo 127.0.0.1:123 2024-05-08T23:26:04.843 INFO:teuthology.orchestra.run.smithi081.stdout: 8 May 23:26:04 ntpd[13005]: Listen normally on 3 enp3s0f1 172.21.15.81:123 2024-05-08T23:26:04.843 INFO:teuthology.orchestra.run.smithi081.stdout: 8 May 23:26:04 ntpd[13005]: Listen normally on 4 lo [::1]:123 2024-05-08T23:26:04.843 INFO:teuthology.orchestra.run.smithi081.stdout: 8 May 23:26:04 ntpd[13005]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d753%5]:123 2024-05-08T23:26:04.843 INFO:teuthology.orchestra.run.smithi081.stdout: 8 May 23:26:04 ntpd[13005]: Listening on routing socket on fd #22 for interface updates 2024-05-08T23:26:04.843 INFO:teuthology.orchestra.run.smithi117.stdout: 8 May 23:26:04 ntpd[13048]: proto: precision = 0.116 usec (-23) 2024-05-08T23:26:04.844 INFO:teuthology.orchestra.run.smithi117.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-08T23:26:04.844 INFO:teuthology.orchestra.run.smithi117.stdout: 8 May 23:26:04 ntpd[13048]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-08T23:26:04.844 INFO:teuthology.orchestra.run.smithi117.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-08T23:26:04.845 INFO:teuthology.orchestra.run.smithi117.stdout: 8 May 23:26:04 ntpd[13048]: restrict ::: KOD does nothing without LIMITED. 2024-05-08T23:26:04.845 INFO:teuthology.orchestra.run.smithi117.stdout: 8 May 23:26:04 ntpd[13048]: Listen and drop on 0 v6wildcard [::]:123 2024-05-08T23:26:04.845 INFO:teuthology.orchestra.run.smithi117.stdout: 8 May 23:26:04 ntpd[13048]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-08T23:26:04.845 INFO:teuthology.orchestra.run.smithi117.stdout: 8 May 23:26:04 ntpd[13048]: Listen normally on 2 lo 127.0.0.1:123 2024-05-08T23:26:04.846 INFO:teuthology.orchestra.run.smithi117.stdout: 8 May 23:26:04 ntpd[13048]: Listen normally on 3 enp3s0f1 172.21.15.117:123 2024-05-08T23:26:04.846 INFO:teuthology.orchestra.run.smithi117.stdout: 8 May 23:26:04 ntpd[13048]: Listen normally on 4 lo [::1]:123 2024-05-08T23:26:04.846 INFO:teuthology.orchestra.run.smithi117.stdout: 8 May 23:26:04 ntpd[13048]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:4463%5]:123 2024-05-08T23:26:04.846 INFO:teuthology.orchestra.run.smithi117.stdout: 8 May 23:26:04 ntpd[13048]: Listening on routing socket on fd #22 for interface updates 2024-05-08T23:26:05.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:05 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:05.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:05 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:05.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:05 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:05.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:05 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:05.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:05 ntpd[13005]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:05.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:05 ntpd[13005]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:05.841 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:05 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:05.841 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:05 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:06.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:06 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:06.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:06 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:06.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:06 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:06.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:06 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:06.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:06 ntpd[13005]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:06.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:06 ntpd[13005]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:07.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:07 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:07.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:07 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:07.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:07 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:07.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:07 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:07.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:07 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:07.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:07 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:07.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:07 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:07.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:07 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:07.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:07 ntpd[13005]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:07.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:07 ntpd[13005]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:07.842 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:07 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:07.842 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:07 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:07.842 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:07 ntpd[13005]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:07.842 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:07 ntpd[13005]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:07.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:07 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:07.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:07 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:07.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:07 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:07.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:07 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:07.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:07 ntpd[13048]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:07.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:07 ntpd[13048]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:08.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:08 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:08.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:08 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:08.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:08 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:08.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:08 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:08.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:08 ntpd[13005]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:08.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:08 ntpd[13005]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:08.842 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:08 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:08.842 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:08 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:08.842 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:08 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:08.842 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:08 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:08.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:08 ntpd[13048]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:08.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:08 ntpd[13048]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:09.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:09 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:09.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:09 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:09.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:09 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:09.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:09 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:09.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:09 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:09.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:09 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:09.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:09 ntpd[13005]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:09.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:09 ntpd[13005]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:09.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:09 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:09.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:09 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:09.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:09 ntpd[13005]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:09.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:09 ntpd[13005]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:09.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:09 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:09.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:09 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:09.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:09 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:09.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:09 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:09.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:09 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:09.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:09 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:09.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:09 ntpd[13048]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:09.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:09 ntpd[13048]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:09.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:09 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:09.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:09 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:09.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:09 ntpd[13048]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:09.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:09 ntpd[13048]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:10.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:10 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:10.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:10 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:10.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:10 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:10.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:10 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:10.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:10 ntpd[13005]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:10.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:10 ntpd[13005]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:10.841 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:10 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:10.841 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:10 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:10.842 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:10 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:10.842 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:10 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:10.842 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:10 ntpd[13048]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:10.842 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:10 ntpd[13048]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:11.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:11 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:11.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:11 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:11.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:11 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:11.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:11 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:11.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:11 ntpd[13005]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:11.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:11 ntpd[13005]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:11.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:11 ntpd[13005]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:11.840 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:11 ntpd[13005]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:11.841 INFO:teuthology.orchestra.run.smithi081.stdout: 8 May 23:26:11 ntpd[13005]: ntpd: time slew -0.000157 s 2024-05-08T23:26:11.841 INFO:teuthology.orchestra.run.smithi081.stdout:ntpd: time slew -0.000157s 2024-05-08T23:26:11.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:11 ntpd[13005]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-08T23:26:11.841 INFO:teuthology.orchestra.run.smithi081.stderr: 8 May 23:26:11 ntpd[13005]: can't open /var/log/ntpstats/loopstats.20240508: Permission denied 2024-05-08T23:26:11.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:11 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:11.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:11 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:11.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:11 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:11.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:11 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:11.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:11 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:11.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:11 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:11.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:11 ntpd[13048]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:11.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:11 ntpd[13048]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:11.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:11 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:11.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:11 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:11.844 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:11 ntpd[13048]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:11.844 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:11 ntpd[13048]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:11.913 INFO:teuthology.orchestra.run.smithi081.stdout: remote refid st t when poll reach delay offset jitter 2024-05-08T23:26:11.913 INFO:teuthology.orchestra.run.smithi081.stdout:============================================================================== 2024-05-08T23:26:11.913 INFO:teuthology.orchestra.run.smithi081.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T23:26:11.913 INFO:teuthology.orchestra.run.smithi081.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T23:26:11.913 INFO:teuthology.orchestra.run.smithi081.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T23:26:11.913 INFO:teuthology.orchestra.run.smithi081.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T23:26:12.841 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:12 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:12.841 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:12 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:12.841 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:12 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:12.841 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:12 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:12.841 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:12 ntpd[13048]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:12.841 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:12 ntpd[13048]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:13.841 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:13 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:13.841 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:13 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:13.842 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:13 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:13.842 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:13 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:13.842 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:13 ntpd[13048]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-08T23:26:13.842 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:13 ntpd[13048]: can't open /var/log/ntpstats/rawstats.20240508: Permission denied 2024-05-08T23:26:13.842 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:13 ntpd[13048]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-08T23:26:13.842 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:13 ntpd[13048]: can't open /var/log/ntpstats/peerstats.20240508: Permission denied 2024-05-08T23:26:13.842 INFO:teuthology.orchestra.run.smithi117.stdout: 8 May 23:26:13 ntpd[13048]: ntpd: time slew +0.000722 s 2024-05-08T23:26:13.842 INFO:teuthology.orchestra.run.smithi117.stdout:ntpd: time slew +0.000722s 2024-05-08T23:26:13.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:13 ntpd[13048]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-08T23:26:13.843 INFO:teuthology.orchestra.run.smithi117.stderr: 8 May 23:26:13 ntpd[13048]: can't open /var/log/ntpstats/loopstats.20240508: Permission denied 2024-05-08T23:26:13.916 INFO:teuthology.orchestra.run.smithi117.stdout: remote refid st t when poll reach delay offset jitter 2024-05-08T23:26:13.917 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================== 2024-05-08T23:26:13.917 INFO:teuthology.orchestra.run.smithi117.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T23:26:13.917 INFO:teuthology.orchestra.run.smithi117.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T23:26:13.917 INFO:teuthology.orchestra.run.smithi117.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T23:26:13.917 INFO:teuthology.orchestra.run.smithi117.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T23:26:13.918 INFO:teuthology.run_tasks:Running task install... 2024-05-08T23:26:13.928 DEBUG:teuthology.task.install:project ceph 2024-05-08T23:26:13.928 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}} 2024-05-08T23:26:13.928 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'} 2024-05-08T23:26:13.928 INFO:teuthology.task.install:Using flavor: default 2024-05-08T23:26:13.934 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T23:26:13.934 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T23:26:13.935 DEBUG:teuthology.orchestra.run.smithi081:> sudo apt-key list | grep Ceph 2024-05-08T23:26:13.936 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-key list | grep Ceph 2024-05-08T23:26:14.038 INFO:teuthology.orchestra.run.smithi081.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-05-08T23:26:14.056 INFO:teuthology.orchestra.run.smithi117.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-05-08T23:26:14.101 INFO:teuthology.orchestra.run.smithi081.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-08T23:26:14.101 INFO:teuthology.orchestra.run.smithi081.stdout:uid [ unknown] Ceph.com (release key) 2024-05-08T23:26:14.102 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-08T23:26:14.102 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-08T23:26:14.102 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:26:14.113 INFO:teuthology.orchestra.run.smithi117.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-08T23:26:14.113 INFO:teuthology.orchestra.run.smithi117.stdout:uid [ unknown] Ceph.com (release key) 2024-05-08T23:26:14.114 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-08T23:26:14.114 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-08T23:26:14.114 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:26:14.260 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default/ 2024-05-08T23:26:14.261 INFO:teuthology.task.install.deb:Package version is 18.2.2-1322-gd21e4f96-1focal 2024-05-08T23:26:14.323 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default/ 2024-05-08T23:26:14.323 INFO:teuthology.task.install.deb:Package version is 18.2.2-1322-gd21e4f96-1focal 2024-05-08T23:26:14.581 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:26:14.581 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-08T23:26:14.597 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get update 2024-05-08T23:26:14.636 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:26:14.637 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-08T23:26:14.653 DEBUG:teuthology.orchestra.run.smithi081:> sudo apt-get update 2024-05-08T23:26:14.750 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-05-08T23:26:14.750 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-05-08T23:26:14.759 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-05-08T23:26:14.780 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-05-08T23:26:14.798 INFO:teuthology.orchestra.run.smithi081.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-05-08T23:26:14.909 INFO:teuthology.orchestra.run.smithi081.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-05-08T23:26:14.990 INFO:teuthology.orchestra.run.smithi081.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-05-08T23:26:15.042 INFO:teuthology.orchestra.run.smithi081.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal InRelease 2024-05-08T23:26:15.072 INFO:teuthology.orchestra.run.smithi117.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal InRelease 2024-05-08T23:26:15.072 INFO:teuthology.orchestra.run.smithi081.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-05-08T23:26:15.117 INFO:teuthology.orchestra.run.smithi081.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-05-08T23:26:15.159 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-05-08T23:26:15.193 INFO:teuthology.orchestra.run.smithi081.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal Release.gpg 2024-05-08T23:26:15.249 INFO:teuthology.orchestra.run.smithi117.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal Release.gpg 2024-05-08T23:26:15.267 INFO:teuthology.orchestra.run.smithi081.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main i386 Packages [3,414 B] 2024-05-08T23:26:15.338 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-05-08T23:26:15.351 INFO:teuthology.orchestra.run.smithi081.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-05-08T23:26:15.442 INFO:teuthology.orchestra.run.smithi081.stdout:Fetched 28.7 kB in 1s (44.8 kB/s) 2024-05-08T23:26:15.528 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main i386 Packages [3,414 B] 2024-05-08T23:26:15.622 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 28.7 kB in 1s (33.3 kB/s) 2024-05-08T23:26:16.557 INFO:teuthology.orchestra.run.smithi081.stdout:Reading package lists... 2024-05-08T23:26:16.583 DEBUG:teuthology.orchestra.run.smithi081:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.2-1322-gd21e4f96-1focal cephadm=18.2.2-1322-gd21e4f96-1focal ceph-mds=18.2.2-1322-gd21e4f96-1focal ceph-mgr=18.2.2-1322-gd21e4f96-1focal ceph-common=18.2.2-1322-gd21e4f96-1focal ceph-fuse=18.2.2-1322-gd21e4f96-1focal ceph-test=18.2.2-1322-gd21e4f96-1focal ceph-volume=18.2.2-1322-gd21e4f96-1focal radosgw=18.2.2-1322-gd21e4f96-1focal python3-rados=18.2.2-1322-gd21e4f96-1focal python3-rgw=18.2.2-1322-gd21e4f96-1focal python3-cephfs=18.2.2-1322-gd21e4f96-1focal python3-rbd=18.2.2-1322-gd21e4f96-1focal libcephfs2=18.2.2-1322-gd21e4f96-1focal libcephfs-dev=18.2.2-1322-gd21e4f96-1focal librados2=18.2.2-1322-gd21e4f96-1focal librbd1=18.2.2-1322-gd21e4f96-1focal rbd-fuse=18.2.2-1322-gd21e4f96-1focal python3-cephfs=18.2.2-1322-gd21e4f96-1focal cephfs-shell=18.2.2-1322-gd21e4f96-1focal cephfs-top=18.2.2-1322-gd21e4f96-1focal cephfs-mirror=18.2.2-1322-gd21e4f96-1focal 2024-05-08T23:26:16.654 INFO:teuthology.orchestra.run.smithi081.stdout:Reading package lists... 2024-05-08T23:26:16.722 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-05-08T23:26:16.747 DEBUG:teuthology.orchestra.run.smithi117:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.2-1322-gd21e4f96-1focal cephadm=18.2.2-1322-gd21e4f96-1focal ceph-mds=18.2.2-1322-gd21e4f96-1focal ceph-mgr=18.2.2-1322-gd21e4f96-1focal ceph-common=18.2.2-1322-gd21e4f96-1focal ceph-fuse=18.2.2-1322-gd21e4f96-1focal ceph-test=18.2.2-1322-gd21e4f96-1focal ceph-volume=18.2.2-1322-gd21e4f96-1focal radosgw=18.2.2-1322-gd21e4f96-1focal python3-rados=18.2.2-1322-gd21e4f96-1focal python3-rgw=18.2.2-1322-gd21e4f96-1focal python3-cephfs=18.2.2-1322-gd21e4f96-1focal python3-rbd=18.2.2-1322-gd21e4f96-1focal libcephfs2=18.2.2-1322-gd21e4f96-1focal libcephfs-dev=18.2.2-1322-gd21e4f96-1focal librados2=18.2.2-1322-gd21e4f96-1focal librbd1=18.2.2-1322-gd21e4f96-1focal rbd-fuse=18.2.2-1322-gd21e4f96-1focal python3-cephfs=18.2.2-1322-gd21e4f96-1focal cephfs-shell=18.2.2-1322-gd21e4f96-1focal cephfs-top=18.2.2-1322-gd21e4f96-1focal cephfs-mirror=18.2.2-1322-gd21e4f96-1focal 2024-05-08T23:26:16.815 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-05-08T23:26:16.865 INFO:teuthology.orchestra.run.smithi081.stdout:Building dependency tree... 2024-05-08T23:26:16.866 INFO:teuthology.orchestra.run.smithi081.stdout:Reading state information... 2024-05-08T23:26:17.025 INFO:teuthology.orchestra.run.smithi117.stdout:Building dependency tree... 2024-05-08T23:26:17.026 INFO:teuthology.orchestra.run.smithi117.stdout:Reading state information... 2024-05-08T23:26:17.059 INFO:teuthology.orchestra.run.smithi081.stdout:The following packages were automatically installed and are no longer required: 2024-05-08T23:26:17.060 INFO:teuthology.orchestra.run.smithi081.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-05-08T23:26:17.061 INFO:teuthology.orchestra.run.smithi081.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-08T23:26:17.061 INFO:teuthology.orchestra.run.smithi081.stdout:The following additional packages will be installed: 2024-05-08T23:26:17.061 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-08T23:26:17.061 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-05-08T23:26:17.061 INFO:teuthology.orchestra.run.smithi081.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-05-08T23:26:17.061 INFO:teuthology.orchestra.run.smithi081.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-05-08T23:26:17.061 INFO:teuthology.orchestra.run.smithi081.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-05-08T23:26:17.062 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-05-08T23:26:17.062 INFO:teuthology.orchestra.run.smithi081.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-05-08T23:26:17.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-05-08T23:26:17.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-05-08T23:26:17.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2024-05-08T23:26:17.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-05-08T23:26:17.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-05-08T23:26:17.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-05-08T23:26:17.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-05-08T23:26:17.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-08T23:26:17.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-05-08T23:26:17.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2024-05-08T23:26:17.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-05-08T23:26:17.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-05-08T23:26:17.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-05-08T23:26:17.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-05-08T23:26:17.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-05-08T23:26:17.063 INFO:teuthology.orchestra.run.smithi081.stdout: qttranslations5-l10n socat xmlstarlet 2024-05-08T23:26:17.063 INFO:teuthology.orchestra.run.smithi081.stdout:Suggested packages: 2024-05-08T23:26:17.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-05-08T23:26:17.064 INFO:teuthology.orchestra.run.smithi081.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-05-08T23:26:17.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-05-08T23:26:17.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-05-08T23:26:17.064 INFO:teuthology.orchestra.run.smithi081.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-05-08T23:26:17.064 INFO:teuthology.orchestra.run.smithi081.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-05-08T23:26:17.064 INFO:teuthology.orchestra.run.smithi081.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-05-08T23:26:17.064 INFO:teuthology.orchestra.run.smithi081.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-08T23:26:17.064 INFO:teuthology.orchestra.run.smithi081.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-05-08T23:26:17.064 INFO:teuthology.orchestra.run.smithi081.stdout:Recommended packages: 2024-05-08T23:26:17.064 INFO:teuthology.orchestra.run.smithi081.stdout: btrfs-tools 2024-05-08T23:26:17.187 INFO:teuthology.orchestra.run.smithi081.stdout:The following NEW packages will be installed: 2024-05-08T23:26:17.187 INFO:teuthology.orchestra.run.smithi081.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-08T23:26:17.187 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-08T23:26:17.187 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-08T23:26:17.187 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-05-08T23:26:17.187 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-05-08T23:26:17.187 INFO:teuthology.orchestra.run.smithi081.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-05-08T23:26:17.187 INFO:teuthology.orchestra.run.smithi081.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-05-08T23:26:17.188 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-05-08T23:26:17.188 INFO:teuthology.orchestra.run.smithi081.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-05-08T23:26:17.188 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-05-08T23:26:17.188 INFO:teuthology.orchestra.run.smithi081.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-05-08T23:26:17.188 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2024-05-08T23:26:17.188 INFO:teuthology.orchestra.run.smithi081.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-05-08T23:26:17.188 INFO:teuthology.orchestra.run.smithi081.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-05-08T23:26:17.188 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-05-08T23:26:17.189 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-05-08T23:26:17.189 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-05-08T23:26:17.189 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-05-08T23:26:17.189 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-05-08T23:26:17.189 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-05-08T23:26:17.189 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-08T23:26:17.189 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-05-08T23:26:17.189 INFO:teuthology.orchestra.run.smithi081.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-05-08T23:26:17.189 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-05-08T23:26:17.189 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-05-08T23:26:17.189 INFO:teuthology.orchestra.run.smithi081.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-05-08T23:26:17.190 INFO:teuthology.orchestra.run.smithi081.stdout:The following packages will be upgraded: 2024-05-08T23:26:17.190 INFO:teuthology.orchestra.run.smithi081.stdout: librados2 librbd1 2024-05-08T23:26:17.218 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages were automatically installed and are no longer required: 2024-05-08T23:26:17.220 INFO:teuthology.orchestra.run.smithi117.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-05-08T23:26:17.220 INFO:teuthology.orchestra.run.smithi117.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-08T23:26:17.220 INFO:teuthology.orchestra.run.smithi117.stdout:The following additional packages will be installed: 2024-05-08T23:26:17.220 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-08T23:26:17.220 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-05-08T23:26:17.220 INFO:teuthology.orchestra.run.smithi117.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-05-08T23:26:17.220 INFO:teuthology.orchestra.run.smithi117.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-05-08T23:26:17.220 INFO:teuthology.orchestra.run.smithi117.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-05-08T23:26:17.221 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-05-08T23:26:17.221 INFO:teuthology.orchestra.run.smithi117.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-05-08T23:26:17.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-05-08T23:26:17.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-05-08T23:26:17.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2024-05-08T23:26:17.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-05-08T23:26:17.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-05-08T23:26:17.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-05-08T23:26:17.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-05-08T23:26:17.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-08T23:26:17.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-05-08T23:26:17.221 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2024-05-08T23:26:17.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-05-08T23:26:17.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-05-08T23:26:17.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-05-08T23:26:17.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-05-08T23:26:17.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-05-08T23:26:17.222 INFO:teuthology.orchestra.run.smithi117.stdout: qttranslations5-l10n socat xmlstarlet 2024-05-08T23:26:17.222 INFO:teuthology.orchestra.run.smithi117.stdout:Suggested packages: 2024-05-08T23:26:17.222 INFO:teuthology.orchestra.run.smithi117.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-05-08T23:26:17.222 INFO:teuthology.orchestra.run.smithi117.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-05-08T23:26:17.223 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-05-08T23:26:17.223 INFO:teuthology.orchestra.run.smithi117.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-05-08T23:26:17.223 INFO:teuthology.orchestra.run.smithi117.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-05-08T23:26:17.223 INFO:teuthology.orchestra.run.smithi117.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-05-08T23:26:17.223 INFO:teuthology.orchestra.run.smithi117.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-05-08T23:26:17.223 INFO:teuthology.orchestra.run.smithi117.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-08T23:26:17.223 INFO:teuthology.orchestra.run.smithi117.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-05-08T23:26:17.223 INFO:teuthology.orchestra.run.smithi117.stdout:Recommended packages: 2024-05-08T23:26:17.223 INFO:teuthology.orchestra.run.smithi117.stdout: btrfs-tools 2024-05-08T23:26:17.260 INFO:teuthology.orchestra.run.smithi081.stdout:2 upgraded, 116 newly installed, 0 to remove and 278 not upgraded. 2024-05-08T23:26:17.260 INFO:teuthology.orchestra.run.smithi081.stdout:Need to get 199 MB of archives. 2024-05-08T23:26:17.260 INFO:teuthology.orchestra.run.smithi081.stdout:After this operation, 977 MB of additional disk space will be used. 2024-05-08T23:26:17.260 INFO:teuthology.orchestra.run.smithi081.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-05-08T23:26:17.282 INFO:teuthology.orchestra.run.smithi081.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-05-08T23:26:17.290 INFO:teuthology.orchestra.run.smithi081.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-05-08T23:26:17.321 INFO:teuthology.orchestra.run.smithi081.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-05-08T23:26:17.342 INFO:teuthology.orchestra.run.smithi117.stdout:The following NEW packages will be installed: 2024-05-08T23:26:17.343 INFO:teuthology.orchestra.run.smithi117.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-08T23:26:17.343 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-08T23:26:17.343 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-08T23:26:17.343 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-05-08T23:26:17.343 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-05-08T23:26:17.343 INFO:teuthology.orchestra.run.smithi117.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-05-08T23:26:17.343 INFO:teuthology.orchestra.run.smithi117.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-05-08T23:26:17.344 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-05-08T23:26:17.344 INFO:teuthology.orchestra.run.smithi117.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-05-08T23:26:17.344 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-05-08T23:26:17.344 INFO:teuthology.orchestra.run.smithi117.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-05-08T23:26:17.344 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2024-05-08T23:26:17.344 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-05-08T23:26:17.344 INFO:teuthology.orchestra.run.smithi117.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-05-08T23:26:17.344 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-05-08T23:26:17.345 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-05-08T23:26:17.345 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-05-08T23:26:17.345 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-05-08T23:26:17.345 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-05-08T23:26:17.345 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-05-08T23:26:17.345 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-08T23:26:17.345 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-05-08T23:26:17.345 INFO:teuthology.orchestra.run.smithi117.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-05-08T23:26:17.345 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-05-08T23:26:17.345 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-05-08T23:26:17.345 INFO:teuthology.orchestra.run.smithi117.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-05-08T23:26:17.346 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages will be upgraded: 2024-05-08T23:26:17.346 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 librbd1 2024-05-08T23:26:17.356 INFO:teuthology.orchestra.run.smithi081.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-05-08T23:26:17.365 INFO:teuthology.orchestra.run.smithi081.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-05-08T23:26:17.379 INFO:teuthology.orchestra.run.smithi081.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-05-08T23:26:17.430 INFO:teuthology.orchestra.run.smithi117.stdout:2 upgraded, 116 newly installed, 0 to remove and 278 not upgraded. 2024-05-08T23:26:17.430 INFO:teuthology.orchestra.run.smithi117.stdout:Need to get 199 MB of archives. 2024-05-08T23:26:17.430 INFO:teuthology.orchestra.run.smithi117.stdout:After this operation, 977 MB of additional disk space will be used. 2024-05-08T23:26:17.431 INFO:teuthology.orchestra.run.smithi117.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-05-08T23:26:17.449 INFO:teuthology.orchestra.run.smithi081.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-05-08T23:26:17.451 INFO:teuthology.orchestra.run.smithi081.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-05-08T23:26:17.454 INFO:teuthology.orchestra.run.smithi117.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-05-08T23:26:17.460 INFO:teuthology.orchestra.run.smithi081.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-05-08T23:26:17.463 INFO:teuthology.orchestra.run.smithi081.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-05-08T23:26:17.463 INFO:teuthology.orchestra.run.smithi081.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-05-08T23:26:17.465 INFO:teuthology.orchestra.run.smithi081.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-05-08T23:26:17.465 INFO:teuthology.orchestra.run.smithi081.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-05-08T23:26:17.467 INFO:teuthology.orchestra.run.smithi117.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-05-08T23:26:17.469 INFO:teuthology.orchestra.run.smithi081.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-05-08T23:26:17.470 INFO:teuthology.orchestra.run.smithi081.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-05-08T23:26:17.472 INFO:teuthology.orchestra.run.smithi081.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-05-08T23:26:17.493 INFO:teuthology.orchestra.run.smithi081.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-05-08T23:26:17.493 INFO:teuthology.orchestra.run.smithi081.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-05-08T23:26:17.494 INFO:teuthology.orchestra.run.smithi081.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-05-08T23:26:17.495 INFO:teuthology.orchestra.run.smithi081.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-08T23:26:17.496 INFO:teuthology.orchestra.run.smithi081.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-05-08T23:26:17.496 INFO:teuthology.orchestra.run.smithi081.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-05-08T23:26:17.498 INFO:teuthology.orchestra.run.smithi081.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-05-08T23:26:17.498 INFO:teuthology.orchestra.run.smithi081.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-05-08T23:26:17.500 INFO:teuthology.orchestra.run.smithi117.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-05-08T23:26:17.500 INFO:teuthology.orchestra.run.smithi081.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-05-08T23:26:17.515 INFO:teuthology.orchestra.run.smithi081.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-05-08T23:26:17.523 INFO:teuthology.orchestra.run.smithi081.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-05-08T23:26:17.530 INFO:teuthology.orchestra.run.smithi081.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-05-08T23:26:17.531 INFO:teuthology.orchestra.run.smithi081.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-05-08T23:26:17.531 INFO:teuthology.orchestra.run.smithi081.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-05-08T23:26:17.532 INFO:teuthology.orchestra.run.smithi081.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-05-08T23:26:17.533 INFO:teuthology.orchestra.run.smithi081.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2024-05-08T23:26:17.535 INFO:teuthology.orchestra.run.smithi081.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-05-08T23:26:17.536 INFO:teuthology.orchestra.run.smithi081.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-05-08T23:26:17.538 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-05-08T23:26:17.538 INFO:teuthology.orchestra.run.smithi081.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-05-08T23:26:17.541 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-05-08T23:26:17.558 INFO:teuthology.orchestra.run.smithi081.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-05-08T23:26:17.562 INFO:teuthology.orchestra.run.smithi081.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.2 [95.9 kB] 2024-05-08T23:26:17.563 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-05-08T23:26:17.564 INFO:teuthology.orchestra.run.smithi081.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-05-08T23:26:17.565 INFO:teuthology.orchestra.run.smithi081.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-05-08T23:26:17.566 INFO:teuthology.orchestra.run.smithi081.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-05-08T23:26:17.566 INFO:teuthology.orchestra.run.smithi081.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-05-08T23:26:17.568 INFO:teuthology.orchestra.run.smithi081.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-05-08T23:26:17.568 INFO:teuthology.orchestra.run.smithi081.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-05-08T23:26:17.579 INFO:teuthology.orchestra.run.smithi081.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-05-08T23:26:17.581 INFO:teuthology.orchestra.run.smithi081.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-08T23:26:17.600 INFO:teuthology.orchestra.run.smithi081.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-05-08T23:26:17.602 INFO:teuthology.orchestra.run.smithi081.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-05-08T23:26:17.604 INFO:teuthology.orchestra.run.smithi081.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-05-08T23:26:17.605 INFO:teuthology.orchestra.run.smithi081.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-05-08T23:26:17.606 INFO:teuthology.orchestra.run.smithi081.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-05-08T23:26:17.607 INFO:teuthology.orchestra.run.smithi081.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-05-08T23:26:17.640 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-05-08T23:26:17.642 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-05-08T23:26:17.655 INFO:teuthology.orchestra.run.smithi117.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-05-08T23:26:17.662 INFO:teuthology.orchestra.run.smithi117.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-05-08T23:26:17.666 INFO:teuthology.orchestra.run.smithi117.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-05-08T23:26:17.668 INFO:teuthology.orchestra.run.smithi117.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-05-08T23:26:17.671 INFO:teuthology.orchestra.run.smithi117.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-05-08T23:26:17.685 INFO:teuthology.orchestra.run.smithi117.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-05-08T23:26:17.686 INFO:teuthology.orchestra.run.smithi117.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-05-08T23:26:17.688 INFO:teuthology.orchestra.run.smithi117.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-05-08T23:26:17.704 INFO:teuthology.orchestra.run.smithi081.stdout:Get:53 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.2-1322-gd21e4f96-1focal [3,446 kB] 2024-05-08T23:26:17.721 INFO:teuthology.orchestra.run.smithi117.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-05-08T23:26:17.722 INFO:teuthology.orchestra.run.smithi117.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-05-08T23:26:17.723 INFO:teuthology.orchestra.run.smithi117.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-05-08T23:26:17.724 INFO:teuthology.orchestra.run.smithi117.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-08T23:26:17.725 INFO:teuthology.orchestra.run.smithi117.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-05-08T23:26:17.725 INFO:teuthology.orchestra.run.smithi117.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-05-08T23:26:17.727 INFO:teuthology.orchestra.run.smithi117.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-05-08T23:26:17.728 INFO:teuthology.orchestra.run.smithi117.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-05-08T23:26:17.747 INFO:teuthology.orchestra.run.smithi081.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-05-08T23:26:17.756 INFO:teuthology.orchestra.run.smithi117.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-05-08T23:26:17.758 INFO:teuthology.orchestra.run.smithi117.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-05-08T23:26:17.771 INFO:teuthology.orchestra.run.smithi081.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-05-08T23:26:17.772 INFO:teuthology.orchestra.run.smithi081.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-05-08T23:26:17.789 INFO:teuthology.orchestra.run.smithi117.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-05-08T23:26:17.797 INFO:teuthology.orchestra.run.smithi117.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-05-08T23:26:17.798 INFO:teuthology.orchestra.run.smithi117.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-05-08T23:26:17.799 INFO:teuthology.orchestra.run.smithi081.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-05-08T23:26:17.799 INFO:teuthology.orchestra.run.smithi081.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-05-08T23:26:17.799 INFO:teuthology.orchestra.run.smithi117.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-05-08T23:26:17.800 INFO:teuthology.orchestra.run.smithi117.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-05-08T23:26:17.800 INFO:teuthology.orchestra.run.smithi081.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-05-08T23:26:17.800 INFO:teuthology.orchestra.run.smithi081.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-05-08T23:26:17.802 INFO:teuthology.orchestra.run.smithi117.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2024-05-08T23:26:17.805 INFO:teuthology.orchestra.run.smithi117.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-05-08T23:26:17.806 INFO:teuthology.orchestra.run.smithi117.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-05-08T23:26:17.809 INFO:teuthology.orchestra.run.smithi117.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-05-08T23:26:17.810 INFO:teuthology.orchestra.run.smithi117.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-05-08T23:26:17.832 INFO:teuthology.orchestra.run.smithi117.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.2 [95.9 kB] 2024-05-08T23:26:17.835 INFO:teuthology.orchestra.run.smithi117.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-05-08T23:26:17.836 INFO:teuthology.orchestra.run.smithi117.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-05-08T23:26:17.837 INFO:teuthology.orchestra.run.smithi117.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-05-08T23:26:17.837 INFO:teuthology.orchestra.run.smithi117.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-05-08T23:26:17.840 INFO:teuthology.orchestra.run.smithi117.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-05-08T23:26:17.840 INFO:teuthology.orchestra.run.smithi117.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-05-08T23:26:17.841 INFO:teuthology.orchestra.run.smithi117.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-05-08T23:26:17.849 INFO:teuthology.orchestra.run.smithi081.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-05-08T23:26:17.849 INFO:teuthology.orchestra.run.smithi081.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-05-08T23:26:17.849 INFO:teuthology.orchestra.run.smithi081.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-05-08T23:26:17.851 INFO:teuthology.orchestra.run.smithi081.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-05-08T23:26:17.851 INFO:teuthology.orchestra.run.smithi081.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-05-08T23:26:17.852 INFO:teuthology.orchestra.run.smithi081.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-05-08T23:26:17.853 INFO:teuthology.orchestra.run.smithi117.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-08T23:26:17.853 INFO:teuthology.orchestra.run.smithi117.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-05-08T23:26:17.857 INFO:teuthology.orchestra.run.smithi081.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-05-08T23:26:17.858 INFO:teuthology.orchestra.run.smithi081.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-05-08T23:26:17.862 INFO:teuthology.orchestra.run.smithi117.stdout:Get:48 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.2-1322-gd21e4f96-1focal [3,446 kB] 2024-05-08T23:26:17.869 INFO:teuthology.orchestra.run.smithi081.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-05-08T23:26:17.873 INFO:teuthology.orchestra.run.smithi117.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-05-08T23:26:17.875 INFO:teuthology.orchestra.run.smithi117.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-05-08T23:26:17.877 INFO:teuthology.orchestra.run.smithi117.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-05-08T23:26:17.878 INFO:teuthology.orchestra.run.smithi117.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-05-08T23:26:17.878 INFO:teuthology.orchestra.run.smithi117.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-05-08T23:26:17.884 INFO:teuthology.orchestra.run.smithi081.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-05-08T23:26:17.890 INFO:teuthology.orchestra.run.smithi081.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-05-08T23:26:17.894 INFO:teuthology.orchestra.run.smithi081.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-05-08T23:26:17.895 INFO:teuthology.orchestra.run.smithi081.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-05-08T23:26:17.921 INFO:teuthology.orchestra.run.smithi081.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-05-08T23:26:17.924 INFO:teuthology.orchestra.run.smithi081.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-05-08T23:26:17.942 INFO:teuthology.orchestra.run.smithi081.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-05-08T23:26:18.109 INFO:teuthology.orchestra.run.smithi081.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-05-08T23:26:18.111 INFO:teuthology.orchestra.run.smithi081.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-05-08T23:26:18.111 INFO:teuthology.orchestra.run.smithi081.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-05-08T23:26:18.113 INFO:teuthology.orchestra.run.smithi081.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-05-08T23:26:18.126 INFO:teuthology.orchestra.run.smithi081.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-08T23:26:18.127 INFO:teuthology.orchestra.run.smithi081.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-05-08T23:26:18.128 INFO:teuthology.orchestra.run.smithi081.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-05-08T23:26:18.128 INFO:teuthology.orchestra.run.smithi081.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-05-08T23:26:18.128 INFO:teuthology.orchestra.run.smithi081.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-05-08T23:26:18.181 INFO:teuthology.orchestra.run.smithi117.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-05-08T23:26:18.203 INFO:teuthology.orchestra.run.smithi117.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-05-08T23:26:18.204 INFO:teuthology.orchestra.run.smithi117.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-05-08T23:26:18.247 INFO:teuthology.orchestra.run.smithi117.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-05-08T23:26:18.247 INFO:teuthology.orchestra.run.smithi117.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-05-08T23:26:18.248 INFO:teuthology.orchestra.run.smithi117.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-05-08T23:26:18.248 INFO:teuthology.orchestra.run.smithi117.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-05-08T23:26:18.249 INFO:teuthology.orchestra.run.smithi117.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-05-08T23:26:18.250 INFO:teuthology.orchestra.run.smithi117.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-05-08T23:26:18.250 INFO:teuthology.orchestra.run.smithi117.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-05-08T23:26:18.253 INFO:teuthology.orchestra.run.smithi117.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-05-08T23:26:18.253 INFO:teuthology.orchestra.run.smithi117.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-05-08T23:26:18.280 INFO:teuthology.orchestra.run.smithi117.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-05-08T23:26:18.285 INFO:teuthology.orchestra.run.smithi117.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-05-08T23:26:18.307 INFO:teuthology.orchestra.run.smithi117.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-05-08T23:26:18.309 INFO:teuthology.orchestra.run.smithi117.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-05-08T23:26:18.310 INFO:teuthology.orchestra.run.smithi117.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-05-08T23:26:18.314 INFO:teuthology.orchestra.run.smithi117.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-05-08T23:26:18.321 INFO:teuthology.orchestra.run.smithi117.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-05-08T23:26:18.322 INFO:teuthology.orchestra.run.smithi117.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-05-08T23:26:18.385 INFO:teuthology.orchestra.run.smithi117.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-05-08T23:26:18.392 INFO:teuthology.orchestra.run.smithi117.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-05-08T23:26:18.398 INFO:teuthology.orchestra.run.smithi117.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-05-08T23:26:18.523 INFO:teuthology.orchestra.run.smithi117.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-05-08T23:26:18.524 INFO:teuthology.orchestra.run.smithi117.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-05-08T23:26:18.524 INFO:teuthology.orchestra.run.smithi117.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-05-08T23:26:18.525 INFO:teuthology.orchestra.run.smithi117.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-05-08T23:26:18.557 INFO:teuthology.orchestra.run.smithi117.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-08T23:26:18.558 INFO:teuthology.orchestra.run.smithi117.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-05-08T23:26:18.560 INFO:teuthology.orchestra.run.smithi117.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-05-08T23:26:18.560 INFO:teuthology.orchestra.run.smithi117.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-05-08T23:26:18.560 INFO:teuthology.orchestra.run.smithi117.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-05-08T23:26:19.757 INFO:teuthology.orchestra.run.smithi117.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.2-1322-gd21e4f96-1focal [4,035 kB] 2024-05-08T23:26:20.562 INFO:teuthology.orchestra.run.smithi117.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.2-1322-gd21e4f96-1focal [744 kB] 2024-05-08T23:26:20.717 INFO:teuthology.orchestra.run.smithi117.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.2-1322-gd21e4f96-1focal [387 kB] 2024-05-08T23:26:20.794 INFO:teuthology.orchestra.run.smithi117.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.2-1322-gd21e4f96-1focal [31.9 kB] 2024-05-08T23:26:20.795 INFO:teuthology.orchestra.run.smithi117.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.2-1322-gd21e4f96-1focal [201 kB] 2024-05-08T23:26:20.805 INFO:teuthology.orchestra.run.smithi117.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.2-1322-gd21e4f96-1focal [62.5 kB] 2024-05-08T23:26:20.809 INFO:teuthology.orchestra.run.smithi117.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.2-1322-gd21e4f96-1focal [365 kB] 2024-05-08T23:26:20.886 INFO:teuthology.orchestra.run.smithi117.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.2-1322-gd21e4f96-1focal [6,591 kB] 2024-05-08T23:26:21.715 INFO:teuthology.orchestra.run.smithi081.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.2-1322-gd21e4f96-1focal [4,035 kB] 2024-05-08T23:26:21.765 INFO:teuthology.orchestra.run.smithi117.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.2-1322-gd21e4f96-1focal [113 kB] 2024-05-08T23:26:21.770 INFO:teuthology.orchestra.run.smithi117.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.2-1322-gd21e4f96-1focal [476 kB] 2024-05-08T23:26:21.844 INFO:teuthology.orchestra.run.smithi117.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.2-1322-gd21e4f96-1focal [25.1 MB] 2024-05-08T23:26:24.848 INFO:teuthology.orchestra.run.smithi081.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.2-1322-gd21e4f96-1focal [744 kB] 2024-05-08T23:26:24.849 INFO:teuthology.orchestra.run.smithi117.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.2-1322-gd21e4f96-1focal [9,604 kB] 2024-05-08T23:26:25.269 INFO:teuthology.orchestra.run.smithi081.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.2-1322-gd21e4f96-1focal [387 kB] 2024-05-08T23:26:25.481 INFO:teuthology.orchestra.run.smithi081.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.2-1322-gd21e4f96-1focal [31.9 kB] 2024-05-08T23:26:25.483 INFO:teuthology.orchestra.run.smithi081.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.2-1322-gd21e4f96-1focal [201 kB] 2024-05-08T23:26:25.620 INFO:teuthology.orchestra.run.smithi081.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.2-1322-gd21e4f96-1focal [62.5 kB] 2024-05-08T23:26:25.639 INFO:teuthology.orchestra.run.smithi117.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.2-1322-gd21e4f96-1focal [238 kB] 2024-05-08T23:26:25.647 INFO:teuthology.orchestra.run.smithi081.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.2-1322-gd21e4f96-1focal [365 kB] 2024-05-08T23:26:25.655 INFO:teuthology.orchestra.run.smithi117.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.2-1322-gd21e4f96-1focal [148 kB] 2024-05-08T23:26:25.664 INFO:teuthology.orchestra.run.smithi117.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.2-1322-gd21e4f96-1focal [1,573 kB] 2024-05-08T23:26:25.807 INFO:teuthology.orchestra.run.smithi117.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.2-1322-gd21e4f96-1focal [6,637 kB] 2024-05-08T23:26:25.844 INFO:teuthology.orchestra.run.smithi081.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.2-1322-gd21e4f96-1focal [6,591 kB] 2024-05-08T23:26:26.419 INFO:teuthology.orchestra.run.smithi117.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.2-1322-gd21e4f96-1focal [24.5 MB] 2024-05-08T23:26:28.300 INFO:teuthology.orchestra.run.smithi081.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.2-1322-gd21e4f96-1focal [113 kB] 2024-05-08T23:26:28.328 INFO:teuthology.orchestra.run.smithi081.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.2-1322-gd21e4f96-1focal [476 kB] 2024-05-08T23:26:28.443 INFO:teuthology.orchestra.run.smithi081.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.2-1322-gd21e4f96-1focal [25.1 MB] 2024-05-08T23:26:31.269 INFO:teuthology.orchestra.run.smithi081.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.2-1322-gd21e4f96-1focal [9,604 kB] 2024-05-08T23:26:32.162 INFO:teuthology.orchestra.run.smithi081.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.2-1322-gd21e4f96-1focal [238 kB] 2024-05-08T23:26:32.174 INFO:teuthology.orchestra.run.smithi081.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.2-1322-gd21e4f96-1focal [148 kB] 2024-05-08T23:26:32.225 INFO:teuthology.orchestra.run.smithi081.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.2-1322-gd21e4f96-1focal [1,573 kB] 2024-05-08T23:26:32.394 INFO:teuthology.orchestra.run.smithi117.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.2-1322-gd21e4f96-1focal [13.7 kB] 2024-05-08T23:26:32.402 INFO:teuthology.orchestra.run.smithi081.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.2-1322-gd21e4f96-1focal [6,637 kB] 2024-05-08T23:26:32.444 INFO:teuthology.orchestra.run.smithi117.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.2-1322-gd21e4f96-1focal [854 kB] 2024-05-08T23:26:32.458 INFO:teuthology.orchestra.run.smithi117.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.2-1322-gd21e4f96-1focal [2,291 kB] 2024-05-08T23:26:32.656 INFO:teuthology.orchestra.run.smithi117.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.2-1322-gd21e4f96-1focal [237 kB] 2024-05-08T23:26:32.695 INFO:teuthology.orchestra.run.smithi117.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.2-1322-gd21e4f96-1focal [143 kB] 2024-05-08T23:26:32.701 INFO:teuthology.orchestra.run.smithi117.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.2-1322-gd21e4f96-1focal [7,012 kB] 2024-05-08T23:26:33.380 INFO:teuthology.orchestra.run.smithi081.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.2-1322-gd21e4f96-1focal [24.5 MB] 2024-05-08T23:26:33.441 INFO:teuthology.orchestra.run.smithi117.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.2-1322-gd21e4f96-1focal [7,876 kB] 2024-05-08T23:26:34.325 INFO:teuthology.orchestra.run.smithi117.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.2-1322-gd21e4f96-1focal [13.9 kB] 2024-05-08T23:26:34.326 INFO:teuthology.orchestra.run.smithi117.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.2-1322-gd21e4f96-1focal [46.7 MB] 2024-05-08T23:26:37.049 INFO:teuthology.orchestra.run.smithi081.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.2-1322-gd21e4f96-1focal [13.7 kB] 2024-05-08T23:26:37.049 INFO:teuthology.orchestra.run.smithi081.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.2-1322-gd21e4f96-1focal [854 kB] 2024-05-08T23:26:37.128 INFO:teuthology.orchestra.run.smithi081.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.2-1322-gd21e4f96-1focal [2,291 kB] 2024-05-08T23:26:37.500 INFO:teuthology.orchestra.run.smithi081.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.2-1322-gd21e4f96-1focal [237 kB] 2024-05-08T23:26:37.542 INFO:teuthology.orchestra.run.smithi081.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.2-1322-gd21e4f96-1focal [143 kB] 2024-05-08T23:26:37.550 INFO:teuthology.orchestra.run.smithi081.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.2-1322-gd21e4f96-1focal [7,012 kB] 2024-05-08T23:26:38.812 INFO:teuthology.orchestra.run.smithi081.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.2-1322-gd21e4f96-1focal [7,876 kB] 2024-05-08T23:26:40.354 INFO:teuthology.orchestra.run.smithi081.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.2-1322-gd21e4f96-1focal [13.9 kB] 2024-05-08T23:26:40.354 INFO:teuthology.orchestra.run.smithi081.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.2-1322-gd21e4f96-1focal [46.7 MB] 2024-05-08T23:26:42.191 INFO:teuthology.orchestra.run.smithi117.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.2-1322-gd21e4f96-1focal [119 kB] 2024-05-08T23:26:42.193 INFO:teuthology.orchestra.run.smithi117.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.2.2-1322-gd21e4f96-1focal [268 kB] 2024-05-08T23:26:42.198 INFO:teuthology.orchestra.run.smithi117.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.2.2-1322-gd21e4f96-1focal [33.0 kB] 2024-05-08T23:26:42.199 INFO:teuthology.orchestra.run.smithi117.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.2.2-1322-gd21e4f96-1focal [22.7 kB] 2024-05-08T23:26:42.200 INFO:teuthology.orchestra.run.smithi117.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.2-1322-gd21e4f96-1focal [37.4 kB] 2024-05-08T23:26:42.201 INFO:teuthology.orchestra.run.smithi117.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.2-1322-gd21e4f96-1focal [13.4 MB] 2024-05-08T23:26:44.093 INFO:teuthology.orchestra.run.smithi117.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.2-1322-gd21e4f96-1focal [107 kB] 2024-05-08T23:26:44.755 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 199 MB in 27s (7,432 kB/s) 2024-05-08T23:26:44.893 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-08T23:26:44.948 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-05-08T23:26:44.952 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-05-08T23:26:44.991 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-05-08T23:26:45.293 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-08T23:26:45.309 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-05-08T23:26:45.352 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-05-08T23:26:45.688 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cmd2. 2024-05-08T23:26:45.704 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-05-08T23:26:45.746 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-05-08T23:26:46.023 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-05-08T23:26:46.040 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-05-08T23:26:46.107 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-05-08T23:26:46.477 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-08T23:26:46.493 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-05-08T23:26:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-05-08T23:26:47.423 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-08T23:26:47.442 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-05-08T23:26:47.714 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-05-08T23:26:47.881 INFO:teuthology.orchestra.run.smithi081.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.2-1322-gd21e4f96-1focal [119 kB] 2024-05-08T23:26:47.883 INFO:teuthology.orchestra.run.smithi081.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.2.2-1322-gd21e4f96-1focal [268 kB] 2024-05-08T23:26:47.889 INFO:teuthology.orchestra.run.smithi081.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.2.2-1322-gd21e4f96-1focal [33.0 kB] 2024-05-08T23:26:47.958 INFO:teuthology.orchestra.run.smithi081.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.2.2-1322-gd21e4f96-1focal [22.7 kB] 2024-05-08T23:26:47.959 INFO:teuthology.orchestra.run.smithi081.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.2-1322-gd21e4f96-1focal [37.4 kB] 2024-05-08T23:26:47.960 INFO:teuthology.orchestra.run.smithi081.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.2-1322-gd21e4f96-1focal [13.4 MB] 2024-05-08T23:26:48.192 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-08T23:26:48.208 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-05-08T23:26:48.253 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T23:26:48.804 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-08T23:26:48.820 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-05-08T23:26:48.864 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T23:26:49.182 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-08T23:26:49.199 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-05-08T23:26:49.231 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T23:26:49.234 INFO:teuthology.orchestra.run.smithi081.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.2-1322-gd21e4f96-1focal [107 kB] 2024-05-08T23:26:49.618 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-05-08T23:26:49.635 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-05-08T23:26:49.676 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-05-08T23:26:49.890 INFO:teuthology.orchestra.run.smithi081.stdout:Fetched 199 MB in 32s (6,207 kB/s) 2024-05-08T23:26:50.039 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-08T23:26:50.045 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../010-librbd1_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:50.093 INFO:teuthology.orchestra.run.smithi081.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-05-08T23:26:50.097 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-05-08T23:26:50.137 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-05-08T23:26:50.196 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librbd1 (18.2.2-1322-gd21e4f96-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-05-08T23:26:50.465 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-08T23:26:50.482 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-05-08T23:26:50.523 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-05-08T23:26:50.859 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../011-librados2_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:50.859 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-cmd2. 2024-05-08T23:26:50.875 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-05-08T23:26:50.909 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-05-08T23:26:50.983 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librados2 (18.2.2-1322-gd21e4f96-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-05-08T23:26:51.207 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-05-08T23:26:51.220 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-05-08T23:26:51.304 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-05-08T23:26:51.666 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-08T23:26:51.685 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-05-08T23:26:51.695 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libcephfs2. 2024-05-08T23:26:51.714 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../012-libcephfs2_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:51.724 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-05-08T23:26:51.753 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libcephfs2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:52.064 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rados. 2024-05-08T23:26:52.080 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../013-python3-rados_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:52.086 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-08T23:26:52.105 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-05-08T23:26:52.122 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rados (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:52.144 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-05-08T23:26:52.475 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-08T23:26:52.491 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:26:52.533 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-ceph-argparse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:52.539 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-08T23:26:52.557 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-05-08T23:26:52.608 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T23:26:52.752 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cephfs. 2024-05-08T23:26:52.768 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../015-python3-cephfs_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:52.802 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cephfs (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:53.143 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-08T23:26:53.146 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-08T23:26:53.159 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-05-08T23:26:53.162 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../016-python3-ceph-common_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:26:53.201 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T23:26:53.238 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:53.566 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-prettytable. 2024-05-08T23:26:53.583 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2024-05-08T23:26:53.587 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-08T23:26:53.603 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-05-08T23:26:53.624 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-05-08T23:26:53.637 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T23:26:53.876 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rbd. 2024-05-08T23:26:53.894 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../018-python3-rbd_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:53.935 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rbd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:54.007 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-05-08T23:26:54.023 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-05-08T23:26:54.065 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-05-08T23:26:54.290 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-05-08T23:26:54.304 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-05-08T23:26:54.346 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-05-08T23:26:54.437 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../010-librbd1_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:54.560 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking librbd1 (18.2.2-1322-gd21e4f96-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-05-08T23:26:54.707 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-05-08T23:26:54.724 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2024-05-08T23:26:54.782 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-05-08T23:26:55.118 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-08T23:26:55.135 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-05-08T23:26:55.176 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-05-08T23:26:55.265 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../011-librados2_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:55.372 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking librados2 (18.2.2-1322-gd21e4f96-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-05-08T23:26:55.496 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librgw2. 2024-05-08T23:26:55.512 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../022-librgw2_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:55.554 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librgw2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:56.143 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libcephfs2. 2024-05-08T23:26:56.161 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../012-libcephfs2_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:56.201 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libcephfs2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:56.448 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rgw. 2024-05-08T23:26:56.465 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../023-python3-rgw_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:56.506 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:56.562 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-rados. 2024-05-08T23:26:56.579 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../013-python3-rados_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:56.620 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-rados (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:56.850 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liboath0. 2024-05-08T23:26:56.867 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-05-08T23:26:56.909 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-05-08T23:26:56.990 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-08T23:26:57.006 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:26:57.039 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-ceph-argparse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:57.236 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libradosstriper1. 2024-05-08T23:26:57.242 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-cephfs. 2024-05-08T23:26:57.252 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../025-libradosstriper1_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:57.260 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../015-python3-cephfs_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:57.303 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libradosstriper1 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:57.309 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-cephfs (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:57.637 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-08T23:26:57.656 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../016-python3-ceph-common_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:26:57.723 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-common. 2024-05-08T23:26:57.728 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:57.742 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../026-ceph-common_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:57.781 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:58.064 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-prettytable. 2024-05-08T23:26:58.082 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2024-05-08T23:26:58.148 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-05-08T23:26:58.400 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-rbd. 2024-05-08T23:26:58.417 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../018-python3-rbd_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:26:58.458 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-rbd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:26:58.820 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-05-08T23:26:58.840 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-05-08T23:26:58.878 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-05-08T23:26:59.264 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-05-08T23:26:59.280 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2024-05-08T23:26:59.322 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-05-08T23:26:59.684 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-08T23:26:59.702 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-05-08T23:26:59.742 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-05-08T23:27:00.095 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package librgw2. 2024-05-08T23:27:00.111 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../022-librgw2_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:00.153 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking librgw2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:00.286 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-base. 2024-05-08T23:27:00.302 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../027-ceph-base_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:00.363 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-base (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:00.981 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-rgw. 2024-05-08T23:27:01.000 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../023-python3-rgw_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:01.056 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-rgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:01.400 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package liboath0. 2024-05-08T23:27:01.416 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-05-08T23:27:01.475 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-05-08T23:27:01.607 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-08T23:27:01.626 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2024-05-08T23:27:01.690 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-05-08T23:27:01.786 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libradosstriper1. 2024-05-08T23:27:01.802 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../025-libradosstriper1_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:01.869 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libradosstriper1 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:02.018 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-natsort. 2024-05-08T23:27:02.037 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2024-05-08T23:27:02.076 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-05-08T23:27:02.272 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ceph-common. 2024-05-08T23:27:02.290 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../026-ceph-common_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:02.330 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:02.379 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-logutils. 2024-05-08T23:27:02.396 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2024-05-08T23:27:02.471 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-05-08T23:27:02.774 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-08T23:27:02.789 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-05-08T23:27:02.832 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-05-08T23:27:03.468 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-mako. 2024-05-08T23:27:03.484 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-05-08T23:27:03.526 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-05-08T23:27:03.846 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-08T23:27:03.863 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-08T23:27:03.904 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-08T23:27:04.241 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-08T23:27:04.257 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-05-08T23:27:04.298 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-05-08T23:27:04.601 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-webob. 2024-05-08T23:27:04.621 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2024-05-08T23:27:04.659 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-05-08T23:27:05.021 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-soupsieve. 2024-05-08T23:27:05.037 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-05-08T23:27:05.079 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-05-08T23:27:05.112 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ceph-base. 2024-05-08T23:27:05.132 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../027-ceph-base_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:05.206 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ceph-base (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:05.381 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-bs4. 2024-05-08T23:27:05.401 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2024-05-08T23:27:05.431 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-05-08T23:27:05.784 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-waitress. 2024-05-08T23:27:05.801 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-05-08T23:27:05.845 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-05-08T23:27:06.162 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-tempita. 2024-05-08T23:27:06.178 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2024-05-08T23:27:06.220 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-05-08T23:27:06.458 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-08T23:27:06.477 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2024-05-08T23:27:06.516 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-05-08T23:27:06.548 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-paste. 2024-05-08T23:27:06.564 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-05-08T23:27:06.606 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-05-08T23:27:06.878 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-natsort. 2024-05-08T23:27:06.894 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2024-05-08T23:27:06.918 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-08T23:27:06.935 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-05-08T23:27:06.936 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-05-08T23:27:06.976 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-05-08T23:27:07.273 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-logutils. 2024-05-08T23:27:07.281 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-08T23:27:07.292 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2024-05-08T23:27:07.297 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2024-05-08T23:27:07.330 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-05-08T23:27:07.331 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-05-08T23:27:07.617 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-08T23:27:07.634 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-webtest. 2024-05-08T23:27:07.634 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-05-08T23:27:07.653 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2024-05-08T23:27:07.692 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-05-08T23:27:07.708 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-05-08T23:27:08.028 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pecan. 2024-05-08T23:27:08.028 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-mako. 2024-05-08T23:27:08.044 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-05-08T23:27:08.046 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-05-08T23:27:08.086 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-05-08T23:27:08.111 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-05-08T23:27:08.433 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-08T23:27:08.440 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-08T23:27:08.451 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2024-05-08T23:27:08.457 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-08T23:27:08.491 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2024-05-08T23:27:08.523 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-08T23:27:08.785 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-08T23:27:08.801 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:08.809 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-08T23:27:08.828 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-05-08T23:27:08.843 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-modules-core (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:08.876 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-05-08T23:27:09.204 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-webob. 2024-05-08T23:27:09.222 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2024-05-08T23:27:09.232 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-08T23:27:09.248 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:09.262 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-05-08T23:27:09.290 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libsqlite3-mod-ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:09.593 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-08T23:27:09.610 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../048-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-05-08T23:27:09.623 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-soupsieve. 2024-05-08T23:27:09.639 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-05-08T23:27:09.642 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-05-08T23:27:09.681 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-05-08T23:27:09.945 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cryptography. 2024-05-08T23:27:09.964 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../049-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-05-08T23:27:10.001 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-bs4. 2024-05-08T23:27:10.003 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-05-08T23:27:10.016 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2024-05-08T23:27:10.059 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-05-08T23:27:10.374 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-openssl. 2024-05-08T23:27:10.393 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../050-python3-openssl_19.0.0-1build1_all.deb ... 2024-05-08T23:27:10.432 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-05-08T23:27:10.445 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-waitress. 2024-05-08T23:27:10.462 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-05-08T23:27:10.507 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-05-08T23:27:10.743 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr. 2024-05-08T23:27:10.760 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../051-ceph-mgr_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:10.793 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:10.857 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-tempita. 2024-05-08T23:27:10.873 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2024-05-08T23:27:10.915 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-05-08T23:27:11.252 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-paste. 2024-05-08T23:27:11.267 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-05-08T23:27:11.310 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-05-08T23:27:11.321 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mon. 2024-05-08T23:27:11.339 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../052-ceph-mon_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:11.379 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mon (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:11.654 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-08T23:27:11.671 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-05-08T23:27:11.713 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-05-08T23:27:12.032 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-08T23:27:12.048 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2024-05-08T23:27:12.090 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-05-08T23:27:12.283 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-osd. 2024-05-08T23:27:12.302 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../053-ceph-osd_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:12.341 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-osd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:12.410 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-webtest. 2024-05-08T23:27:12.429 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2024-05-08T23:27:12.468 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-05-08T23:27:12.813 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-pecan. 2024-05-08T23:27:12.830 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-05-08T23:27:12.862 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-05-08T23:27:13.259 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-08T23:27:13.277 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2024-05-08T23:27:13.317 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2024-05-08T23:27:13.637 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-08T23:27:13.654 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:13.703 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ceph-mgr-modules-core (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:14.100 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-08T23:27:14.115 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:14.158 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libsqlite3-mod-ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:14.477 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph. 2024-05-08T23:27:14.495 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../054-ceph_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:14.528 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-08T23:27:14.535 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:14.548 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../048-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-05-08T23:27:14.586 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-05-08T23:27:14.871 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-fuse. 2024-05-08T23:27:14.887 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../055-ceph-fuse_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:14.897 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-cryptography. 2024-05-08T23:27:14.916 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../049-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-05-08T23:27:14.921 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:14.955 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-05-08T23:27:15.257 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mds. 2024-05-08T23:27:15.276 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-openssl. 2024-05-08T23:27:15.276 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../056-ceph-mds_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:15.292 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../050-python3-openssl_19.0.0-1build1_all.deb ... 2024-05-08T23:27:15.307 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mds (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:15.325 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-05-08T23:27:15.695 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ceph-mgr. 2024-05-08T23:27:15.711 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../051-ceph-mgr_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:15.753 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ceph-mgr (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:15.852 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package cephadm. 2024-05-08T23:27:15.868 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../057-cephadm_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:15.910 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:16.246 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-08T23:27:16.263 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../058-python3-asyncssh_1.12.2-1_all.deb ... 2024-05-08T23:27:16.281 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ceph-mon. 2024-05-08T23:27:16.298 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../052-ceph-mon_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:16.304 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-05-08T23:27:16.339 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ceph-mon (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:16.691 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jinja2. 2024-05-08T23:27:16.708 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2ubuntu0.2_all.deb ... 2024-05-08T23:27:16.749 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.2) ... 2024-05-08T23:27:17.035 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-08T23:27:17.054 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:17.093 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:17.318 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ceph-osd. 2024-05-08T23:27:17.335 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../053-ceph-osd_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:17.376 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ceph-osd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:17.438 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jwt. 2024-05-08T23:27:17.454 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-05-08T23:27:17.496 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-05-08T23:27:17.790 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-attr. 2024-05-08T23:27:17.810 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2024-05-08T23:27:17.849 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-05-08T23:27:18.218 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pluggy. 2024-05-08T23:27:18.236 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2024-05-08T23:27:18.276 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-05-08T23:27:18.621 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-py. 2024-05-08T23:27:18.639 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-05-08T23:27:18.704 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-05-08T23:27:19.030 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-atomicwrites. 2024-05-08T23:27:19.046 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-05-08T23:27:19.088 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-05-08T23:27:19.390 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-packaging. 2024-05-08T23:27:19.408 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2024-05-08T23:27:19.457 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-packaging (20.3-1) ... 2024-05-08T23:27:19.605 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ceph. 2024-05-08T23:27:19.620 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../054-ceph_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:19.654 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:19.726 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pytest. 2024-05-08T23:27:19.746 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2024-05-08T23:27:19.785 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-05-08T23:27:20.024 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ceph-fuse. 2024-05-08T23:27:20.044 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../055-ceph-fuse_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:20.082 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ceph-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:20.121 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-08T23:27:20.138 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2024-05-08T23:27:20.196 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-08T23:27:20.440 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libjs-underscore. 2024-05-08T23:27:20.456 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-05-08T23:27:20.499 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-05-08T23:27:20.510 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ceph-mds. 2024-05-08T23:27:20.529 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../056-ceph-mds_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:20.569 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ceph-mds (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:20.768 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-05-08T23:27:20.784 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-05-08T23:27:20.826 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-05-08T23:27:21.113 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package cephadm. 2024-05-08T23:27:21.130 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../057-cephadm_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:21.171 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:21.187 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-routes. 2024-05-08T23:27:21.204 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2024-05-08T23:27:21.245 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-05-08T23:27:21.516 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-08T23:27:21.534 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../058-python3-asyncssh_1.12.2-1_all.deb ... 2024-05-08T23:27:21.574 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-08T23:27:21.574 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-05-08T23:27:21.589 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:21.632 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-dashboard (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:21.977 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-jinja2. 2024-05-08T23:27:21.995 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2ubuntu0.2_all.deb ... 2024-05-08T23:27:22.035 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.2) ... 2024-05-08T23:27:22.380 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-08T23:27:22.396 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:22.438 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ceph-mgr-cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:22.808 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-jwt. 2024-05-08T23:27:22.824 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-05-08T23:27:22.866 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-05-08T23:27:22.965 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-decorator. 2024-05-08T23:27:22.984 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-05-08T23:27:23.015 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-05-08T23:27:23.194 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-attr. 2024-05-08T23:27:23.213 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2024-05-08T23:27:23.252 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-05-08T23:27:23.318 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-05-08T23:27:23.334 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-05-08T23:27:23.376 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-05-08T23:27:23.613 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-pluggy. 2024-05-08T23:27:23.630 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2024-05-08T23:27:23.671 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-05-08T23:27:23.694 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-scipy. 2024-05-08T23:27:23.712 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-05-08T23:27:23.752 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-05-08T23:27:24.005 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-py. 2024-05-08T23:27:24.025 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-05-08T23:27:24.055 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-05-08T23:27:24.400 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-atomicwrites. 2024-05-08T23:27:24.420 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-05-08T23:27:24.458 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-05-08T23:27:24.761 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-packaging. 2024-05-08T23:27:24.778 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2024-05-08T23:27:24.844 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-packaging (20.3-1) ... 2024-05-08T23:27:25.062 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-05-08T23:27:25.080 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-05-08T23:27:25.120 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-05-08T23:27:25.147 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-pytest. 2024-05-08T23:27:25.163 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2024-05-08T23:27:25.230 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-05-08T23:27:25.567 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-08T23:27:25.583 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2024-05-08T23:27:25.658 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-08T23:27:25.699 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-joblib. 2024-05-08T23:27:25.718 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2024-05-08T23:27:25.757 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-05-08T23:27:25.911 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libjs-underscore. 2024-05-08T23:27:25.930 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-05-08T23:27:25.969 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-05-08T23:27:26.144 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-sklearn. 2024-05-08T23:27:26.160 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-05-08T23:27:26.201 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-05-08T23:27:26.238 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-05-08T23:27:26.255 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-05-08T23:27:26.288 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-05-08T23:27:26.633 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-routes. 2024-05-08T23:27:26.650 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2024-05-08T23:27:26.691 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-05-08T23:27:26.918 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-08T23:27:26.938 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:26.976 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:26.994 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-08T23:27:27.013 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:27.044 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ceph-mgr-dashboard (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:28.112 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cachetools. 2024-05-08T23:27:28.128 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2024-05-08T23:27:28.170 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-05-08T23:27:28.472 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-decorator. 2024-05-08T23:27:28.492 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-05-08T23:27:28.517 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-08T23:27:28.535 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-05-08T23:27:28.555 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-05-08T23:27:28.575 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-05-08T23:27:28.841 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-05-08T23:27:28.861 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-05-08T23:27:28.882 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-08T23:27:28.896 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-05-08T23:27:28.898 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-05-08T23:27:28.929 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-05-08T23:27:29.218 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-scipy. 2024-05-08T23:27:29.233 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-05-08T23:27:29.241 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rsa. 2024-05-08T23:27:29.260 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2024-05-08T23:27:29.267 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-05-08T23:27:29.299 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rsa (4.0-3) ... 2024-05-08T23:27:29.611 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-google-auth. 2024-05-08T23:27:29.630 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2024-05-08T23:27:29.669 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-05-08T23:27:29.990 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-blinker. 2024-05-08T23:27:30.008 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-05-08T23:27:30.048 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-05-08T23:27:30.376 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-oauthlib. 2024-05-08T23:27:30.392 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-05-08T23:27:30.434 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-05-08T23:27:30.560 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-05-08T23:27:30.578 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-05-08T23:27:30.651 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-05-08T23:27:30.746 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-08T23:27:30.766 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-05-08T23:27:30.804 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-05-08T23:27:31.124 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-websocket. 2024-05-08T23:27:31.141 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-05-08T23:27:31.182 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-05-08T23:27:31.230 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-joblib. 2024-05-08T23:27:31.247 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2024-05-08T23:27:31.288 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-05-08T23:27:31.460 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-08T23:27:31.479 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-05-08T23:27:31.510 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-05-08T23:27:31.684 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-sklearn. 2024-05-08T23:27:31.700 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-05-08T23:27:31.741 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-05-08T23:27:32.040 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-08T23:27:32.058 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:32.098 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-k8sevents (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:32.409 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-08T23:27:32.425 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2024-05-08T23:27:32.450 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-08T23:27:32.467 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:32.467 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-05-08T23:27:32.508 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:32.837 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-08T23:27:32.853 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-05-08T23:27:32.895 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-05-08T23:27:33.148 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package jq. 2024-05-08T23:27:33.167 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-05-08T23:27:33.206 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-05-08T23:27:33.467 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package socat. 2024-05-08T23:27:33.487 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2024-05-08T23:27:33.517 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking socat (1.7.3.3-2) ... 2024-05-08T23:27:33.653 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-cachetools. 2024-05-08T23:27:33.670 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2024-05-08T23:27:33.711 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-05-08T23:27:33.820 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package xmlstarlet. 2024-05-08T23:27:33.839 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-05-08T23:27:33.878 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-05-08T23:27:34.124 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-08T23:27:34.140 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-05-08T23:27:34.182 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-05-08T23:27:34.441 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-test. 2024-05-08T23:27:34.457 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../096-ceph-test_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:34.499 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-test (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:34.553 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-08T23:27:34.569 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-05-08T23:27:34.611 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-05-08T23:27:34.940 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-rsa. 2024-05-08T23:27:34.956 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2024-05-08T23:27:34.998 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-rsa (4.0-3) ... 2024-05-08T23:27:35.372 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-google-auth. 2024-05-08T23:27:35.392 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2024-05-08T23:27:35.761 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-05-08T23:27:36.115 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-blinker. 2024-05-08T23:27:36.133 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-05-08T23:27:36.189 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-05-08T23:27:36.509 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-oauthlib. 2024-05-08T23:27:36.529 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-05-08T23:27:36.567 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-05-08T23:27:36.896 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-08T23:27:36.912 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-05-08T23:27:36.971 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-05-08T23:27:37.257 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-websocket. 2024-05-08T23:27:37.277 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-05-08T23:27:37.315 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-05-08T23:27:37.643 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-08T23:27:37.659 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-05-08T23:27:37.701 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-05-08T23:27:38.315 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-08T23:27:38.331 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:38.373 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ceph-mgr-k8sevents (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:38.669 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-volume. 2024-05-08T23:27:38.686 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../097-ceph-volume_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:38.726 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-08T23:27:38.727 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-volume (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:38.743 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2024-05-08T23:27:38.784 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-05-08T23:27:39.179 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-08T23:27:39.195 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-05-08T23:27:39.237 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-05-08T23:27:39.249 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-08T23:27:39.268 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../098-cephfs-mirror_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:39.307 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking cephfs-mirror (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:39.532 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package jq. 2024-05-08T23:27:39.550 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-05-08T23:27:39.589 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-05-08T23:27:39.635 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-colorama. 2024-05-08T23:27:39.655 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2024-05-08T23:27:39.693 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-05-08T23:27:39.867 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package socat. 2024-05-08T23:27:39.883 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2024-05-08T23:27:39.925 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking socat (1.7.3.3-2) ... 2024-05-08T23:27:39.954 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package cephfs-shell. 2024-05-08T23:27:39.970 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../100-cephfs-shell_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:40.012 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking cephfs-shell (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:40.253 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package xmlstarlet. 2024-05-08T23:27:40.269 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-05-08T23:27:40.282 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package cephfs-top. 2024-05-08T23:27:40.300 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../101-cephfs-top_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:40.311 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-05-08T23:27:40.365 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking cephfs-top (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:40.618 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package fonts-mathjax. 2024-05-08T23:27:40.636 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-05-08T23:27:40.676 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-05-08T23:27:40.708 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ceph-test. 2024-05-08T23:27:40.724 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../096-ceph-test_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:40.766 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ceph-test (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:41.247 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-08T23:27:41.265 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../103-libcephfs-dev_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:41.304 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libcephfs-dev (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:41.677 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package nvme-cli. 2024-05-08T23:27:41.698 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-05-08T23:27:41.753 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-05-08T23:27:42.091 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-08T23:27:42.108 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-05-08T23:27:42.149 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-05-08T23:27:42.518 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libjs-mathjax. 2024-05-08T23:27:42.701 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-05-08T23:27:42.701 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-05-08T23:27:44.337 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python-natsort-doc. 2024-05-08T23:27:44.355 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2024-05-08T23:27:44.395 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-05-08T23:27:44.714 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-webencodings. 2024-05-08T23:27:44.735 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-05-08T23:27:44.772 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-05-08T23:27:44.929 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ceph-volume. 2024-05-08T23:27:44.947 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../097-ceph-volume_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:45.012 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ceph-volume (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:45.100 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-html5lib. 2024-05-08T23:27:45.118 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2024-05-08T23:27:45.158 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-05-08T23:27:45.367 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-08T23:27:45.386 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../098-cephfs-mirror_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:45.441 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking cephfs-mirror (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:45.487 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-05-08T23:27:45.504 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-05-08T23:27:45.544 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-05-08T23:27:45.769 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-colorama. 2024-05-08T23:27:45.787 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2024-05-08T23:27:45.861 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-05-08T23:27:46.006 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pastescript. 2024-05-08T23:27:46.023 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2024-05-08T23:27:46.064 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-08T23:27:46.122 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package cephfs-shell. 2024-05-08T23:27:46.139 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../100-cephfs-shell_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:46.189 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking cephfs-shell (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:46.393 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-psutil. 2024-05-08T23:27:46.410 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-05-08T23:27:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-05-08T23:27:46.485 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package cephfs-top. 2024-05-08T23:27:46.500 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../101-cephfs-top_18.2.2-1322-gd21e4f96-1focal_all.deb ... 2024-05-08T23:27:46.541 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking cephfs-top (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:46.804 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-08T23:27:46.810 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package fonts-mathjax. 2024-05-08T23:27:46.821 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-05-08T23:27:46.829 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-05-08T23:27:46.861 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-05-08T23:27:46.869 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-05-08T23:27:47.173 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-simplejson. 2024-05-08T23:27:47.189 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-05-08T23:27:47.230 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-05-08T23:27:47.397 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-08T23:27:47.415 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../103-libcephfs-dev_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:47.455 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libcephfs-dev (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:47.509 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-08T23:27:47.525 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-05-08T23:27:47.566 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-05-08T23:27:47.862 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package nvme-cli. 2024-05-08T23:27:47.879 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-05-08T23:27:47.963 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-05-08T23:27:48.267 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-08T23:27:48.278 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package radosgw. 2024-05-08T23:27:48.285 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-05-08T23:27:48.297 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../116-radosgw_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:48.337 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking radosgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:48.342 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-05-08T23:27:48.732 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libjs-mathjax. 2024-05-08T23:27:48.751 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-05-08T23:27:48.794 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-05-08T23:27:49.790 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package rbd-fuse. 2024-05-08T23:27:49.809 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../117-rbd-fuse_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:49.848 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking rbd-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:50.133 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-attr (19.3.0-2) ... 2024-05-08T23:27:50.428 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-05-08T23:27:50.538 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python-natsort-doc. 2024-05-08T23:27:50.553 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-05-08T23:27:50.559 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2024-05-08T23:27:50.604 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-05-08T23:27:50.680 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-05-08T23:27:50.881 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:50.915 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-webencodings. 2024-05-08T23:27:50.934 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-05-08T23:27:50.990 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-05-08T23:27:51.327 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-html5lib. 2024-05-08T23:27:51.344 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2024-05-08T23:27:51.384 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-05-08T23:27:51.440 INFO:teuthology.orchestra.run.smithi117.stdout:Adding system user cephadm....done 2024-05-08T23:27:51.604 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-05-08T23:27:51.746 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-05-08T23:27:51.754 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-05-08T23:27:51.772 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-05-08T23:27:51.812 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-05-08T23:27:52.106 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-08T23:27:52.308 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-pastescript. 2024-05-08T23:27:52.326 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2024-05-08T23:27:52.365 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-08T23:27:52.399 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-05-08T23:27:52.525 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-05-08T23:27:52.676 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-05-08T23:27:52.713 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-psutil. 2024-05-08T23:27:52.731 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-05-08T23:27:52.771 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-05-08T23:27:52.986 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-05-08T23:27:53.132 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-08T23:27:53.149 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-05-08T23:27:53.190 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-05-08T23:27:53.408 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-05-08T23:27:53.510 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package python3-simplejson. 2024-05-08T23:27:53.527 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-05-08T23:27:53.568 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-05-08T23:27:53.731 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-05-08T23:27:53.862 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-08T23:27:53.880 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-05-08T23:27:53.920 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-05-08T23:27:54.015 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-05-08T23:27:54.300 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-05-08T23:27:54.585 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-05-08T23:27:54.708 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package radosgw. 2024-05-08T23:27:54.725 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../116-radosgw_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:54.766 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking radosgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:54.877 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-ceph-argparse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:55.187 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-05-08T23:27:55.312 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-05-08T23:27:55.597 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-05-08T23:27:55.873 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-05-08T23:27:56.149 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-05-08T23:27:56.152 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package rbd-fuse. 2024-05-08T23:27:56.172 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../117-rbd-fuse_18.2.2-1322-gd21e4f96-1focal_amd64.deb ... 2024-05-08T23:27:56.210 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking rbd-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:56.283 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-05-08T23:27:56.562 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-attr (19.3.0-2) ... 2024-05-08T23:27:56.668 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-05-08T23:27:56.857 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-05-08T23:27:56.960 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-05-08T23:27:56.983 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-05-08T23:27:57.134 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-05-08T23:27:57.246 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-05-08T23:27:57.319 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:27:57.579 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-05-08T23:27:57.847 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.2) ... 2024-05-08T23:27:57.895 INFO:teuthology.orchestra.run.smithi081.stdout:Adding system user cephadm....done 2024-05-08T23:27:58.084 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-05-08T23:27:58.193 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-05-08T23:27:58.207 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-05-08T23:27:58.477 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-packaging (20.3-1) ... 2024-05-08T23:27:58.553 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-08T23:27:58.745 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-05-08T23:27:58.846 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-05-08T23:27:58.904 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-05-08T23:27:58.972 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-05-08T23:27:59.030 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-05-08T23:27:59.098 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-05-08T23:27:59.315 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-05-08T23:27:59.441 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-05-08T23:27:59.708 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-05-08T23:27:59.843 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-05-08T23:28:00.009 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2024-05-08T23:28:00.120 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-05-08T23:28:00.377 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-05-08T23:28:00.445 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-05-08T23:28:00.737 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up socat (1.7.3.3-2) ... 2024-05-08T23:28:00.739 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-05-08T23:28:00.863 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:01.015 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-05-08T23:28:01.189 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T23:28:01.306 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-05-08T23:28:01.325 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-ceph-argparse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:01.626 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-05-08T23:28:01.644 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-05-08T23:28:01.762 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-05-08T23:28:01.770 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-05-08T23:28:01.921 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-05-08T23:28:02.036 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-05-08T23:28:02.197 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-05-08T23:28:02.354 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-05-08T23:28:02.523 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T23:28:02.639 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-05-08T23:28:02.665 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-05-08T23:28:02.765 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-05-08T23:28:02.792 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-05-08T23:28:03.110 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-05-08T23:28:03.141 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-05-08T23:28:03.419 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-05-08T23:28:03.442 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-05-08T23:28:03.718 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-05-08T23:28:03.720 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-05-08T23:28:03.847 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-08T23:28:04.078 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-05-08T23:28:04.115 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-05-08T23:28:04.415 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.2) ... 2024-05-08T23:28:04.441 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-05-08T23:28:04.749 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-05-08T23:28:05.026 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-05-08T23:28:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-packaging (20.3-1) ... 2024-05-08T23:28:05.347 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-05-08T23:28:05.400 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-05-08T23:28:05.478 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-05-08T23:28:05.612 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-05-08T23:28:05.653 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-05-08T23:28:05.821 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-05-08T23:28:05.905 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-05-08T23:28:06.139 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-05-08T23:28:06.339 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-05-08T23:28:06.481 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-05-08T23:28:06.633 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-05-08T23:28:06.759 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-05-08T23:28:07.007 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2024-05-08T23:28:07.085 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-05-08T23:28:07.369 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-05-08T23:28:07.384 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-05-08T23:28:07.752 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up socat (1.7.3.3-2) ... 2024-05-08T23:28:07.878 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:08.199 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T23:28:08.333 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-05-08T23:28:08.710 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-05-08T23:28:08.836 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-05-08T23:28:08.860 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-05-08T23:28:08.967 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-05-08T23:28:08.995 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-05-08T23:28:09.296 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-05-08T23:28:09.336 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-05-08T23:28:09.628 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-05-08T23:28:09.631 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T23:28:09.782 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-05-08T23:28:09.908 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-05-08T23:28:09.921 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T23:28:10.047 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-05-08T23:28:10.234 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-05-08T23:28:10.390 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-05-08T23:28:10.537 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-05-08T23:28:10.683 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-05-08T23:28:10.822 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-05-08T23:28:10.981 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-08T23:28:11.060 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-05-08T23:28:11.265 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-05-08T23:28:11.419 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-05-08T23:28:11.583 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-05-08T23:28:11.884 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-05-08T23:28:12.276 INFO:teuthology.orchestra.run.smithi081.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-05-08T23:28:12.474 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-05-08T23:28:12.537 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-05-08T23:28:12.600 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-05-08T23:28:12.688 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-05-08T23:28:12.743 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-05-08T23:28:12.860 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rsa (4.0-3) ... 2024-05-08T23:28:13.022 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-05-08T23:28:13.145 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-05-08T23:28:13.271 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-05-08T23:28:13.424 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-05-08T23:28:13.583 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-05-08T23:28:13.676 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-05-08T23:28:13.776 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-05-08T23:28:13.848 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-08T23:28:14.018 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-05-08T23:28:14.166 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-05-08T23:28:14.320 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-05-08T23:28:14.551 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librados2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:14.763 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-05-08T23:28:14.889 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librgw2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:15.048 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libsqlite3-mod-ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:15.174 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libcephfs2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:15.300 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-routes (2.4.1-2) ... 2024-05-08T23:28:15.602 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libradosstriper1 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:15.711 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-05-08T23:28:15.767 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-05-08T23:28:15.918 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-05-08T23:28:16.037 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librbd1 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:16.163 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-modules-core (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:16.289 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:16.319 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-05-08T23:28:16.515 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-08T23:28:16.515 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-08T23:28:16.621 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-05-08T23:28:16.905 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-05-08T23:28:16.966 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libcephfs-dev (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:17.031 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-05-08T23:28:17.092 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rados (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:17.218 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up cephfs-top (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:17.352 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-05-08T23:28:17.374 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-05-08T23:28:17.709 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-05-08T23:28:18.094 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-05-08T23:28:18.248 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rbd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:18.357 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up rbd-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:18.446 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-05-08T23:28:18.458 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:18.567 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cephfs (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:18.693 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up cephfs-shell (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:18.845 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:19.391 INFO:teuthology.orchestra.run.smithi117.stdout:Adding group ceph....done 2024-05-08T23:28:19.459 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-05-08T23:28:19.585 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-05-08T23:28:19.736 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-05-08T23:28:19.809 INFO:teuthology.orchestra.run.smithi117.stdout:Adding system user ceph....done 2024-05-08T23:28:19.871 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-rsa (4.0-3) ... 2024-05-08T23:28:19.977 INFO:teuthology.orchestra.run.smithi117.stdout:Setting system user ceph properties....done 2024-05-08T23:28:19.996 INFO:teuthology.orchestra.run.smithi117.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-08T23:28:20.150 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-08T23:28:20.155 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-05-08T23:28:20.281 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-05-08T23:28:20.480 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-08T23:28:20.582 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-05-08T23:28:20.900 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-08T23:28:20.940 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up cephfs-mirror (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:21.145 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-08T23:28:21.146 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-08T23:28:21.210 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-05-08T23:28:21.590 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-test (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:21.595 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up librados2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:21.717 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up radosgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:21.823 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-05-08T23:28:21.966 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up librgw2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:22.092 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libsqlite3-mod-ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:22.192 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-08T23:28:22.192 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-08T23:28:22.243 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libcephfs2 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:22.369 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-routes (2.4.1-2) ... 2024-05-08T23:28:22.671 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libradosstriper1 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:22.685 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-base (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:22.780 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-05-08T23:28:23.070 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-08T23:28:23.074 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up librbd1 (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:23.200 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ceph-mgr-modules-core (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:23.326 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ceph-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:23.520 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mds (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:23.581 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-08T23:28:23.581 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-08T23:28:23.752 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-08T23:28:23.752 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-08T23:28:24.052 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libcephfs-dev (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:24.162 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-rados (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:24.221 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:24.313 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up cephfs-top (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:24.439 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-05-08T23:28:24.448 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-08T23:28:24.448 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-08T23:28:24.889 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-osd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:25.343 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-rbd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:25.409 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-08T23:28:25.409 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-08T23:28:25.485 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up rbd-fuse (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:25.586 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-rgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:25.687 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up python3-cephfs (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:25.797 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up cephfs-shell (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:25.868 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-k8sevents (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:25.957 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ceph-common (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:25.994 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-diskprediction-local (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:26.137 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mon (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:26.366 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-08T23:28:26.366 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-08T23:28:26.526 INFO:teuthology.orchestra.run.smithi081.stdout:Adding group ceph....done 2024-05-08T23:28:26.813 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:26.886 INFO:teuthology.orchestra.run.smithi081.stdout:Adding system user ceph....done 2024-05-08T23:28:26.955 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:27.057 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-dashboard (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:27.067 INFO:teuthology.orchestra.run.smithi081.stdout:Setting system user ceph properties....done 2024-05-08T23:28:27.085 INFO:teuthology.orchestra.run.smithi081.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-08T23:28:27.191 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-volume (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:27.238 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-08T23:28:27.543 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-08T23:28:27.594 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-05-08T23:28:27.745 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-05-08T23:28:28.008 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up cephfs-mirror (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:28.072 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-05-08T23:28:28.236 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-08T23:28:28.237 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-08T23:28:28.676 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ceph-test (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:28.827 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up radosgw (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:29.287 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-08T23:28:29.287 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-08T23:28:29.713 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ceph-base (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:30.090 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-08T23:28:30.557 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ceph-mds (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:30.796 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-08T23:28:30.796 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-08T23:28:30.933 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-05-08T23:28:31.242 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ceph-mgr (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:31.496 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-08T23:28:31.497 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-08T23:28:31.573 INFO:teuthology.orchestra.run.smithi117.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-08T23:28:31.575 DEBUG:teuthology.orchestra.run.smithi117:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-08T23:28:31.645 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-05-08T23:28:31.856 INFO:teuthology.orchestra.run.smithi117.stdout:Building dependency tree... 2024-05-08T23:28:31.857 INFO:teuthology.orchestra.run.smithi117.stdout:Reading state information... 2024-05-08T23:28:31.944 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ceph-osd (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:32.066 INFO:teuthology.orchestra.run.smithi117.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-05-08T23:28:32.066 INFO:teuthology.orchestra.run.smithi117.stdout:flex is already the newest version (2.6.4-6.2). 2024-05-08T23:28:32.066 INFO:teuthology.orchestra.run.smithi117.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-05-08T23:28:32.066 INFO:teuthology.orchestra.run.smithi117.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-05-08T23:28:32.066 INFO:teuthology.orchestra.run.smithi117.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-05-08T23:28:32.067 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages were automatically installed and are no longer required: 2024-05-08T23:28:32.067 INFO:teuthology.orchestra.run.smithi117.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-05-08T23:28:32.067 INFO:teuthology.orchestra.run.smithi117.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-08T23:28:32.067 INFO:teuthology.orchestra.run.smithi117.stdout:The following additional packages will be installed: 2024-05-08T23:28:32.067 INFO:teuthology.orchestra.run.smithi117.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-05-08T23:28:32.068 INFO:teuthology.orchestra.run.smithi117.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2024-05-08T23:28:32.068 INFO:teuthology.orchestra.run.smithi117.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2024-05-08T23:28:32.068 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2024-05-08T23:28:32.068 INFO:teuthology.orchestra.run.smithi117.stdout: usb-modeswitch-data wpasupplicant 2024-05-08T23:28:32.069 INFO:teuthology.orchestra.run.smithi117.stdout:Suggested packages: 2024-05-08T23:28:32.069 INFO:teuthology.orchestra.run.smithi117.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2024-05-08T23:28:32.069 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2024-05-08T23:28:32.069 INFO:teuthology.orchestra.run.smithi117.stdout: libengine-pkcs11-openssl 2024-05-08T23:28:32.130 INFO:teuthology.orchestra.run.smithi117.stdout:The following NEW packages will be installed: 2024-05-08T23:28:32.130 INFO:teuthology.orchestra.run.smithi117.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-05-08T23:28:32.131 INFO:teuthology.orchestra.run.smithi117.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2024-05-08T23:28:32.131 INFO:teuthology.orchestra.run.smithi117.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-05-08T23:28:32.131 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2024-05-08T23:28:32.132 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2024-05-08T23:28:32.132 INFO:teuthology.orchestra.run.smithi117.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-08T23:28:32.132 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages will be upgraded: 2024-05-08T23:28:32.133 INFO:teuthology.orchestra.run.smithi117.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-05-08T23:28:32.317 INFO:teuthology.orchestra.run.smithi117.stdout:6 upgraded, 31 newly installed, 0 to remove and 272 not upgraded. 2024-05-08T23:28:32.317 INFO:teuthology.orchestra.run.smithi117.stdout:Need to get 25.3 MB of archives. 2024-05-08T23:28:32.317 INFO:teuthology.orchestra.run.smithi117.stdout:After this operation, 79.9 MB of additional disk space will be used. 2024-05-08T23:28:32.317 INFO:teuthology.orchestra.run.smithi117.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-05-08T23:28:32.435 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-08T23:28:32.435 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-08T23:28:32.863 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ceph-mgr-k8sevents (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:32.939 INFO:teuthology.orchestra.run.smithi117.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-05-08T23:28:32.941 INFO:teuthology.orchestra.run.smithi117.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-05-08T23:28:32.942 INFO:teuthology.orchestra.run.smithi117.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-05-08T23:28:32.955 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-05-08T23:28:32.957 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.22 [1,323 kB] 2024-05-08T23:28:33.014 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ceph-mgr-diskprediction-local (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:33.055 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-05-08T23:28:33.057 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-05-08T23:28:33.064 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-05-08T23:28:33.069 INFO:teuthology.orchestra.run.smithi117.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-05-08T23:28:33.070 INFO:teuthology.orchestra.run.smithi117.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.7 [60.7 kB] 2024-05-08T23:28:33.157 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ceph-mon (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:33.237 INFO:teuthology.orchestra.run.smithi117.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-05-08T23:28:33.385 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-08T23:28:33.385 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-08T23:28:33.515 INFO:teuthology.orchestra.run.smithi117.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-05-08T23:28:33.645 INFO:teuthology.orchestra.run.smithi117.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-05-08T23:28:33.649 INFO:teuthology.orchestra.run.smithi117.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-05-08T23:28:33.745 INFO:teuthology.orchestra.run.smithi117.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2024-05-08T23:28:33.748 INFO:teuthology.orchestra.run.smithi117.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-05-08T23:28:33.831 INFO:teuthology.orchestra.run.smithi117.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.18-0ubuntu0.20.04.1 [116 kB] 2024-05-08T23:28:33.848 INFO:teuthology.orchestra.run.smithi117.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-05-08T23:28:33.849 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ceph-mgr-cephadm (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:33.917 INFO:teuthology.orchestra.run.smithi117.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-05-08T23:28:33.918 INFO:teuthology.orchestra.run.smithi117.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.22 [1,586 kB] 2024-05-08T23:28:33.992 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ceph (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:34.001 INFO:teuthology.orchestra.run.smithi117.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-05-08T23:28:34.118 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ceph-mgr-dashboard (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:34.165 INFO:teuthology.orchestra.run.smithi117.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-05-08T23:28:34.286 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ceph-volume (18.2.2-1322-gd21e4f96-1focal) ... 2024-05-08T23:28:34.689 INFO:teuthology.orchestra.run.smithi081.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-05-08T23:28:34.779 INFO:teuthology.orchestra.run.smithi117.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2024-05-08T23:28:34.832 INFO:teuthology.orchestra.run.smithi081.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-05-08T23:28:34.878 INFO:teuthology.orchestra.run.smithi117.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-05-08T23:28:34.967 INFO:teuthology.orchestra.run.smithi117.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-05-08T23:28:34.976 INFO:teuthology.orchestra.run.smithi117.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-05-08T23:28:34.977 INFO:teuthology.orchestra.run.smithi117.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-05-08T23:28:34.978 INFO:teuthology.orchestra.run.smithi117.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2024-05-08T23:28:34.979 INFO:teuthology.orchestra.run.smithi117.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.18-0ubuntu0.20.04.1 [1,054 kB] 2024-05-08T23:28:35.012 INFO:teuthology.orchestra.run.smithi117.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2024-05-08T23:28:35.016 INFO:teuthology.orchestra.run.smithi117.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.18-0ubuntu0.20.04.1 [13.5 MB] 2024-05-08T23:28:35.134 INFO:teuthology.orchestra.run.smithi081.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-05-08T23:28:35.435 INFO:teuthology.orchestra.run.smithi117.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2024-05-08T23:28:35.635 INFO:teuthology.orchestra.run.smithi117.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2024-05-08T23:28:35.725 INFO:teuthology.orchestra.run.smithi117.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2024-05-08T23:28:35.726 INFO:teuthology.orchestra.run.smithi117.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-05-08T23:28:35.859 INFO:teuthology.orchestra.run.smithi117.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-05-08T23:28:36.358 INFO:teuthology.orchestra.run.smithi117.stdout:Preconfiguring packages ... 2024-05-08T23:28:36.559 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 25.3 MB in 4s (6,648 kB/s) 2024-05-08T23:28:36.834 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153098 files and directories currently installed.) 2024-05-08T23:28:36.839 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-05-08T23:28:36.993 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-05-08T23:28:38.120 INFO:teuthology.orchestra.run.smithi081.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-05-08T23:28:38.124 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-05-08T23:28:38.742 INFO:teuthology.orchestra.run.smithi081.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-08T23:28:38.745 DEBUG:teuthology.orchestra.run.smithi081:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-08T23:28:38.814 INFO:teuthology.orchestra.run.smithi081.stdout:Reading package lists... 2024-05-08T23:28:39.027 INFO:teuthology.orchestra.run.smithi081.stdout:Building dependency tree... 2024-05-08T23:28:39.028 INFO:teuthology.orchestra.run.smithi081.stdout:Reading state information... 2024-05-08T23:28:39.239 INFO:teuthology.orchestra.run.smithi081.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-05-08T23:28:39.240 INFO:teuthology.orchestra.run.smithi081.stdout:flex is already the newest version (2.6.4-6.2). 2024-05-08T23:28:39.240 INFO:teuthology.orchestra.run.smithi081.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-05-08T23:28:39.240 INFO:teuthology.orchestra.run.smithi081.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-05-08T23:28:39.240 INFO:teuthology.orchestra.run.smithi081.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-05-08T23:28:39.240 INFO:teuthology.orchestra.run.smithi081.stdout:The following packages were automatically installed and are no longer required: 2024-05-08T23:28:39.240 INFO:teuthology.orchestra.run.smithi081.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-05-08T23:28:39.240 INFO:teuthology.orchestra.run.smithi081.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-08T23:28:39.241 INFO:teuthology.orchestra.run.smithi081.stdout:The following additional packages will be installed: 2024-05-08T23:28:39.241 INFO:teuthology.orchestra.run.smithi081.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-05-08T23:28:39.241 INFO:teuthology.orchestra.run.smithi081.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2024-05-08T23:28:39.242 INFO:teuthology.orchestra.run.smithi081.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2024-05-08T23:28:39.242 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2024-05-08T23:28:39.242 INFO:teuthology.orchestra.run.smithi081.stdout: usb-modeswitch-data wpasupplicant 2024-05-08T23:28:39.243 INFO:teuthology.orchestra.run.smithi081.stdout:Suggested packages: 2024-05-08T23:28:39.243 INFO:teuthology.orchestra.run.smithi081.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2024-05-08T23:28:39.243 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2024-05-08T23:28:39.243 INFO:teuthology.orchestra.run.smithi081.stdout: libengine-pkcs11-openssl 2024-05-08T23:28:39.304 INFO:teuthology.orchestra.run.smithi081.stdout:The following NEW packages will be installed: 2024-05-08T23:28:39.304 INFO:teuthology.orchestra.run.smithi081.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-05-08T23:28:39.304 INFO:teuthology.orchestra.run.smithi081.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2024-05-08T23:28:39.305 INFO:teuthology.orchestra.run.smithi081.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-05-08T23:28:39.305 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2024-05-08T23:28:39.305 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2024-05-08T23:28:39.305 INFO:teuthology.orchestra.run.smithi081.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-08T23:28:39.305 INFO:teuthology.orchestra.run.smithi081.stdout:The following packages will be upgraded: 2024-05-08T23:28:39.307 INFO:teuthology.orchestra.run.smithi081.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-05-08T23:28:39.491 INFO:teuthology.orchestra.run.smithi081.stdout:6 upgraded, 31 newly installed, 0 to remove and 272 not upgraded. 2024-05-08T23:28:39.491 INFO:teuthology.orchestra.run.smithi081.stdout:Need to get 25.3 MB of archives. 2024-05-08T23:28:39.491 INFO:teuthology.orchestra.run.smithi081.stdout:After this operation, 79.9 MB of additional disk space will be used. 2024-05-08T23:28:39.491 INFO:teuthology.orchestra.run.smithi081.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-05-08T23:28:39.510 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153098 files and directories currently installed.) 2024-05-08T23:28:39.515 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-05-08T23:28:39.673 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-05-08T23:28:40.071 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-05-08T23:28:40.118 INFO:teuthology.orchestra.run.smithi081.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-05-08T23:28:40.120 INFO:teuthology.orchestra.run.smithi081.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-05-08T23:28:40.121 INFO:teuthology.orchestra.run.smithi081.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-05-08T23:28:40.133 INFO:teuthology.orchestra.run.smithi081.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-05-08T23:28:40.136 INFO:teuthology.orchestra.run.smithi081.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.22 [1,323 kB] 2024-05-08T23:28:40.234 INFO:teuthology.orchestra.run.smithi081.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-05-08T23:28:40.237 INFO:teuthology.orchestra.run.smithi081.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-05-08T23:28:40.237 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-05-08T23:28:40.244 INFO:teuthology.orchestra.run.smithi081.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-05-08T23:28:40.249 INFO:teuthology.orchestra.run.smithi081.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-05-08T23:28:40.249 INFO:teuthology.orchestra.run.smithi081.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.7 [60.7 kB] 2024-05-08T23:28:40.416 INFO:teuthology.orchestra.run.smithi081.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-05-08T23:28:40.608 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-05-08T23:28:40.697 INFO:teuthology.orchestra.run.smithi081.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-05-08T23:28:40.826 INFO:teuthology.orchestra.run.smithi081.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-05-08T23:28:40.832 INFO:teuthology.orchestra.run.smithi081.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-05-08T23:28:40.927 INFO:teuthology.orchestra.run.smithi081.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2024-05-08T23:28:40.931 INFO:teuthology.orchestra.run.smithi081.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-05-08T23:28:41.014 INFO:teuthology.orchestra.run.smithi081.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.18-0ubuntu0.20.04.1 [116 kB] 2024-05-08T23:28:41.030 INFO:teuthology.orchestra.run.smithi081.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-05-08T23:28:41.098 INFO:teuthology.orchestra.run.smithi081.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-05-08T23:28:41.098 INFO:teuthology.orchestra.run.smithi081.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.22 [1,586 kB] 2024-05-08T23:28:41.182 INFO:teuthology.orchestra.run.smithi081.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-05-08T23:28:41.242 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153098 files and directories currently installed.) 2024-05-08T23:28:41.246 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-05-08T23:28:41.346 INFO:teuthology.orchestra.run.smithi081.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-05-08T23:28:41.379 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-05-08T23:28:41.826 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-05-08T23:28:41.954 INFO:teuthology.orchestra.run.smithi081.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2024-05-08T23:28:41.984 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-05-08T23:28:42.053 INFO:teuthology.orchestra.run.smithi081.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-05-08T23:28:42.142 INFO:teuthology.orchestra.run.smithi081.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-05-08T23:28:42.152 INFO:teuthology.orchestra.run.smithi081.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-05-08T23:28:42.153 INFO:teuthology.orchestra.run.smithi081.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-05-08T23:28:42.154 INFO:teuthology.orchestra.run.smithi081.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2024-05-08T23:28:42.155 INFO:teuthology.orchestra.run.smithi081.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.18-0ubuntu0.20.04.1 [1,054 kB] 2024-05-08T23:28:42.187 INFO:teuthology.orchestra.run.smithi081.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2024-05-08T23:28:42.192 INFO:teuthology.orchestra.run.smithi081.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.18-0ubuntu0.20.04.1 [13.5 MB] 2024-05-08T23:28:42.439 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.22_amd64.deb ... 2024-05-08T23:28:42.556 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.22) over (1.1.1f-1ubuntu2.16) ... 2024-05-08T23:28:42.564 INFO:teuthology.orchestra.run.smithi081.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2024-05-08T23:28:42.806 INFO:teuthology.orchestra.run.smithi081.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2024-05-08T23:28:42.889 INFO:teuthology.orchestra.run.smithi081.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2024-05-08T23:28:42.889 INFO:teuthology.orchestra.run.smithi081.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-05-08T23:28:43.001 INFO:teuthology.orchestra.run.smithi081.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-05-08T23:28:43.059 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-08T23:28:43.077 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-05-08T23:28:43.117 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-05-08T23:28:43.470 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package dump. 2024-05-08T23:28:43.489 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-05-08T23:28:43.499 INFO:teuthology.orchestra.run.smithi081.stdout:Preconfiguring packages ... 2024-05-08T23:28:43.528 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking dump (0.4b46-6) ... 2024-05-08T23:28:43.695 INFO:teuthology.orchestra.run.smithi081.stdout:Fetched 25.3 MB in 4s (6,707 kB/s) 2024-05-08T23:28:43.832 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package indent. 2024-05-08T23:28:43.849 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-05-08T23:28:43.891 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-05-08T23:28:43.996 INFO:teuthology.orchestra.run.smithi081.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153098 files and directories currently installed.) 2024-05-08T23:28:44.001 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-05-08T23:28:44.166 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-05-08T23:28:44.243 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-08T23:28:44.261 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-05-08T23:28:44.301 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-05-08T23:28:44.588 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-08T23:28:44.608 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.7_amd64.deb ... 2024-05-08T23:28:44.638 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.7) ... 2024-05-08T23:28:44.899 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-08T23:28:44.917 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-05-08T23:28:44.965 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T23:28:45.318 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-08T23:28:45.337 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T23:28:45.355 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-05-08T23:28:45.376 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T23:28:45.654 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-08T23:28:45.670 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T23:28:45.737 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T23:28:46.057 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-08T23:28:46.074 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T23:28:46.115 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T23:28:46.586 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-08T23:28:46.603 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2024-05-08T23:28:46.645 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2024-05-08T23:28:46.784 INFO:teuthology.orchestra.run.smithi081.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153098 files and directories currently installed.) 2024-05-08T23:28:46.789 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-05-08T23:28:46.946 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-05-08T23:28:46.955 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-08T23:28:46.976 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-05-08T23:28:47.013 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-05-08T23:28:47.379 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-05-08T23:28:47.425 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-08T23:28:47.442 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../14-libpq5_12.18-0ubuntu0.20.04.1_amd64.deb ... 2024-05-08T23:28:47.483 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libpq5:amd64 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T23:28:47.544 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-05-08T23:28:47.819 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-08T23:28:47.839 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T23:28:47.871 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-05-08T23:28:47.878 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T23:28:48.230 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-08T23:28:48.248 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T23:28:48.288 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T23:28:48.532 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-08T23:28:48.549 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.22_amd64.deb ... 2024-05-08T23:28:48.564 INFO:teuthology.orchestra.run.smithi081.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153098 files and directories currently installed.) 2024-05-08T23:28:48.569 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-05-08T23:28:48.590 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.22) ... 2024-05-08T23:28:48.703 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-05-08T23:28:49.094 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-08T23:28:49.110 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2024-05-08T23:28:49.134 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-05-08T23:28:49.152 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-05-08T23:28:49.267 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-05-08T23:28:49.547 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package modemmanager. 2024-05-08T23:28:49.567 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T23:28:49.615 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T23:28:49.805 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.22_amd64.deb ... 2024-05-08T23:28:49.922 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.22) over (1.1.1f-1ubuntu2.16) ... 2024-05-08T23:28:50.219 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package wpasupplicant. 2024-05-08T23:28:50.237 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2024-05-08T23:28:50.277 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2024-05-08T23:28:50.434 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-08T23:28:50.452 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-05-08T23:28:50.492 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-05-08T23:28:50.833 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package network-manager. 2024-05-08T23:28:50.849 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-05-08T23:28:50.853 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package dump. 2024-05-08T23:28:50.871 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-05-08T23:28:50.914 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking dump (0.4b46-6) ... 2024-05-08T23:28:50.915 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-05-08T23:28:51.226 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package indent. 2024-05-08T23:28:51.243 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-05-08T23:28:51.284 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-05-08T23:28:51.629 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-08T23:28:51.647 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-05-08T23:28:51.687 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-05-08T23:28:51.899 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ppp. 2024-05-08T23:28:51.917 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-05-08T23:28:52.032 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-08T23:28:52.049 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.7_amd64.deb ... 2024-05-08T23:28:52.091 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.7) ... 2024-05-08T23:28:52.187 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-05-08T23:28:52.393 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-08T23:28:52.410 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-05-08T23:28:52.452 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T23:28:52.521 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package pptp-linux. 2024-05-08T23:28:52.538 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-05-08T23:28:52.579 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-05-08T23:28:52.830 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-08T23:28:52.847 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T23:28:52.888 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T23:28:52.949 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-08T23:28:52.968 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-05-08T23:28:53.025 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-05-08T23:28:53.191 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-08T23:28:53.208 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T23:28:53.249 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T23:28:53.285 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-08T23:28:53.302 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2024-05-08T23:28:53.343 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2024-05-08T23:28:53.664 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package postgresql-client-12. 2024-05-08T23:28:53.681 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../26-postgresql-client-12_12.18-0ubuntu0.20.04.1_amd64.deb ... 2024-05-08T23:28:53.722 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking postgresql-client-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T23:28:53.754 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-08T23:28:53.772 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T23:28:53.812 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T23:28:54.207 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-08T23:28:54.226 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2024-05-08T23:28:54.265 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2024-05-08T23:28:54.310 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package postgresql-common. 2024-05-08T23:28:54.327 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2024-05-08T23:28:54.392 INFO:teuthology.orchestra.run.smithi117.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-08T23:28:54.420 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2024-05-08T23:28:54.627 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-08T23:28:54.646 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-05-08T23:28:54.685 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-05-08T23:28:54.910 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package postgresql-12. 2024-05-08T23:28:54.930 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../28-postgresql-12_12.18-0ubuntu0.20.04.1_amd64.deb ... 2024-05-08T23:28:54.978 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking postgresql-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T23:28:55.050 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-08T23:28:55.066 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../14-libpq5_12.18-0ubuntu0.20.04.1_amd64.deb ... 2024-05-08T23:28:55.104 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libpq5:amd64 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T23:28:55.474 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-08T23:28:55.492 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T23:28:55.557 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T23:28:55.919 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-08T23:28:55.936 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T23:28:56.002 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T23:28:56.254 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-08T23:28:56.272 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.22_amd64.deb ... 2024-05-08T23:28:56.312 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.22) ... 2024-05-08T23:28:56.611 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package postgresql. 2024-05-08T23:28:56.632 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2024-05-08T23:28:56.799 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2024-05-08T23:28:56.866 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-08T23:28:56.884 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2024-05-08T23:28:56.925 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-05-08T23:28:57.096 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package postgresql-client. 2024-05-08T23:28:57.113 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2024-05-08T23:28:57.154 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2024-05-08T23:28:57.328 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package modemmanager. 2024-05-08T23:28:57.344 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-05-08T23:28:57.396 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T23:28:57.415 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-08T23:28:57.432 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2024-05-08T23:28:57.473 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2024-05-08T23:28:57.717 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-08T23:28:57.736 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2024-05-08T23:28:57.775 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-05-08T23:28:58.059 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package wpasupplicant. 2024-05-08T23:28:58.077 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2024-05-08T23:28:58.117 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2024-05-08T23:28:58.178 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-08T23:28:58.198 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-05-08T23:28:58.247 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-05-08T23:28:58.548 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2024-05-08T23:28:58.706 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package network-manager. 2024-05-08T23:28:58.724 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-05-08T23:28:58.789 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-05-08T23:28:58.792 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.22) ... 2024-05-08T23:28:59.085 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libpq5:amd64 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T23:28:59.211 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-05-08T23:28:59.337 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-05-08T23:28:59.463 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2024-05-08T23:28:59.632 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package ppp. 2024-05-08T23:28:59.651 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-05-08T23:28:59.883 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-08T23:28:59.883 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-08T23:28:59.936 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-05-08T23:29:00.312 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package pptp-linux. 2024-05-08T23:29:00.329 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-05-08T23:29:00.358 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up postgresql-client-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T23:29:00.370 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-05-08T23:29:00.765 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-08T23:29:00.786 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-05-08T23:29:00.840 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-05-08T23:29:01.135 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-08T23:29:01.153 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2024-05-08T23:29:01.185 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2024-05-08T23:29:01.265 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-08T23:29:01.355 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-05-08T23:29:01.489 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-05-08T23:29:01.523 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package postgresql-client-12. 2024-05-08T23:29:01.540 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../26-postgresql-client-12_12.18-0ubuntu0.20.04.1_amd64.deb ... 2024-05-08T23:29:01.581 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking postgresql-client-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T23:29:01.615 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2024-05-08T23:29:02.253 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package postgresql-common. 2024-05-08T23:29:02.269 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2024-05-08T23:29:02.332 INFO:teuthology.orchestra.run.smithi081.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-08T23:29:02.354 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2024-05-08T23:29:02.835 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package postgresql-12. 2024-05-08T23:29:02.852 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../28-postgresql-12_12.18-0ubuntu0.20.04.1_amd64.deb ... 2024-05-08T23:29:02.896 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking postgresql-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T23:29:03.023 INFO:teuthology.orchestra.run.smithi117.stdout:Adding user postgres to group ssl-cert 2024-05-08T23:29:03.271 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:03.272 INFO:teuthology.orchestra.run.smithi117.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-08T23:29:03.528 INFO:teuthology.orchestra.run.smithi117.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-08T23:29:03.529 INFO:teuthology.orchestra.run.smithi117.stdout:Removing obsolete dictionary files: 2024-05-08T23:29:03.851 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-08T23:29:04.608 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.22) ... 2024-05-08T23:29:04.663 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package postgresql. 2024-05-08T23:29:04.682 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2024-05-08T23:29:04.709 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-05-08T23:29:04.738 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2024-05-08T23:29:04.810 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-05-08T23:29:04.970 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-05-08T23:29:04.982 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package postgresql-client. 2024-05-08T23:29:05.000 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2024-05-08T23:29:05.049 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2024-05-08T23:29:05.334 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-08T23:29:05.352 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2024-05-08T23:29:05.392 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2024-05-08T23:29:05.670 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-08T23:29:05.687 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2024-05-08T23:29:05.728 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-05-08T23:29:05.911 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-05-08T23:29:06.133 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up dump (0.4b46-6) ... 2024-05-08T23:29:06.139 INFO:teuthology.orchestra.run.smithi081.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-08T23:29:06.156 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-05-08T23:29:06.208 INFO:teuthology.orchestra.run.smithi081.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-05-08T23:29:06.214 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-08T23:29:06.277 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T23:29:06.394 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.7) ... 2024-05-08T23:29:06.516 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2024-05-08T23:29:06.553 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-05-08T23:29:06.679 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-05-08T23:29:06.729 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.22) ... 2024-05-08T23:29:06.865 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2024-05-08T23:29:06.990 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up postgresql-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T23:29:07.056 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libpq5:amd64 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T23:29:07.206 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-05-08T23:29:07.341 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-05-08T23:29:07.354 INFO:teuthology.orchestra.run.smithi117.stdout:Creating new PostgreSQL cluster 12/main ... 2024-05-08T23:29:07.366 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2024-05-08T23:29:07.464 INFO:teuthology.orchestra.run.smithi117.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-08T23:29:07.464 INFO:teuthology.orchestra.run.smithi117.stdout:This user must also own the server process. 2024-05-08T23:29:07.464 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:07.465 INFO:teuthology.orchestra.run.smithi117.stdout:The database cluster will be initialized with locale "en_US". 2024-05-08T23:29:07.465 INFO:teuthology.orchestra.run.smithi117.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-08T23:29:07.465 INFO:teuthology.orchestra.run.smithi117.stdout:The default text search configuration will be set to "english". 2024-05-08T23:29:07.465 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:07.465 INFO:teuthology.orchestra.run.smithi117.stdout:Data page checksums are disabled. 2024-05-08T23:29:07.465 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:07.465 INFO:teuthology.orchestra.run.smithi117.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2024-05-08T23:29:07.465 INFO:teuthology.orchestra.run.smithi117.stdout:creating subdirectories ... ok 2024-05-08T23:29:07.465 INFO:teuthology.orchestra.run.smithi117.stdout:selecting dynamic shared memory implementation ... posix 2024-05-08T23:29:07.467 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2024-05-08T23:29:07.494 INFO:teuthology.orchestra.run.smithi117.stdout:selecting default max_connections ... 100 2024-05-08T23:29:07.517 INFO:teuthology.orchestra.run.smithi117.stdout:selecting default shared_buffers ... 128MB 2024-05-08T23:29:07.524 INFO:teuthology.orchestra.run.smithi117.stdout:selecting default time zone ... Etc/UTC 2024-05-08T23:29:07.525 INFO:teuthology.orchestra.run.smithi117.stdout:creating configuration files ... ok 2024-05-08T23:29:07.631 INFO:teuthology.orchestra.run.smithi117.stdout:running bootstrap script ... ok 2024-05-08T23:29:07.886 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-08T23:29:07.886 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-08T23:29:08.205 INFO:teuthology.orchestra.run.smithi117.stdout:performing post-bootstrap initialization ... ok 2024-05-08T23:29:08.371 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up postgresql-client-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T23:29:08.932 INFO:teuthology.orchestra.run.smithi117.stdout:syncing data to disk ... ok 2024-05-08T23:29:08.932 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:08.932 INFO:teuthology.orchestra.run.smithi117.stdout:Success. You can now start the database server using: 2024-05-08T23:29:08.932 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:08.933 INFO:teuthology.orchestra.run.smithi117.stdout: pg_ctlcluster 12 main start 2024-05-08T23:29:08.933 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:09.252 INFO:teuthology.orchestra.run.smithi117.stdout:Ver Cluster Port Status Owner Data directory Log file 2024-05-08T23:29:09.252 INFO:teuthology.orchestra.run.smithi117.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2024-05-08T23:29:09.278 INFO:teuthology.orchestra.run.smithi081.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-08T23:29:09.338 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-08T23:29:09.986 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-05-08T23:29:10.111 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-05-08T23:29:10.262 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2024-05-08T23:29:11.280 INFO:teuthology.orchestra.run.smithi081.stdout:Adding user postgres to group ssl-cert 2024-05-08T23:29:11.534 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:11.535 INFO:teuthology.orchestra.run.smithi081.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-08T23:29:11.780 INFO:teuthology.orchestra.run.smithi081.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-08T23:29:11.780 INFO:teuthology.orchestra.run.smithi081.stdout:Removing obsolete dictionary files: 2024-05-08T23:29:12.110 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-08T23:29:12.272 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T23:29:12.397 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T23:29:12.523 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-05-08T23:29:12.683 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T23:29:12.792 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2024-05-08T23:29:12.881 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.22) ... 2024-05-08T23:29:12.910 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-05-08T23:29:13.016 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-05-08T23:29:13.167 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-05-08T23:29:13.293 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-05-08T23:29:13.619 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:13.619 INFO:teuthology.orchestra.run.smithi117.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-08T23:29:13.619 INFO:teuthology.orchestra.run.smithi117.stdout:which means they are currently configured by ifupdown: 2024-05-08T23:29:13.619 INFO:teuthology.orchestra.run.smithi117.stdout:- enp3s0f1 2024-05-08T23:29:13.619 INFO:teuthology.orchestra.run.smithi117.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-08T23:29:13.619 INFO:teuthology.orchestra.run.smithi117.stdout:remove their configuration from /etc/network/interfaces. 2024-05-08T23:29:13.619 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:13.771 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-08T23:29:14.096 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-08T23:29:14.290 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-05-08T23:29:14.524 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up dump (0.4b46-6) ... 2024-05-08T23:29:14.602 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-08T23:29:14.621 INFO:teuthology.orchestra.run.smithi081.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-08T23:29:14.684 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T23:29:14.835 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.7) ... 2024-05-08T23:29:14.969 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-05-08T23:29:15.090 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T23:29:15.095 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-05-08T23:29:15.225 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-05-08T23:29:15.297 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2024-05-08T23:29:15.367 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T23:29:15.424 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up postgresql-12 (12.18-0ubuntu0.20.04.1) ... 2024-05-08T23:29:15.496 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2024-05-08T23:29:15.619 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2024-05-08T23:29:15.770 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T23:29:15.793 INFO:teuthology.orchestra.run.smithi081.stdout:Creating new PostgreSQL cluster 12/main ... 2024-05-08T23:29:15.812 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2024-05-08T23:29:15.858 INFO:teuthology.orchestra.run.smithi081.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-08T23:29:15.858 INFO:teuthology.orchestra.run.smithi081.stdout:This user must also own the server process. 2024-05-08T23:29:15.858 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:15.858 INFO:teuthology.orchestra.run.smithi081.stdout:The database cluster will be initialized with locale "en_US". 2024-05-08T23:29:15.858 INFO:teuthology.orchestra.run.smithi081.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-08T23:29:15.858 INFO:teuthology.orchestra.run.smithi081.stdout:The default text search configuration will be set to "english". 2024-05-08T23:29:15.858 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:15.858 INFO:teuthology.orchestra.run.smithi081.stdout:Data page checksums are disabled. 2024-05-08T23:29:15.858 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:15.858 INFO:teuthology.orchestra.run.smithi081.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2024-05-08T23:29:15.859 INFO:teuthology.orchestra.run.smithi081.stdout:creating subdirectories ... ok 2024-05-08T23:29:15.859 INFO:teuthology.orchestra.run.smithi081.stdout:selecting dynamic shared memory implementation ... posix 2024-05-08T23:29:15.881 INFO:teuthology.orchestra.run.smithi081.stdout:selecting default max_connections ... 100 2024-05-08T23:29:15.888 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T23:29:15.898 INFO:teuthology.orchestra.run.smithi081.stdout:selecting default shared_buffers ... 128MB 2024-05-08T23:29:15.905 INFO:teuthology.orchestra.run.smithi081.stdout:selecting default time zone ... Etc/UTC 2024-05-08T23:29:15.905 INFO:teuthology.orchestra.run.smithi081.stdout:creating configuration files ... ok 2024-05-08T23:29:16.005 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T23:29:16.009 INFO:teuthology.orchestra.run.smithi081.stdout:running bootstrap script ... ok 2024-05-08T23:29:16.279 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-08T23:29:16.279 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-08T23:29:16.531 INFO:teuthology.orchestra.run.smithi081.stdout:performing post-bootstrap initialization ... ok 2024-05-08T23:29:16.808 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-05-08T23:29:16.917 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-08T23:29:17.060 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-05-08T23:29:17.237 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-05-08T23:29:17.330 INFO:teuthology.orchestra.run.smithi081.stdout:syncing data to disk ... ok 2024-05-08T23:29:17.330 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:17.330 INFO:teuthology.orchestra.run.smithi081.stdout:Success. You can now start the database server using: 2024-05-08T23:29:17.331 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:17.331 INFO:teuthology.orchestra.run.smithi081.stdout: pg_ctlcluster 12 main start 2024-05-08T23:29:17.331 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:17.538 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-05-08T23:29:17.664 INFO:teuthology.orchestra.run.smithi081.stdout:Ver Cluster Port Status Owner Data directory Log file 2024-05-08T23:29:17.664 INFO:teuthology.orchestra.run.smithi081.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2024-05-08T23:29:17.738 INFO:teuthology.orchestra.run.smithi081.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-08T23:29:20.822 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T23:29:20.948 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T23:29:21.107 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-05-08T23:29:21.275 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T23:29:21.418 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2024-05-08T23:29:21.544 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-05-08T23:29:22.254 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:22.254 INFO:teuthology.orchestra.run.smithi081.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-08T23:29:22.254 INFO:teuthology.orchestra.run.smithi081.stdout:which means they are currently configured by ifupdown: 2024-05-08T23:29:22.254 INFO:teuthology.orchestra.run.smithi081.stdout:- enp3s0f1 2024-05-08T23:29:22.254 INFO:teuthology.orchestra.run.smithi081.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-08T23:29:22.254 INFO:teuthology.orchestra.run.smithi081.stdout:remove their configuration from /etc/network/interfaces. 2024-05-08T23:29:22.254 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:22.414 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-08T23:29:22.742 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-08T23:29:23.299 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-08T23:29:23.788 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-05-08T23:29:23.957 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-05-08T23:29:24.100 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-05-08T23:29:24.226 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2024-05-08T23:29:24.385 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2024-05-08T23:29:24.511 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T23:29:24.637 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-05-08T23:29:24.758 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-05-08T23:29:24.788 INFO:teuthology.orchestra.run.smithi081.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-05-08T23:29:24.867 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-05-08T23:29:25.062 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-08T23:29:25.063 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-08T23:29:25.608 INFO:teuthology.orchestra.run.smithi081.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-05-08T23:29:25.702 INFO:teuthology.orchestra.run.smithi117.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-08T23:29:25.707 DEBUG:teuthology.parallel:result is None 2024-05-08T23:29:25.709 INFO:teuthology.orchestra.run.smithi081.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-08T23:29:25.835 INFO:teuthology.orchestra.run.smithi081.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-05-08T23:29:26.011 INFO:teuthology.orchestra.run.smithi081.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-05-08T23:29:26.313 INFO:teuthology.orchestra.run.smithi081.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-05-08T23:29:33.743 INFO:teuthology.orchestra.run.smithi081.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-05-08T23:29:33.843 INFO:teuthology.orchestra.run.smithi081.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-05-08T23:29:34.708 INFO:teuthology.orchestra.run.smithi081.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-08T23:29:34.712 DEBUG:teuthology.parallel:result is None 2024-05-08T23:29:34.713 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:29:34.936 DEBUG:teuthology.orchestra.run.smithi081:> dpkg-query -W -f '${Version}' ceph 2024-05-08T23:29:34.961 INFO:teuthology.orchestra.run.smithi081.stdout:18.2.2-1322-gd21e4f96-1focal 2024-05-08T23:29:34.961 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322-gd21e4f96-1focal 2024-05-08T23:29:34.961 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322-gd21e4f96-1focal is installed. 2024-05-08T23:29:34.963 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:29:35.192 DEBUG:teuthology.orchestra.run.smithi117:> dpkg-query -W -f '${Version}' ceph 2024-05-08T23:29:35.215 INFO:teuthology.orchestra.run.smithi117.stdout:18.2.2-1322-gd21e4f96-1focal 2024-05-08T23:29:35.216 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322-gd21e4f96-1focal 2024-05-08T23:29:35.216 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322-gd21e4f96-1focal is installed. 2024-05-08T23:29:35.218 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T23:29:35.218 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:29:35.218 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T23:29:35.235 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:29:35.235 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T23:29:35.251 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T23:29:35.251 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:29:35.251 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T23:29:35.268 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T23:29:35.326 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:29:35.327 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T23:29:35.343 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T23:29:35.355 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T23:29:35.355 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:29:35.355 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T23:29:35.370 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T23:29:35.380 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:29:35.380 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T23:29:35.394 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T23:29:35.452 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T23:29:35.452 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:29:35.452 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T23:29:35.468 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T23:29:35.527 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:29:35.527 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T23:29:35.544 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T23:29:35.600 INFO:teuthology.run_tasks:Running task ceph... 2024-05-08T23:29:35.687 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-08T23:29:35.687 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 777 /var/log/ceph 2024-05-08T23:29:35.690 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /var/log/ceph 2024-05-08T23:29:35.705 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-08T23:29:35.705 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T23:29:35.708 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T23:29:35.719 INFO:tasks.ceph:Creating extra log directories... 2024-05-08T23:29:35.719 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T23:29:35.757 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T23:29:35.784 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-08T23:29:35.784 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-08T23:29:35.784 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-08T23:29:35.784 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:29:35.784 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-08T23:29:35.790 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-08T23:29:35.829 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:29:35.829 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-08T23:29:35.836 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-08T23:29:35.881 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-08T23:29:35.881 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-08T23:29:35.881 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698173', 'branch': 'wip-pdonnell-testing-20240508.150423-reef', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/pool-perm}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7698173', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240508.150423-reef'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240508.150423-reef', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 65, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 2738, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'sleep_before_teardown': 0, 'subset': '17/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240508.150423-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi081.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi117.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_pool_perm']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_19:22:21', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-08T23:29:35.882 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T23:29:35.884 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T23:29:35.924 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T23:29:35.933 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T23:29:35.950 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:29:35.950 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T23:29:35.955 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T23:29:35.955 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_1 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-05-08 23:24:38.981742309 +0000 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-05-08 23:24:38.757747473 +0000 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-05-08 23:24:38.757747473 +0000 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: - 2024-05-08T23:29:35.962 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T23:29:36.018 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-05-08T23:29:36.018 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-05-08T23:29:36.018 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000431279 s, 1.2 MB/s 2024-05-08T23:29:36.019 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T23:29:36.027 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_2 2024-05-08T23:29:36.033 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T23:29:36.033 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:29:36.033 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 6h/6d Inode: 365 Links: 1 2024-05-08T23:29:36.033 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:29:36.033 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-05-08 23:24:39.397732721 +0000 2024-05-08T23:29:36.033 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-05-08 23:24:39.185737607 +0000 2024-05-08T23:29:36.033 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-05-08 23:24:39.185737607 +0000 2024-05-08T23:29:36.033 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: - 2024-05-08T23:29:36.034 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T23:29:36.090 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-05-08T23:29:36.090 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-05-08T23:29:36.090 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000398526 s, 1.3 MB/s 2024-05-08T23:29:36.092 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T23:29:36.098 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_3 2024-05-08T23:29:36.103 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T23:29:36.103 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:29:36.103 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 6h/6d Inode: 372 Links: 1 2024-05-08T23:29:36.103 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:29:36.103 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-05-08 23:24:39.861722025 +0000 2024-05-08T23:29:36.103 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-05-08 23:24:39.653726820 +0000 2024-05-08T23:29:36.103 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-05-08 23:24:39.653726820 +0000 2024-05-08T23:29:36.104 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: - 2024-05-08T23:29:36.104 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T23:29:36.114 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-05-08T23:29:36.114 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-05-08T23:29:36.114 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000397147 s, 1.3 MB/s 2024-05-08T23:29:36.115 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T23:29:36.164 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_4 2024-05-08T23:29:36.171 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T23:29:36.171 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:29:36.171 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 6h/6d Inode: 378 Links: 1 2024-05-08T23:29:36.171 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:29:36.172 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-05-08 23:24:40.309711699 +0000 2024-05-08T23:29:36.172 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-05-08 23:24:40.081716955 +0000 2024-05-08T23:29:36.172 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-05-08 23:24:40.081716955 +0000 2024-05-08T23:29:36.172 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: - 2024-05-08T23:29:36.172 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T23:29:36.185 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-05-08T23:29:36.185 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-05-08T23:29:36.185 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000370449 s, 1.4 MB/s 2024-05-08T23:29:36.186 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T23:29:36.194 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T23:29:36.194 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:29:36.194 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T23:29:36.201 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T23:29:36.201 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-05-08T23:29:36.252 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T23:29:36.252 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:29:36.252 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 358 Links: 1 2024-05-08T23:29:36.252 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:29:36.252 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-05-08 23:24:36.930912391 +0000 2024-05-08T23:29:36.252 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-05-08 23:24:36.714917368 +0000 2024-05-08T23:29:36.252 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-05-08 23:24:36.714917368 +0000 2024-05-08T23:29:36.252 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2024-05-08T23:29:36.253 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T23:29:36.265 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-05-08T23:29:36.265 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-05-08T23:29:36.265 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000389512 s, 1.3 MB/s 2024-05-08T23:29:36.266 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T23:29:36.275 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-05-08T23:29:36.280 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T23:29:36.280 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:29:36.281 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 366 Links: 1 2024-05-08T23:29:36.281 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:29:36.281 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-05-08 23:24:37.350902716 +0000 2024-05-08T23:29:36.281 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-05-08 23:24:37.122907968 +0000 2024-05-08T23:29:36.281 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-05-08 23:24:37.122907968 +0000 2024-05-08T23:29:36.281 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2024-05-08T23:29:36.281 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T23:29:36.334 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-05-08T23:29:36.334 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-05-08T23:29:36.334 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.00040127 s, 1.3 MB/s 2024-05-08T23:29:36.335 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T23:29:36.343 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-05-08T23:29:36.349 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T23:29:36.349 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:29:36.349 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 371 Links: 1 2024-05-08T23:29:36.350 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:29:36.350 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-05-08 23:24:37.794892486 +0000 2024-05-08T23:29:36.350 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-05-08 23:24:37.570897646 +0000 2024-05-08T23:29:36.350 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-05-08 23:24:37.570897646 +0000 2024-05-08T23:29:36.350 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2024-05-08T23:29:36.350 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T23:29:36.363 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-05-08T23:29:36.363 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-05-08T23:29:36.363 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000433303 s, 1.2 MB/s 2024-05-08T23:29:36.365 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T23:29:36.372 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-05-08T23:29:36.377 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T23:29:36.377 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:29:36.377 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 378 Links: 1 2024-05-08T23:29:36.377 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:29:36.377 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-05-08 23:24:38.242882165 +0000 2024-05-08T23:29:36.377 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-05-08 23:24:38.050886588 +0000 2024-05-08T23:29:36.378 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-05-08 23:24:38.050886588 +0000 2024-05-08T23:29:36.378 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2024-05-08T23:29:36.378 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T23:29:36.435 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-05-08T23:29:36.436 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-05-08T23:29:36.436 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000401851 s, 1.3 MB/s 2024-05-08T23:29:36.437 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T23:29:36.445 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T23:29:36.445 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi081.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T23:29:36.445 INFO:tasks.ceph:Generating config... 2024-05-08T23:29:36.447 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-08T23:29:36.447 INFO:tasks.ceph:[client] debug client = 20 2024-05-08T23:29:36.447 INFO:tasks.ceph:[client] debug ms = 1 2024-05-08T23:29:36.447 INFO:tasks.ceph:[client] pid file = 2024-05-08T23:29:36.447 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-08T23:29:36.447 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-08T23:29:36.447 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-08T23:29:36.447 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-08T23:29:36.447 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-08T23:29:36.447 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-08T23:29:36.447 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-08T23:29:36.447 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-08T23:29:36.447 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-08T23:29:36.448 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-08T23:29:36.448 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-08T23:29:36.448 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-08T23:29:36.448 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-08T23:29:36.448 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-08T23:29:36.448 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-08T23:29:36.448 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-08T23:29:36.448 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-08T23:29:36.448 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-08T23:29:36.448 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-08T23:29:36.448 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-08T23:29:36.448 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-08T23:29:36.448 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-08T23:29:36.448 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-08T23:29:36.448 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-08T23:29:36.448 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-08T23:29:36.449 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-08T23:29:36.449 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-08T23:29:36.449 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-08T23:29:36.449 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-08T23:29:36.449 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-08T23:29:36.449 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-08T23:29:36.449 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-08T23:29:36.449 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-08T23:29:36.449 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-08T23:29:36.449 INFO:tasks.ceph:Setting up mon.a... 2024-05-08T23:29:36.449 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-08T23:29:36.490 INFO:teuthology.orchestra.run.smithi081.stdout:creating /etc/ceph/ceph.keyring 2024-05-08T23:29:36.506 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-08T23:29:36.546 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T23:29:36.603 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.81'), ('mon.b', '172.21.15.117'), ('mon.c', '[v2:172.21.15.117:3301,v1:172.21.15.117:6790]')] 2024-05-08T23:29:36.603 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.81,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-08T23:29:36.604 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:29:36.604 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T23:29:36.608 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.81 --add b 172.21.15.117 --addv c '[v2:172.21.15.117:3301,v1:172.21.15.117:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:29:36.653 INFO:teuthology.orchestra.run.smithi081.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:29:36.653 INFO:teuthology.orchestra.run.smithi081.stdout:monmaptool: generated fsid dee3de5e-bc15-439a-9578-b5583aefc890 2024-05-08T23:29:36.653 INFO:teuthology.orchestra.run.smithi081.stdout:setting min_mon_release = pacific 2024-05-08T23:29:36.653 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 0 2024-05-08T23:29:36.654 INFO:teuthology.orchestra.run.smithi081.stdout:fsid dee3de5e-bc15-439a-9578-b5583aefc890 2024-05-08T23:29:36.654 INFO:teuthology.orchestra.run.smithi081.stdout:last_changed 2024-05-08T23:29:36.653947+0000 2024-05-08T23:29:36.654 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-05-08T23:29:36.653947+0000 2024-05-08T23:29:36.654 INFO:teuthology.orchestra.run.smithi081.stdout:min_mon_release 16 (pacific) 2024-05-08T23:29:36.654 INFO:teuthology.orchestra.run.smithi081.stdout:election_strategy: 1 2024-05-08T23:29:36.654 INFO:teuthology.orchestra.run.smithi081.stdout:0: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.a 2024-05-08T23:29:36.654 INFO:teuthology.orchestra.run.smithi081.stdout:1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-05-08T23:29:36.654 INFO:teuthology.orchestra.run.smithi081.stdout:2: [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] mon.c 2024-05-08T23:29:36.654 INFO:teuthology.orchestra.run.smithi081.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-08T23:29:36.655 DEBUG:teuthology.orchestra.run.smithi081:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T23:29:36.661 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID dee3de5e-bc15-439a-9578-b5583aefc890... 2024-05-08T23:29:36.662 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T23:29:36.705 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T23:29:36.737 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2024-05-08T23:29:36.737 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2024-05-08T23:29:36.737 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T23:29:36.737 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2024-05-08T23:29:36.737 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.738 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2024-05-08T23:29:36.738 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.738 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2024-05-08T23:29:36.738 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.738 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2024-05-08T23:29:36.738 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2024-05-08T23:29:36.738 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.738 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2024-05-08T23:29:36.738 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2024-05-08T23:29:36.738 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.738 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T23:29:36.738 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2024-05-08T23:29:36.738 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.739 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T23:29:36.739 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T23:29:36.739 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.739 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2024-05-08T23:29:36.739 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.739 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2024-05-08T23:29:36.739 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2024-05-08T23:29:36.739 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2024-05-08T23:29:36.739 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2024-05-08T23:29:36.739 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2024-05-08T23:29:36.739 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2024-05-08T23:29:36.739 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2024-05-08T23:29:36.739 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T23:29:36.739 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T23:29:36.739 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2024-05-08T23:29:36.740 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.740 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T23:29:36.740 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.740 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2024-05-08T23:29:36.740 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.740 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2024-05-08T23:29:36.740 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.740 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2024-05-08T23:29:36.740 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2024-05-08T23:29:36.740 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2024-05-08T23:29:36.740 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "172.21.15.81,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-05-08T23:29:36.740 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = dee3de5e-bc15-439a-9578-b5583aefc890 2024-05-08T23:29:36.740 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.740 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2024-05-08T23:29:36.740 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.741 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2024-05-08T23:29:36.742 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.742 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2024-05-08T23:29:36.742 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.742 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2024-05-08T23:29:36.742 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.742 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2024-05-08T23:29:36.742 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2024-05-08T23:29:36.742 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.742 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2024-05-08T23:29:36.742 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2024-05-08T23:29:36.742 INFO:teuthology.orchestra.run.smithi081.stdout: bdev async discard = True 2024-05-08T23:29:36.742 INFO:teuthology.orchestra.run.smithi081.stdout: bdev enable discard = True 2024-05-08T23:29:36.742 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore allocator = bitmap 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 1/20 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 1/20 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 4/10 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2024-05-08T23:29:36.743 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-05-08T23:29:36.744 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2024-05-08T23:29:36.744 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2024-05-08T23:29:36.744 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2024-05-08T23:29:36.744 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2024-05-08T23:29:36.744 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2024-05-08T23:29:36.744 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2024-05-08T23:29:36.744 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.744 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2024-05-08T23:29:36.744 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-05-08T23:29:36.744 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2024-05-08T23:29:36.744 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2024-05-08T23:29:36.744 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2024-05-08T23:29:36.744 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2024-05-08T23:29:36.744 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2024-05-08T23:29:36.745 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2024-05-08T23:29:36.745 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2024-05-08T23:29:36.745 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2024-05-08T23:29:36.745 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2024-05-08T23:29:36.745 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.745 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T23:29:36.745 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T23:29:36.745 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T23:29:36.745 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.745 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T23:29:36.745 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T23:29:36.745 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T23:29:36.745 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.745 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2024-05-08T23:29:36.746 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2024-05-08T23:29:36.746 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.746 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2024-05-08T23:29:36.746 INFO:teuthology.orchestra.run.smithi081.stdout: mon op complaint time = 120 2024-05-08T23:29:36.746 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:29:36.746 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2024-05-08T23:29:36.746 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2024-05-08T23:29:36.746 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2024-05-08T23:29:36.746 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2024-05-08T23:29:36.746 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T23:29:36.746 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T23:29:36.746 INFO:teuthology.orchestra.run.smithi081.stdout: client mount timeout = 600 2024-05-08T23:29:36.746 INFO:teuthology.orchestra.run.smithi081.stdout: debug client = 20 2024-05-08T23:29:36.746 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-05-08T23:29:36.746 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = "" 2024-05-08T23:29:36.747 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2024-05-08T23:29:36.747 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2024-05-08T23:29:36.747 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2024-05-08T23:29:36.747 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2024-05-08T23:29:36.747 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2024-05-08T23:29:36.747 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2024-05-08T23:29:36.747 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2024-05-08T23:29:36.747 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2024-05-08T23:29:36.747 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2024-05-08T23:29:36.747 INFO:teuthology.orchestra.run.smithi081.stdout:[mds] 2024-05-08T23:29:36.747 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds = 20 2024-05-08T23:29:36.747 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds balancer = 20 2024-05-08T23:29:36.747 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-05-08T23:29:36.747 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug frag = True 2024-05-08T23:29:36.747 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug scatterstat = True 2024-05-08T23:29:36.748 INFO:teuthology.orchestra.run.smithi081.stdout: mds op complaint time = 180 2024-05-08T23:29:36.748 INFO:teuthology.orchestra.run.smithi081.stdout: mds verify scatter = True 2024-05-08T23:29:36.748 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2024-05-08T23:29:36.748 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2024-05-08T23:29:36.748 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2024-05-08T23:29:36.748 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2024-05-08T23:29:36.748 INFO:teuthology.orchestra.run.smithi117.stdout: chdir = "" 2024-05-08T23:29:36.749 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T23:29:36.749 INFO:teuthology.orchestra.run.smithi117.stdout: auth supported = cephx 2024-05-08T23:29:36.749 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.749 INFO:teuthology.orchestra.run.smithi117.stdout: filestore xattr use omap = true 2024-05-08T23:29:36.749 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.749 INFO:teuthology.orchestra.run.smithi117.stdout: mon clock drift allowed = 1.000 2024-05-08T23:29:36.749 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.749 INFO:teuthology.orchestra.run.smithi117.stdout: osd crush chooseleaf type = 0 2024-05-08T23:29:36.749 INFO:teuthology.orchestra.run.smithi117.stdout: auth debug = true 2024-05-08T23:29:36.749 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.749 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on old message = true 2024-05-08T23:29:36.749 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on bug = true 2024-05-08T23:29:36.749 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.749 INFO:teuthology.orchestra.run.smithi117.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T23:29:36.750 INFO:teuthology.orchestra.run.smithi117.stdout: mon pg warn max object skew = 0 2024-05-08T23:29:36.750 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.750 INFO:teuthology.orchestra.run.smithi117.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T23:29:36.750 INFO:teuthology.orchestra.run.smithi117.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T23:29:36.750 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.750 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default size = 2 2024-05-08T23:29:36.750 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.750 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow primary affinity = true 2024-05-08T23:29:36.750 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow pg remap = true 2024-05-08T23:29:36.750 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on legacy crush tunables = false 2024-05-08T23:29:36.750 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on crush straw calc version zero = false 2024-05-08T23:29:36.750 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on no sortbitwise = false 2024-05-08T23:29:36.750 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on osd down out interval zero = false 2024-05-08T23:29:36.750 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on too few osds = false 2024-05-08T23:29:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T23:29:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T23:29:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: mon_allow_pool_size_one = true 2024-05-08T23:29:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T23:29:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: osd default data pool replay window = 5 2024-05-08T23:29:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: mon allow pool delete = true 2024-05-08T23:29:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: mon cluster log file level = debug 2024-05-08T23:29:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: debug asserts on shutdown = true 2024-05-08T23:29:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: mon health detail to clog = false 2024-05-08T23:29:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: mon host = "172.21.15.81,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-05-08T23:29:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = dee3de5e-bc15-439a-9578-b5583aefc890 2024-05-08T23:29:36.752 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.752 INFO:teuthology.orchestra.run.smithi117.stdout:[osd] 2024-05-08T23:29:36.752 INFO:teuthology.orchestra.run.smithi117.stdout: osd journal size = 100 2024-05-08T23:29:36.752 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.752 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub load threshold = 5.0 2024-05-08T23:29:36.752 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub max interval = 600 2024-05-08T23:29:36.752 INFO:teuthology.orchestra.run.smithi117.stdout: osd mclock profile = high_recovery_ops 2024-05-08T23:29:36.752 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.752 INFO:teuthology.orchestra.run.smithi117.stdout: osd recover clone overlap = true 2024-05-08T23:29:36.752 INFO:teuthology.orchestra.run.smithi117.stdout: osd recovery max chunk = 1048576 2024-05-08T23:29:36.752 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.752 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug shutdown = true 2024-05-08T23:29:36.752 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug op order = true 2024-05-08T23:29:36.752 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify stray on activate = true 2024-05-08T23:29:36.752 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.753 INFO:teuthology.orchestra.run.smithi117.stdout: osd open classes on start = true 2024-05-08T23:29:36.753 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug pg log writeout = true 2024-05-08T23:29:36.753 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.753 INFO:teuthology.orchestra.run.smithi117.stdout: osd deep scrub update digest min age = 30 2024-05-08T23:29:36.753 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.753 INFO:teuthology.orchestra.run.smithi117.stdout: osd map max advance = 10 2024-05-08T23:29:36.753 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.753 INFO:teuthology.orchestra.run.smithi117.stdout: journal zero on create = true 2024-05-08T23:29:36.753 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.753 INFO:teuthology.orchestra.run.smithi117.stdout: filestore ondisk finisher threads = 3 2024-05-08T23:29:36.753 INFO:teuthology.orchestra.run.smithi117.stdout: filestore apply finisher threads = 3 2024-05-08T23:29:36.753 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.753 INFO:teuthology.orchestra.run.smithi117.stdout: bdev debug aio = true 2024-05-08T23:29:36.753 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug misdirected ops = true 2024-05-08T23:29:36.753 INFO:teuthology.orchestra.run.smithi117.stdout: bdev async discard = True 2024-05-08T23:29:36.754 INFO:teuthology.orchestra.run.smithi117.stdout: bdev enable discard = True 2024-05-08T23:29:36.754 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore allocator = bitmap 2024-05-08T23:29:36.754 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore block size = 96636764160 2024-05-08T23:29:36.754 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore fsck on mount = True 2024-05-08T23:29:36.754 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluefs = 1/20 2024-05-08T23:29:36.754 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluestore = 1/20 2024-05-08T23:29:36.754 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-05-08T23:29:36.754 INFO:teuthology.orchestra.run.smithi117.stdout: debug osd = 20 2024-05-08T23:29:36.754 INFO:teuthology.orchestra.run.smithi117.stdout: debug rocksdb = 4/10 2024-05-08T23:29:36.754 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T23:29:36.754 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd full ratio = 0.9 2024-05-08T23:29:36.754 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd nearfull ratio = 0.8 2024-05-08T23:29:36.754 INFO:teuthology.orchestra.run.smithi117.stdout: osd failsafe full ratio = 0.95 2024-05-08T23:29:36.754 INFO:teuthology.orchestra.run.smithi117.stdout: osd objectstore = bluestore 2024-05-08T23:29:36.754 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr] 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout: debug mgr = 20 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout: mgr/telemetry/nag = false 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout:[mon] 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout: debug paxos = 20 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-05-08T23:29:36.755 INFO:teuthology.orchestra.run.smithi117.stdout: mon data avail warn = 5 2024-05-08T23:29:36.756 INFO:teuthology.orchestra.run.smithi117.stdout: mon mgr mkfs grace = 240 2024-05-08T23:29:36.756 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-05-08T23:29:36.756 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd reporter subtree level = osd 2024-05-08T23:29:36.756 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd prime pg temp = true 2024-05-08T23:29:36.756 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-05-08T23:29:36.756 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.756 INFO:teuthology.orchestra.run.smithi117.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T23:29:36.756 INFO:teuthology.orchestra.run.smithi117.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T23:29:36.756 INFO:teuthology.orchestra.run.smithi117.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T23:29:36.756 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.756 INFO:teuthology.orchestra.run.smithi117.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T23:29:36.756 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T23:29:36.756 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T23:29:36.756 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.756 INFO:teuthology.orchestra.run.smithi117.stdout: # 1m isn't quite enough 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout: mon_down_mkfs_grace = 2m 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_filestore_osds = false 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout: mon op complaint time = 120 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout:[client] 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout: rgw cache enabled = true 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable ops log = true 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable usage log = true 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout: client mount timeout = 600 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout: debug client = 20 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = "" 2024-05-08T23:29:36.757 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.a] 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.b] 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.c] 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c] 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d] 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout:[mds] 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds = 20 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds balancer = 20 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug frag = True 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug scatterstat = True 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout: mds op complaint time = 180 2024-05-08T23:29:36.758 INFO:teuthology.orchestra.run.smithi117.stdout: mds verify scatter = True 2024-05-08T23:29:36.759 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-05-08T23:29:36.759 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-05-08T23:29:36.759 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-05-08T23:29:36.760 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-08T23:29:36.760 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-08T23:29:36.814 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-08T23:29:36.814 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:29:36.814 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-08T23:29:36.819 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:29:36.819 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-08T23:29:36.825 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:29:36.825 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:29:36.825 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T23:29:36.826 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T23:29:36.849 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:29:36.849 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:29:36.853 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:29:36.853 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:29:36.853 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T23:29:36.853 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T23:29:36.881 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:29:36.881 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:29:36.885 INFO:tasks.ceph:Setting up mon nodes... 2024-05-08T23:29:36.885 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-08T23:29:36.886 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T23:29:36.951 INFO:teuthology.orchestra.run.smithi081.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T23:29:36.954 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T23:29:37.004 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T23:29:37.019 INFO:tasks.ceph:Setting up mds nodes... 2024-05-08T23:29:37.019 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-08T23:29:37.071 INFO:teuthology.orchestra.run.smithi081.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-08T23:29:37.074 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-08T23:29:37.088 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-08T23:29:37.133 INFO:teuthology.orchestra.run.smithi081.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-08T23:29:37.136 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-08T23:29:37.146 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-08T23:29:37.196 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-08T23:29:37.199 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-08T23:29:37.214 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-08T23:29:37.258 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-08T23:29:37.261 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-08T23:29:37.274 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-08T23:29:37.274 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T23:29:37.317 INFO:teuthology.orchestra.run.smithi081.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-08T23:29:37.329 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-08T23:29:37.365 INFO:teuthology.orchestra.run.smithi081.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-08T23:29:37.378 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-08T23:29:37.422 INFO:teuthology.orchestra.run.smithi117.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-08T23:29:37.435 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-08T23:29:37.519 INFO:teuthology.orchestra.run.smithi117.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-08T23:29:37.532 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-08T23:29:37.532 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi081.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T23:29:37.532 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-08T23:29:37.549 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T23:29:37.549 INFO:tasks.ceph:role: osd.0 2024-05-08T23:29:37.549 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:29:37.549 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T23:29:38.070 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:29:38.070 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:29:38.071 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:29:38.071 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 2024-05-08T23:29:38.071 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:29:38.071 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:29:38.071 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:29:38.071 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:29:38.071 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:29:38.071 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:29:38.072 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-05-08T23:29:38.073 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-08T23:29:38.124 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-08T23:29:38.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:29:38.134 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T23:29:38.135 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-08T23:29:38.149 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T23:29:38.149 INFO:tasks.ceph:role: osd.1 2024-05-08T23:29:38.150 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:29:38.150 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T23:29:38.703 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:29:38.703 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:29:38.703 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:29:38.703 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 2024-05-08T23:29:38.703 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:29:38.703 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:29:38.703 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:29:38.704 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:29:38.704 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:29:38.704 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:29:38.705 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-05-08T23:29:38.705 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-08T23:29:38.792 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-08T23:29:38.804 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T23:29:38.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:29:38.805 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-08T23:29:38.815 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T23:29:38.815 INFO:tasks.ceph:role: osd.2 2024-05-08T23:29:38.815 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:29:38.815 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T23:29:39.309 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:29:39.309 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:29:39.309 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:29:39.309 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 2024-05-08T23:29:39.309 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:29:39.309 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:29:39.309 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:29:39.309 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:29:39.309 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:29:39.309 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:29:39.311 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-05-08T23:29:39.311 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-08T23:29:39.362 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-08T23:29:39.371 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T23:29:39.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:29:39.372 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-08T23:29:39.428 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T23:29:39.428 INFO:tasks.ceph:role: osd.3 2024-05-08T23:29:39.429 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:29:39.429 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T23:29:39.940 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:29:39.941 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:29:39.941 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:29:39.941 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 2024-05-08T23:29:39.941 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:29:39.941 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:29:39.941 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:29:39.941 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:29:39.941 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:29:39.941 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:29:39.942 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-05-08T23:29:39.943 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-08T23:29:39.999 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-08T23:29:40.011 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T23:29:40.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:29:40.012 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:29:40.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:40.142+0000 7fa8c3332880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-08T23:29:40.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:40.146+0000 7fa8c3332880 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-08T23:29:40.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:40.146+0000 7fa8c3332880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T23:29:40.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:40.146+0000 7fa8c3332880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T23:29:40.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:40.146+0000 7fa8c3332880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T23:29:40.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:40.150+0000 7fa8c3332880 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-08T23:29:43.152 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-08T23:29:43.166 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:29:43.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:43.206+0000 7fb12316d880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-08T23:29:43.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:43.206+0000 7fb12316d880 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-08T23:29:43.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:43.206+0000 7fb12316d880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T23:29:43.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:43.206+0000 7fb12316d880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T23:29:43.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:43.206+0000 7fb12316d880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T23:29:43.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:43.206+0000 7fb12316d880 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-08T23:29:46.215 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-08T23:29:46.231 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:29:46.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:46.314+0000 7f23bd20d880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-08T23:29:46.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:46.314+0000 7f23bd20d880 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-08T23:29:46.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:46.314+0000 7f23bd20d880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T23:29:46.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:46.314+0000 7f23bd20d880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T23:29:46.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:46.314+0000 7f23bd20d880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T23:29:46.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:46.314+0000 7f23bd20d880 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-08T23:29:49.323 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-08T23:29:49.337 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:29:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:49.374+0000 7fa879c62880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-08T23:29:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:49.374+0000 7fa879c62880 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-08T23:29:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:49.374+0000 7fa879c62880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T23:29:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:49.374+0000 7fa879c62880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T23:29:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:49.374+0000 7fa879c62880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T23:29:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:29:49.378+0000 7fa879c62880 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-08T23:29:52.378 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-08T23:29:52.435 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-08T23:29:52.453 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T23:29:52.453 INFO:tasks.ceph:role: osd.4 2024-05-08T23:29:52.453 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:29:52.453 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T23:29:53.023 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:29:53.023 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:29:53.023 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:29:53.024 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 2024-05-08T23:29:53.024 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:29:53.024 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:29:53.024 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:29:53.024 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:29:53.024 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:29:53.024 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:29:53.025 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-05-08T23:29:53.026 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-08T23:29:53.115 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-08T23:29:53.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:29:53.130 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T23:29:53.130 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-08T23:29:53.140 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T23:29:53.141 INFO:tasks.ceph:role: osd.5 2024-05-08T23:29:53.141 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:29:53.141 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T23:29:53.676 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:29:53.676 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:29:53.676 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:29:53.676 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 2024-05-08T23:29:53.676 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:29:53.676 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:29:53.676 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:29:53.676 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:29:53.676 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:29:53.676 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:29:53.678 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-05-08T23:29:53.678 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-08T23:29:53.728 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-08T23:29:53.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:29:53.742 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T23:29:53.742 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-08T23:29:53.752 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T23:29:53.753 INFO:tasks.ceph:role: osd.6 2024-05-08T23:29:53.753 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:29:53.753 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T23:29:54.210 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:29:54.210 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:29:54.210 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:29:54.210 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 2024-05-08T23:29:54.210 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:29:54.210 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:29:54.210 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:29:54.210 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:29:54.210 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:29:54.210 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:29:54.212 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-05-08T23:29:54.212 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-08T23:29:54.271 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-08T23:29:54.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:29:54.283 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T23:29:54.283 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-08T23:29:54.294 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T23:29:54.294 INFO:tasks.ceph:role: osd.7 2024-05-08T23:29:54.295 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:29:54.295 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T23:29:54.787 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:29:54.787 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:29:54.787 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:29:54.787 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 2024-05-08T23:29:54.787 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:29:54.787 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:29:54.787 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:29:54.787 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:29:54.788 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:29:54.788 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:29:54.789 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-05-08T23:29:54.789 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-08T23:29:54.843 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-08T23:29:54.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:29:54.855 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: /sbin/restorecon: command not found 2024-05-08T23:29:54.855 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:29:54.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:29:54.932+0000 7f16e4a1d880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-08T23:29:54.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:29:54.932+0000 7f16e4a1d880 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-08T23:29:54.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:29:54.932+0000 7f16e4a1d880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T23:29:54.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:29:54.932+0000 7f16e4a1d880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T23:29:54.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:29:54.932+0000 7f16e4a1d880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T23:29:54.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:29:54.932+0000 7f16e4a1d880 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-08T23:29:57.961 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-08T23:29:57.978 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:29:58.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:29:58.020+0000 7fbd17a02880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-08T23:29:58.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:29:58.020+0000 7fbd17a02880 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-08T23:29:58.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:29:58.020+0000 7fbd17a02880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T23:29:58.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:29:58.020+0000 7fbd17a02880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T23:29:58.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:29:58.020+0000 7fbd17a02880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T23:29:58.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:29:58.020+0000 7fbd17a02880 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-08T23:30:01.053 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-08T23:30:01.068 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:30:01.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:01.156+0000 7f1c23b2d880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-08T23:30:01.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:01.156+0000 7f1c23b2d880 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-08T23:30:01.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:01.156+0000 7f1c23b2d880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T23:30:01.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:01.156+0000 7f1c23b2d880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T23:30:01.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:01.156+0000 7f1c23b2d880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T23:30:01.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:01.156+0000 7f1c23b2d880 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-08T23:30:04.132 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-08T23:30:04.147 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:30:04.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:04.188+0000 7fc99ee4f880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-08T23:30:04.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:04.188+0000 7fc99ee4f880 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-08T23:30:04.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:04.188+0000 7fc99ee4f880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T23:30:04.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:04.188+0000 7fc99ee4f880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T23:30:04.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:04.188+0000 7fc99ee4f880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T23:30:04.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:04.188+0000 7fc99ee4f880 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-08T23:30:05.883 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:30:05.885 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:30:07.204 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-08T23:30:07.221 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-08T23:30:07.221 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:30:07.221 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-08T23:30:07.237 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:30:07.237 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-08T23:30:07.295 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:30:07.295 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-08T23:30:07.311 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:30:07.311 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-08T23:30:07.320 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:30:07.320 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-08T23:30:07.378 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:30:07.378 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-08T23:30:07.388 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:30:07.388 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-08T23:30:07.399 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:30:07.399 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-08T23:30:07.415 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:30:07.415 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-08T23:30:07.473 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:30:07.473 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-08T23:30:07.483 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:30:07.484 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-08T23:30:07.496 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:30:07.496 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-08T23:30:07.506 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:30:07.506 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-08T23:30:07.521 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:30:07.521 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-08T23:30:07.532 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:30:07.532 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-08T23:30:07.539 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:30:07.539 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-08T23:30:07.545 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:30:07.545 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-08T23:30:07.579 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:30:07.579 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-08T23:30:07.585 INFO:tasks.ceph:Adding keys to all mons... 2024-05-08T23:30:07.586 DEBUG:teuthology.orchestra.run.smithi081:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T23:30:07.588 DEBUG:teuthology.orchestra.run.smithi117:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T23:30:07.599 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr.y] 2024-05-08T23:30:07.599 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDgCjxmgG3AOBAAK6T63BGID2I5zE0Z3VR4TQ== 2024-05-08T23:30:07.599 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2024-05-08T23:30:07.599 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDhCjxmmtpFBBAAO8W7v1ZRGwo4hFj/fYPECw== 2024-05-08T23:30:07.599 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c] 2024-05-08T23:30:07.599 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDhCjxmXD37BxAALgQN2Yixer9fPYj2YdLW/w== 2024-05-08T23:30:07.599 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.0] 2024-05-08T23:30:07.600 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDkCjxmUOfLCBAAc0gYW9BV8BE+h7Qhhc01IQ== 2024-05-08T23:30:07.600 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.1] 2024-05-08T23:30:07.600 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDnCjxmKcSHDBAA5dF6MWsChaZUZQskZYtrKg== 2024-05-08T23:30:07.600 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.2] 2024-05-08T23:30:07.600 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDqCjxmeYjnEhAAAVMSJ8jGR+dThJL51VsQxA== 2024-05-08T23:30:07.600 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.3] 2024-05-08T23:30:07.600 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDtCjxmlPycFhAAx+amZqgwyvwydi2d5dBZmg== 2024-05-08T23:30:07.600 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr.x] 2024-05-08T23:30:07.600 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDhCjxmUtdWABAARBlm/xpvB6mlamPG1+thYg== 2024-05-08T23:30:07.600 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2024-05-08T23:30:07.600 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDhCjxms5vJCxAA7vARVP0vvogEEps6DJFK1g== 2024-05-08T23:30:07.600 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d] 2024-05-08T23:30:07.600 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDhCjxmpIKFDxAACxYpOsx1laPsl0QNG+NFdw== 2024-05-08T23:30:07.600 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.4] 2024-05-08T23:30:07.600 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDyCjxmcnu+NxAAJ5cn9LG2uTjAZzW2Qv2y5A== 2024-05-08T23:30:07.601 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.5] 2024-05-08T23:30:07.601 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQD2CjxmQNJoARAAfINQxQDKESMqyuxmEFwwJA== 2024-05-08T23:30:07.601 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.6] 2024-05-08T23:30:07.601 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQD5CjxmGcKCCRAApkAe4V2JvttFUqcwVa7PbA== 2024-05-08T23:30:07.601 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.7] 2024-05-08T23:30:07.601 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQD8CjxmIbmICxAAqF8ZE7Gxj1pLKuaCvqOIRw== 2024-05-08T23:30:07.601 INFO:teuthology.orchestra.run.smithi117.stdout:[client.0] 2024-05-08T23:30:07.601 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDhCjxmQWD3EhAAkS5P2FRmiLBI9qjk3JjiyA== 2024-05-08T23:30:07.601 INFO:teuthology.orchestra.run.smithi117.stdout:[client.2] 2024-05-08T23:30:07.601 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDhCjxmhpPVFRAA2gIDHl4KehKcgXKX0Rs9tg== 2024-05-08T23:30:07.601 INFO:teuthology.orchestra.run.smithi117.stdout:[client.1] 2024-05-08T23:30:07.601 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDhCjxmuQBJGRAAA7KQ8CtfMfkVgM89n1QBMA== 2024-05-08T23:30:07.601 INFO:teuthology.orchestra.run.smithi117.stdout:[client.3] 2024-05-08T23:30:07.601 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDhCjxmf6gIHxAARsUHPVNk3HzYs3lQyqt2Cg== 2024-05-08T23:30:07.603 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr.y] 2024-05-08T23:30:07.603 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQDgCjxmgG3AOBAAK6T63BGID2I5zE0Z3VR4TQ== 2024-05-08T23:30:07.603 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2024-05-08T23:30:07.603 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQDhCjxmmtpFBBAAO8W7v1ZRGwo4hFj/fYPECw== 2024-05-08T23:30:07.603 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2024-05-08T23:30:07.603 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQDhCjxmXD37BxAALgQN2Yixer9fPYj2YdLW/w== 2024-05-08T23:30:07.603 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.0] 2024-05-08T23:30:07.603 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQDkCjxmUOfLCBAAc0gYW9BV8BE+h7Qhhc01IQ== 2024-05-08T23:30:07.603 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.1] 2024-05-08T23:30:07.603 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQDnCjxmKcSHDBAA5dF6MWsChaZUZQskZYtrKg== 2024-05-08T23:30:07.604 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.2] 2024-05-08T23:30:07.604 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQDqCjxmeYjnEhAAAVMSJ8jGR+dThJL51VsQxA== 2024-05-08T23:30:07.604 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.3] 2024-05-08T23:30:07.604 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQDtCjxmlPycFhAAx+amZqgwyvwydi2d5dBZmg== 2024-05-08T23:30:07.604 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr.x] 2024-05-08T23:30:07.604 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQDhCjxmUtdWABAARBlm/xpvB6mlamPG1+thYg== 2024-05-08T23:30:07.604 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2024-05-08T23:30:07.604 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQDhCjxms5vJCxAA7vARVP0vvogEEps6DJFK1g== 2024-05-08T23:30:07.604 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2024-05-08T23:30:07.604 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQDhCjxmpIKFDxAACxYpOsx1laPsl0QNG+NFdw== 2024-05-08T23:30:07.604 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.4] 2024-05-08T23:30:07.604 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQDyCjxmcnu+NxAAJ5cn9LG2uTjAZzW2Qv2y5A== 2024-05-08T23:30:07.604 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.5] 2024-05-08T23:30:07.604 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQD2CjxmQNJoARAAfINQxQDKESMqyuxmEFwwJA== 2024-05-08T23:30:07.604 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.6] 2024-05-08T23:30:07.605 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQD5CjxmGcKCCRAApkAe4V2JvttFUqcwVa7PbA== 2024-05-08T23:30:07.605 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.7] 2024-05-08T23:30:07.605 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQD8CjxmIbmICxAAqF8ZE7Gxj1pLKuaCvqOIRw== 2024-05-08T23:30:07.605 INFO:teuthology.orchestra.run.smithi081.stdout:[client.0] 2024-05-08T23:30:07.605 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQDhCjxmQWD3EhAAkS5P2FRmiLBI9qjk3JjiyA== 2024-05-08T23:30:07.605 INFO:teuthology.orchestra.run.smithi081.stdout:[client.2] 2024-05-08T23:30:07.605 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQDhCjxmhpPVFRAA2gIDHl4KehKcgXKX0Rs9tg== 2024-05-08T23:30:07.605 INFO:teuthology.orchestra.run.smithi081.stdout:[client.1] 2024-05-08T23:30:07.605 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQDhCjxmuQBJGRAAA7KQ8CtfMfkVgM89n1QBMA== 2024-05-08T23:30:07.605 INFO:teuthology.orchestra.run.smithi081.stdout:[client.3] 2024-05-08T23:30:07.605 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQDhCjxmf6gIHxAARsUHPVNk3HzYs3lQyqt2Cg== 2024-05-08T23:30:07.605 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T23:30:07.649 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T23:30:07.701 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:30:07.703 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:30:07.754 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:30:07.757 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:30:07.807 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:07.809 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:07.855 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:07.860 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:07.912 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:07.914 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:07.961 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:07.963 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:08.009 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T23:30:08.012 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T23:30:08.103 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:30:08.106 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:30:08.158 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:30:08.160 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:30:08.213 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:08.249 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:08.300 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:08.302 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:08.350 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:08.352 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:08.399 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:08.401 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:30:08.448 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:30:08.450 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:30:08.502 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:30:08.504 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:30:08.556 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:30:08.559 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:30:08.611 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:30:08.613 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:30:08.662 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-08T23:30:08.662 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-08T23:30:08.677 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T23:30:08.765 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-08T23:30:08.780 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-08T23:30:08.797 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T23:30:08.923 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-08T23:30:08.940 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-08T23:30:08.949 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T23:30:09.043 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-08T23:30:09.059 DEBUG:teuthology.orchestra.run.smithi081:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:30:09.061 DEBUG:teuthology.orchestra.run.smithi117:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:30:09.066 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-08T23:30:09.066 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T23:30:09.067 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T23:30:09.109 INFO:tasks.ceph.mon.a:Started 2024-05-08T23:30:09.109 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T23:30:09.109 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T23:30:09.111 INFO:tasks.ceph.mon.b:Started 2024-05-08T23:30:09.112 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T23:30:09.112 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T23:30:09.114 INFO:tasks.ceph.mon.c:Started 2024-05-08T23:30:09.114 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-08T23:30:09.114 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-08T23:30:09.114 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-08T23:30:09.116 INFO:tasks.ceph.mgr.y:Started 2024-05-08T23:30:09.116 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-08T23:30:09.117 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-08T23:30:09.119 INFO:tasks.ceph.mgr.x:Started 2024-05-08T23:30:09.119 INFO:tasks.ceph:Setting crush tunables to default 2024-05-08T23:30:09.119 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd crush tunables default 2024-05-08T23:30:09.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7bb175700 1 Processor -- start 2024-05-08T23:30:09.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7bb175700 1 -- start start 2024-05-08T23:30:09.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7bb175700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 0x7fa7b4150410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:09.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7bb175700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 0x7fa7b40ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:09.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7bb175700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b4159980 con 0x7fa7b40b1fd0 2024-05-08T23:30:09.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7bb175700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b4150950 con 0x7fa7b40ce2d0 2024-05-08T23:30:09.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7bb175700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b40ceca0 con 0x7fa7b4150030 2024-05-08T23:30:09.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7b9712700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 0x7fa7b40ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:09.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7b8f11700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 0x7fa7b4150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:09.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7b9712700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 0x7fa7b40ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47144/0 (socket says 172.21.15.81:47144) 2024-05-08T23:30:09.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7b8f11700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 0x7fa7b4150410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:33692/0 (socket says 172.21.15.81:33692) 2024-05-08T23:30:09.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7b9712700 1 -- 172.21.15.81:0/2479198068 learned_addr learned my addr 172.21.15.81:0/2479198068 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:09.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7b3fff700 1 --1- 172.21.15.81:0/2479198068 >> v1:172.21.15.117:6789/0 conn(0x7fa7b40b1fd0 0x7fa7b406daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:35240/0 (socket says 172.21.15.81:35240) 2024-05-08T23:30:09.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7b9712700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 msgr2=0x7fa7b40ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:30:09.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7b9712700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 msgr2=0x7fa7b40ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:09.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 0x7fa7b40ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:09.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 msgr2=0x7fa7b4150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:30:09.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 msgr2=0x7fa7b4150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:09.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 0x7fa7b4150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:09.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 0x7fa7b4150410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:09.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.218+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 0x7fa7b40ce760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:09.424 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.422+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 0x7fa7b40ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:09.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.422+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 0x7fa7b4150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:09.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.422+0000 7fa7b9712700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 msgr2=0x7fa7b40ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:30:09.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.422+0000 7fa7b9712700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 msgr2=0x7fa7b40ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:09.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.422+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 0x7fa7b40ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:09.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.422+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 0x7fa7b40ce760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:30:09.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.422+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 msgr2=0x7fa7b4150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:30:09.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.422+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 msgr2=0x7fa7b4150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:09.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.422+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 0x7fa7b4150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:09.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.422+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 0x7fa7b4150410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:30:09.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.826+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 0x7fa7b40ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:09.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.826+0000 7fa7b9712700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 msgr2=0x7fa7b40ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:30:09.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.826+0000 7fa7b9712700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 msgr2=0x7fa7b40ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:09.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.826+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 0x7fa7b40ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:09.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.826+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 0x7fa7b40ce760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:30:09.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.826+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 0x7fa7b4150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:09.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.826+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 msgr2=0x7fa7b4150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:30:09.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.826+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 msgr2=0x7fa7b4150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:09.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.826+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 0x7fa7b4150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:09.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:09.826+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 0x7fa7b4150410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:30:10.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:10.630+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 0x7fa7b40ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:10.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:10.630+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 0x7fa7b4150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:10.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:10.630+0000 7fa7b9712700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 msgr2=0x7fa7b40ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:30:10.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:10.630+0000 7fa7b9712700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 msgr2=0x7fa7b40ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:10.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:10.630+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 0x7fa7b40ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:10.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:10.630+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 0x7fa7b40ce760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:30:10.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:10.630+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 msgr2=0x7fa7b4150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:30:10.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:10.630+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 msgr2=0x7fa7b4150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:10.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:10.630+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 0x7fa7b4150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:10.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:10.630+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 0x7fa7b4150410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:30:12.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.218+0000 7fa7b27fc700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 msgr2=0x7fa7b4150410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:12.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.218+0000 7fa7b27fc700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.117:3300/0 conn(0x7fa7b4150030 0x7fa7b4150410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:12.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.218+0000 7fa7b27fc700 1 -- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 msgr2=0x7fa7b40ce760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:12.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.218+0000 7fa7b27fc700 1 --2- 172.21.15.81:0/2479198068 >> v2:172.21.15.81:3300/0 conn(0x7fa7b40ce2d0 0x7fa7b40ce760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:12.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.218+0000 7fa7b27fc700 1 -- 172.21.15.81:0/2479198068 >> v1:172.21.15.117:6789/0 conn(0x7fa7b40b1fd0 legacy=0x7fa7b406daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:12.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.218+0000 7fa7b27fc700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7a0005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:12.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.218+0000 7fa7b27fc700 1 -- 172.21.15.81:0/2479198068 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b406e1b0 con 0x7fa7a00061f0 2024-05-08T23:30:12.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.218+0000 7fa7b27fc700 1 -- 172.21.15.81:0/2479198068 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b40ceca0 con 0x7fa7a0003730 2024-05-08T23:30:12.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.218+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7a0005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:12.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.222+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 msgr2=0x7fa7a0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:30:12.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.222+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 msgr2=0x7fa7a0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:12.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.222+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7a0005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:12.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.222+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7a0005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:12.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.426+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7a0005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:12.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.426+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 msgr2=0x7fa7a0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:30:12.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.426+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 msgr2=0x7fa7a0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:12.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.426+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7a0005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:12.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.426+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7a0005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:30:12.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.830+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7a0005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:12.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.830+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 msgr2=0x7fa7a0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:30:12.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.830+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 msgr2=0x7fa7a0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:12.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.830+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7a0005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:12.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:12.830+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7a0005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:30:13.636 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:13.634+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7a0005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:13.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:13.634+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 msgr2=0x7fa7a0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:30:13.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:13.634+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 msgr2=0x7fa7a0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:13.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:13.634+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7a0005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:13.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:13.634+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7a0005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:30:14.294 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b37fe700 1 -- 172.21.15.81:0/2479198068 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897259559 0 0) 0x7fa7b406e1b0 con 0x7fa7a00061f0 2024-05-08T23:30:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b37fe700 1 -- 172.21.15.81:0/2479198068 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa79c0035d0 con 0x7fa7a00061f0 2024-05-08T23:30:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b37fe700 1 -- 172.21.15.81:0/2479198068 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4061596290 0 0) 0x7fa79c0035d0 con 0x7fa7a00061f0 2024-05-08T23:30:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b37fe700 1 -- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 msgr2=0x7fa7a0005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b37fe700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7a0005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b37fe700 1 -- 172.21.15.81:0/2479198068 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a000afb0 con 0x7fa7a00061f0 2024-05-08T23:30:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b37fe700 1 -- 172.21.15.81:0/2479198068 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7fa7a80016f0 con 0x7fa7a00061f0 2024-05-08T23:30:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b37fe700 1 -- 172.21.15.81:0/2479198068 >> v1:172.21.15.81:6789/0 conn(0x7fa7a00061f0 legacy=0x7fa7a0008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b9712700 1 -- 172.21.15.81:0/2479198068 reap_dead start 2024-05-08T23:30:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b37fe700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa79c003a20 0x7fa79c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b37fe700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa79c004870 0x7fa79c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b37fe700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa79c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b37fe700 1 -- 172.21.15.81:0/2479198068 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b40ceca0 con 0x7fa79c003a20 2024-05-08T23:30:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b37fe700 1 -- 172.21.15.81:0/2479198068 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c005560 con 0x7fa79c004870 2024-05-08T23:30:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b37fe700 1 -- 172.21.15.81:0/2479198068 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c005720 con 0x7fa7a0003730 2024-05-08T23:30:14.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa79c003a20 0x7fa79c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa79c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b3fff700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa79c004870 0x7fa79c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b9712700 1 -- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 msgr2=0x7fa79c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:30:14.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b9712700 1 -- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 msgr2=0x7fa79c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:14.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa79c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:14.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa79c00ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:14.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 msgr2=0x7fa79c00ba30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:14.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa79c00ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa79c004870 msgr2=0x7fa79c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:14.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa79c004870 0x7fa79c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.290+0000 7fa7b8f11700 1 -- 172.21.15.81:0/2479198068 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c0035d0 con 0x7fa79c003a20 2024-05-08T23:30:14.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.294+0000 7fa7b8f11700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa79c003a20 0x7fa79c003e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009100 tx=0x7fa79c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127873ab7a0286ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:14.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.294+0000 7fa7b37fe700 1 -- 172.21.15.81:0/2479198068 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a401e070 con 0x7fa79c003a20 2024-05-08T23:30:14.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.294+0000 7fa7b37fe700 1 -- 172.21.15.81:0/2479198068 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a400c040 con 0x7fa79c003a20 2024-05-08T23:30:14.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.294+0000 7fa7b37fe700 1 -- 172.21.15.81:0/2479198068 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4019740 con 0x7fa79c003a20 2024-05-08T23:30:14.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.294+0000 7fa7bb175700 1 -- 172.21.15.81:0/2479198068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa79c003a20 msgr2=0x7fa79c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:14.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.294+0000 7fa7bb175700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa79c003a20 0x7fa79c003e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009100 tx=0x7fa79c0058a0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.294+0000 7fa7bb175700 1 -- 172.21.15.81:0/2479198068 shutdown_connections 2024-05-08T23:30:14.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.294+0000 7fa7bb175700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa79c00ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.294+0000 7fa7bb175700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa79c004870 0x7fa79c004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.294+0000 7fa7bb175700 1 --2- 172.21.15.81:0/2479198068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa79c003a20 0x7fa79c003e20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.294+0000 7fa7bb175700 1 -- 172.21.15.81:0/2479198068 >> 172.21.15.81:0/2479198068 conn(0x7fa7b40b0cd0 msgr2=0x7fa7b40b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:14.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.294+0000 7fa7bb175700 1 -- 172.21.15.81:0/2479198068 shutdown_connections 2024-05-08T23:30:14.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.294+0000 7fa7bb175700 1 -- 172.21.15.81:0/2479198068 wait complete. 2024-05-08T23:30:14.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.298+0000 7fa7bb175700 1 Processor -- start 2024-05-08T23:30:14.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.298+0000 7fa7bb175700 1 -- start start 2024-05-08T23:30:14.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7bb175700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa79c004870 0x7fa7b4138ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7bb175700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7b41397d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7bb175700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b41533b0 0x7fa7b4139d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7bb175700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b4153bd0 con 0x7fa7b41533b0 2024-05-08T23:30:14.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7bb175700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b414c5e0 con 0x7fa79c004870 2024-05-08T23:30:14.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7bb175700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b41538f0 con 0x7fa7a0003730 2024-05-08T23:30:14.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b9712700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b41533b0 0x7fa7b4139d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b9712700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b41533b0 0x7fa7b4139d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47232/0 (socket says 172.21.15.81:47232) 2024-05-08T23:30:14.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b9712700 1 -- 172.21.15.81:0/2959166729 learned_addr learned my addr 172.21.15.81:0/2959166729 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:14.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b8f11700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa79c004870 0x7fa7b4138ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b3fff700 1 --2- 172.21.15.81:0/2959166729 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7b41397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b9712700 1 -- 172.21.15.81:0/2959166729 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 msgr2=0x7fa7b41397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:14.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2959166729 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7b41397d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b9712700 1 -- 172.21.15.81:0/2959166729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa79c004870 msgr2=0x7fa7b4138ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:14.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2959166729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa79c004870 0x7fa7b4138ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b9712700 1 -- 172.21.15.81:0/2959166729 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b414bf90 con 0x7fa7b41533b0 2024-05-08T23:30:14.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b9712700 1 --2- 172.21.15.81:0/2959166729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b41533b0 0x7fa7b4139d10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009eb0 tx=0x7fa7ac015d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:14.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b17fa700 1 -- 172.21.15.81:0/2959166729 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac003a40 con 0x7fa7b41533b0 2024-05-08T23:30:14.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b17fa700 1 -- 172.21.15.81:0/2959166729 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac010040 con 0x7fa7b41533b0 2024-05-08T23:30:14.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7bb175700 1 -- 172.21.15.81:0/2959166729 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b414c270 con 0x7fa7b41533b0 2024-05-08T23:30:14.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b17fa700 1 -- 172.21.15.81:0/2959166729 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac003c00 con 0x7fa7b41533b0 2024-05-08T23:30:14.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7bb175700 1 -- 172.21.15.81:0/2959166729 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b4139400 con 0x7fa7b41533b0 2024-05-08T23:30:14.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b17fa700 1 -- 172.21.15.81:0/2959166729 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa7ac0088d0 con 0x7fa7b41533b0 2024-05-08T23:30:14.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7b17fa700 1 -- 172.21.15.81:0/2959166729 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa7ac018440 con 0x7fa7b41533b0 2024-05-08T23:30:14.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.302+0000 7fa7bb175700 1 -- 172.21.15.81:0/2959166729 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b414bdf0 con 0x7fa7b41533b0 2024-05-08T23:30:14.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.306+0000 7fa7b17fa700 1 -- 172.21.15.81:0/2959166729 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fa7ac040dc0 con 0x7fa7b41533b0 2024-05-08T23:30:14.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.378+0000 7fa7bb175700 1 -- 172.21.15.81:0/2959166729 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa7b40710b0 con 0x7fa7b41533b0 2024-05-08T23:30:14.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.382+0000 7fa7b17fa700 1 -- 172.21.15.81:0/2959166729 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa7ac02f020 con 0x7fa7b41533b0 2024-05-08T23:30:14.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.382+0000 7fa7bb175700 1 -- 172.21.15.81:0/2959166729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b41533b0 msgr2=0x7fa7b4139d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:14.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.382+0000 7fa7bb175700 1 --2- 172.21.15.81:0/2959166729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b41533b0 0x7fa7b4139d10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009eb0 tx=0x7fa7ac015d80 comp rx=0 tx=0).stop 2024-05-08T23:30:14.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.382+0000 7fa7bb175700 1 -- 172.21.15.81:0/2959166729 shutdown_connections 2024-05-08T23:30:14.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.382+0000 7fa7bb175700 1 --2- 172.21.15.81:0/2959166729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b41533b0 0x7fa7b4139d10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.382+0000 7fa7bb175700 1 --2- 172.21.15.81:0/2959166729 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7a0003730 0x7fa7b41397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.382+0000 7fa7bb175700 1 --2- 172.21.15.81:0/2959166729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa79c004870 0x7fa7b4138ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.382+0000 7fa7bb175700 1 -- 172.21.15.81:0/2959166729 >> 172.21.15.81:0/2959166729 conn(0x7fa7b40b0cd0 msgr2=0x7fa7b40ccb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:14.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.382+0000 7fa7bb175700 1 -- 172.21.15.81:0/2959166729 shutdown_connections 2024-05-08T23:30:14.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:14.382+0000 7fa7bb175700 1 -- 172.21.15.81:0/2959166729 wait complete. 2024-05-08T23:30:14.388 INFO:teuthology.orchestra.run.smithi081.stderr:adjusted tunables profile to default 2024-05-08T23:30:14.399 INFO:tasks.ceph:check_enable_crimson: False 2024-05-08T23:30:14.399 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-08T23:30:14.399 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:30:14.399 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-08T23:30:14.412 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:30:14.412 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-08T23:30:14.422 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:30:14.422 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-08T23:30:14.431 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:30:14.431 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-08T23:30:14.440 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:30:14.440 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-08T23:30:14.457 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:30:14.457 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-08T23:30:14.480 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:14.478+0000 7f937ff502c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:30:14.490 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:14.487+0000 7fae133c22c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:30:14.503 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:30:14.503 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-08T23:30:14.560 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:30:14.560 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-08T23:30:14.569 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 1d689f2b-d609-4b84-a66c-04b9ec10ba36 0 2024-05-08T23:30:14.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.675+0000 7f68af1a2700 1 Processor -- start 2024-05-08T23:30:14.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.675+0000 7f68af1a2700 1 -- start start 2024-05-08T23:30:14.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.675+0000 7f68af1a2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a80ce9e0 0x7f68a80ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.675+0000 7f68af1a2700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f68a80cf380 0x7f68a80cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.675+0000 7f68af1a2700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a815f9d0 con 0x7f68a815b020 2024-05-08T23:30:14.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.675+0000 7f68af1a2700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a815f220 con 0x7f68a80cf380 2024-05-08T23:30:14.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.675+0000 7f68af1a2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a80cfd50 con 0x7f68a80ce9e0 2024-05-08T23:30:14.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.675+0000 7f68ad99f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a80ce9e0 0x7f68a80ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.675+0000 7f68ae9a1700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f68a80cf380 0x7f68a80cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.675+0000 7f68ae1a0700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f68a815b020 0x7f68a815eb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:34362/0 (socket says 172.21.15.117:34362) 2024-05-08T23:30:14.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.675+0000 7f68ae1a0700 1 -- 172.21.15.117:0/1765996756 learned_addr learned my addr 172.21.15.117:0/1765996756 (peer_addr_for_me v1:172.21.15.117:0/0) 2024-05-08T23:30:14.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.675+0000 7f68ad99f700 1 -- 172.21.15.117:0/1765996756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a80ce9e0 msgr2=0x7f68a80ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:30:14.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.675+0000 7f68ad99f700 1 -- 172.21.15.117:0/1765996756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a80ce9e0 msgr2=0x7f68a80ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:14.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.675+0000 7f68ad99f700 1 --2- 172.21.15.117:0/1765996756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a80ce9e0 0x7f68a80ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:14.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68ad99f700 1 --2- 172.21.15.117:0/1765996756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a80ce9e0 0x7f68a80ce020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:14.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68ae9a1700 1 -- 172.21.15.117:0/1765996756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a80ce9e0 msgr2=0x7f68a80ce020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:14.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68ae9a1700 1 --2- 172.21.15.117:0/1765996756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a80ce9e0 0x7f68a80ce020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68ae9a1700 1 -- 172.21.15.117:0/1765996756 >> v1:172.21.15.117:6789/0 conn(0x7f68a815b020 legacy=0x7f68a815eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:14.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68ad19e700 1 -- 172.21.15.117:0/1765996756 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299399099 0 0) 0x7f68a815f9d0 con 0x7f68a815b020 2024-05-08T23:30:14.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68ae9a1700 1 -- 172.21.15.117:0/1765996756 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a80cdb90 con 0x7f68a80cf380 2024-05-08T23:30:14.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68ae9a1700 1 --2- 172.21.15.117:0/1765996756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a80cf380 0x7f68a80cf810 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009d90 tx=0x7f68a000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=938ef9ddd1333713 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:14.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68ad19e700 1 -- 172.21.15.117:0/1765996756 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a000e070 con 0x7f68a80cf380 2024-05-08T23:30:14.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68ad19e700 1 -- 172.21.15.117:0/1765996756 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a000bbd0 con 0x7f68a80cf380 2024-05-08T23:30:14.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68ad19e700 1 -- 172.21.15.117:0/1765996756 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a0012430 con 0x7f68a80cf380 2024-05-08T23:30:14.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68af1a2700 1 -- 172.21.15.117:0/1765996756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a80cf380 msgr2=0x7f68a80cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:14.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68af1a2700 1 --2- 172.21.15.117:0/1765996756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a80cf380 0x7f68a80cf810 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009d90 tx=0x7f68a000b330 comp rx=0 tx=0).stop 2024-05-08T23:30:14.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68af1a2700 1 -- 172.21.15.117:0/1765996756 shutdown_connections 2024-05-08T23:30:14.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68af1a2700 1 --2- 172.21.15.117:0/1765996756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a80cf380 0x7f68a80cf810 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68af1a2700 1 --2- 172.21.15.117:0/1765996756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a80ce9e0 0x7f68a80ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68af1a2700 1 -- 172.21.15.117:0/1765996756 >> 172.21.15.117:0/1765996756 conn(0x7f68a8099a70 msgr2=0x7f68a8099e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:14.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68af1a2700 1 -- 172.21.15.117:0/1765996756 shutdown_connections 2024-05-08T23:30:14.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68af1a2700 1 -- 172.21.15.117:0/1765996756 wait complete. 2024-05-08T23:30:14.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68af1a2700 1 Processor -- start 2024-05-08T23:30:14.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.679+0000 7f68af1a2700 1 -- start start 2024-05-08T23:30:14.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68af1a2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68a80ce9e0 0x7f68a8156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68af1a2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a80cf380 0x7f68a80b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68af1a2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a815b020 0x7f68a8159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68af1a2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a80b6b30 con 0x7f68a80ce9e0 2024-05-08T23:30:14.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68af1a2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a815c150 con 0x7f68a80cf380 2024-05-08T23:30:14.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68af1a2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a815c3f0 con 0x7f68a815b020 2024-05-08T23:30:14.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ae9a1700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a815b020 0x7f68a8159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ad99f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a80cf380 0x7f68a80b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ae9a1700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a815b020 0x7f68a8159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:35606/0 (socket says 172.21.15.117:35606) 2024-05-08T23:30:14.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ae9a1700 1 -- 172.21.15.117:0/551281081 learned_addr learned my addr 172.21.15.117:0/551281081 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-05-08T23:30:14.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ad99f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a80cf380 0x7f68a80b6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41040/0 (socket says 172.21.15.117:41040) 2024-05-08T23:30:14.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ae9a1700 1 -- 172.21.15.117:0/551281081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a815b020 msgr2=0x7f68a8159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-08T23:30:14.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ae9a1700 1 -- 172.21.15.117:0/551281081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a815b020 msgr2=0x7f68a8159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:30:14.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ae9a1700 1 --2- 172.21.15.117:0/551281081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a815b020 0x7f68a8159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:14.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ae1a0700 1 --2- 172.21.15.117:0/551281081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68a80ce9e0 0x7f68a8156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ae9a1700 1 --2- 172.21.15.117:0/551281081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a815b020 0x7f68a8159390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:14.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ad99f700 1 -- 172.21.15.117:0/551281081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a815b020 msgr2=0x7f68a8159390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:30:14.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ad99f700 1 --2- 172.21.15.117:0/551281081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a815b020 0x7f68a8159390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ad99f700 1 -- 172.21.15.117:0/551281081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68a80ce9e0 msgr2=0x7f68a8156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:14.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ad99f700 1 --2- 172.21.15.117:0/551281081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68a80ce9e0 0x7f68a8156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ad99f700 1 -- 172.21.15.117:0/551281081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a0009a70 con 0x7f68a80cf380 2024-05-08T23:30:14.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ae1a0700 1 --2- 172.21.15.117:0/551281081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68a80ce9e0 0x7f68a8156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:14.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68ad99f700 1 --2- 172.21.15.117:0/551281081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a80cf380 0x7f68a80b6270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f689800d510 tx=0x7f689800be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:14.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f689effd700 1 -- 172.21.15.117:0/551281081 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898004220 con 0x7f68a80cf380 2024-05-08T23:30:14.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f689effd700 1 -- 172.21.15.117:0/551281081 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6898013040 con 0x7f68a80cf380 2024-05-08T23:30:14.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68af1a2700 1 -- 172.21.15.117:0/551281081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a815c6d0 con 0x7f68a80cf380 2024-05-08T23:30:14.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f689effd700 1 -- 172.21.15.117:0/551281081 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898011890 con 0x7f68a80cf380 2024-05-08T23:30:14.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68af1a2700 1 -- 172.21.15.117:0/551281081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a81574b0 con 0x7f68a80cf380 2024-05-08T23:30:14.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f689effd700 1 -- 172.21.15.117:0/551281081 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f6898011cf0 con 0x7f68a80cf380 2024-05-08T23:30:14.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f689effd700 1 -- 172.21.15.117:0/551281081 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f68980194d0 con 0x7f68a80cf380 2024-05-08T23:30:14.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.683+0000 7f68af1a2700 1 -- 172.21.15.117:0/551281081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a815cbf0 con 0x7f68a80cf380 2024-05-08T23:30:14.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.687+0000 7f689effd700 1 -- 172.21.15.117:0/551281081 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f68a815cbf0 con 0x7f68a80cf380 2024-05-08T23:30:14.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.759+0000 7f68af1a2700 1 -- 172.21.15.117:0/551281081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1d689f2b-d609-4b84-a66c-04b9ec10ba36", "id": 0} v 0) v1 -- 0x7f68a80672a0 con 0x7f68a80cf380 2024-05-08T23:30:14.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.771+0000 7f689effd700 1 -- 172.21.15.117:0/551281081 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1d689f2b-d609-4b84-a66c-04b9ec10ba36", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6898027060 con 0x7f68a80cf380 2024-05-08T23:30:14.774 INFO:teuthology.orchestra.run.smithi117.stdout:0 2024-05-08T23:30:14.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.771+0000 7f68af1a2700 1 -- 172.21.15.117:0/551281081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a80cf380 msgr2=0x7f68a80b6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:14.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.771+0000 7f68af1a2700 1 --2- 172.21.15.117:0/551281081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a80cf380 0x7f68a80b6270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f689800d510 tx=0x7f689800be50 comp rx=0 tx=0).stop 2024-05-08T23:30:14.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.771+0000 7f68af1a2700 1 -- 172.21.15.117:0/551281081 shutdown_connections 2024-05-08T23:30:14.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.771+0000 7f68af1a2700 1 --2- 172.21.15.117:0/551281081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68a815b020 0x7f68a8159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.771+0000 7f68af1a2700 1 --2- 172.21.15.117:0/551281081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a80cf380 0x7f68a80b6270 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.771+0000 7f68af1a2700 1 --2- 172.21.15.117:0/551281081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68a80ce9e0 0x7f68a8156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.771+0000 7f68af1a2700 1 -- 172.21.15.117:0/551281081 >> 172.21.15.117:0/551281081 conn(0x7f68a8099a70 msgr2=0x7f68a80cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:14.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.771+0000 7f68af1a2700 1 -- 172.21.15.117:0/551281081 shutdown_connections 2024-05-08T23:30:14.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.771+0000 7f68af1a2700 1 -- 172.21.15.117:0/551281081 wait complete. 2024-05-08T23:30:14.788 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 32ab34f3-4f16-44dd-9f62-91dc5e94cf83 1 2024-05-08T23:30:14.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.891+0000 7f441f59e700 1 Processor -- start 2024-05-08T23:30:14.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.891+0000 7f441f59e700 1 -- start start 2024-05-08T23:30:14.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.891+0000 7f441f59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f44200c3fa0 0x7f44200b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.891+0000 7f441f59e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44200b37e0 con 0x7f44200cde50 2024-05-08T23:30:14.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.891+0000 7f441f59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4420161c60 con 0x7f44200c4400 2024-05-08T23:30:14.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.891+0000 7f441f59e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44200b1fd0 con 0x7f44200c3fa0 2024-05-08T23:30:14.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441ed9d700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f44200c4400 0x7f44200cf960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:34376/0 (socket says 172.21.15.117:34376) 2024-05-08T23:30:14.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441ed9d700 1 -- 172.21.15.117:0/216861168 learned_addr learned my addr 172.21.15.117:0/216861168 (peer_addr_for_me v1:172.21.15.117:0/0) 2024-05-08T23:30:14.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441dd9b700 1 --2- 172.21.15.117:0/216861168 >> v2:172.21.15.81:3300/0 conn(0x7f44200c3fa0 0x7f44200b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 -- 172.21.15.117:0/216861168 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968858022 0 0) 0x7f4420161c60 con 0x7f44200c4400 2024-05-08T23:30:14.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 -- 172.21.15.117:0/216861168 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44080035d0 con 0x7f44200c4400 2024-05-08T23:30:14.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 -- 172.21.15.117:0/216861168 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1053600305 0 0) 0x7f44080035d0 con 0x7f44200c4400 2024-05-08T23:30:14.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 -- 172.21.15.117:0/216861168 >> v2:172.21.15.81:3300/0 conn(0x7f44200c3fa0 msgr2=0x7f44200b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:14.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 --2- 172.21.15.117:0/216861168 >> v2:172.21.15.81:3300/0 conn(0x7f44200c3fa0 0x7f44200b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 -- 172.21.15.117:0/216861168 >> v1:172.21.15.81:6789/0 conn(0x7f44200cde50 legacy=0x7f44200b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:14.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 -- 172.21.15.117:0/216861168 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44200b22b0 con 0x7f44200c4400 2024-05-08T23:30:14.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 -- 172.21.15.117:0/216861168 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f44180019f0 con 0x7f44200c4400 2024-05-08T23:30:14.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 -- 172.21.15.117:0/216861168 >> v1:172.21.15.117:6789/0 conn(0x7f44200c4400 legacy=0x7f44200cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:14.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 --2- 172.21.15.117:0/216861168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4408003a20 0x7f4408003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 --2- 172.21.15.117:0/216861168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4408004870 0x7f4408004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 --2- 172.21.15.117:0/216861168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4408005230 0x7f440800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 -- 172.21.15.117:0/216861168 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200b1fd0 con 0x7f4408005230 2024-05-08T23:30:14.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 -- 172.21.15.117:0/216861168 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440800c0d0 con 0x7f4408004870 2024-05-08T23:30:14.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 -- 172.21.15.117:0/216861168 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440800c290 con 0x7f4408003a20 2024-05-08T23:30:14.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441e59c700 1 --2- 172.21.15.117:0/216861168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4408003a20 0x7f4408003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441dd9b700 1 --2- 172.21.15.117:0/216861168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4408004870 0x7f4408004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441ed9d700 1 --2- 172.21.15.117:0/216861168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4408005230 0x7f440800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441e59c700 1 -- 172.21.15.117:0/216861168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4408003a20 msgr2=0x7f4408003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:30:14.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441e59c700 1 -- 172.21.15.117:0/216861168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4408003a20 msgr2=0x7f4408003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:14.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441e59c700 1 --2- 172.21.15.117:0/216861168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4408003a20 0x7f4408003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:14.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441e59c700 1 --2- 172.21.15.117:0/216861168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4408003a20 0x7f4408003e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:14.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441dd9b700 1 -- 172.21.15.117:0/216861168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4408003a20 msgr2=0x7f4408003e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:14.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441dd9b700 1 --2- 172.21.15.117:0/216861168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4408003a20 0x7f4408003e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441dd9b700 1 -- 172.21.15.117:0/216861168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4408005230 msgr2=0x7f440800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:14.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441dd9b700 1 --2- 172.21.15.117:0/216861168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4408005230 0x7f440800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441ed9d700 1 -- 172.21.15.117:0/216861168 reap_dead start 2024-05-08T23:30:14.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441dd9b700 1 -- 172.21.15.117:0/216861168 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44080035d0 con 0x7f4408004870 2024-05-08T23:30:14.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441dd9b700 1 --2- 172.21.15.117:0/216861168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4408004870 0x7f4408004cf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4414009b70 tx=0x7f44080058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f3376819a858bb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:14.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 -- 172.21.15.117:0/216861168 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441400e6c0 con 0x7f4408004870 2024-05-08T23:30:14.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 -- 172.21.15.117:0/216861168 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441400e820 con 0x7f4408004870 2024-05-08T23:30:14.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441d59a700 1 -- 172.21.15.117:0/216861168 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441400eb90 con 0x7f4408004870 2024-05-08T23:30:14.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441f59e700 1 -- 172.21.15.117:0/216861168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4408004870 msgr2=0x7f4408004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:14.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441f59e700 1 --2- 172.21.15.117:0/216861168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4408004870 0x7f4408004cf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4414009b70 tx=0x7f44080058a0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441f59e700 1 -- 172.21.15.117:0/216861168 shutdown_connections 2024-05-08T23:30:14.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441f59e700 1 --2- 172.21.15.117:0/216861168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4408004870 0x7f4408004cf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441f59e700 1 -- 172.21.15.117:0/216861168 >> 172.21.15.117:0/216861168 conn(0x7f44200b2970 msgr2=0x7f44200b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:14.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441f59e700 1 -- 172.21.15.117:0/216861168 shutdown_connections 2024-05-08T23:30:14.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.895+0000 7f441f59e700 1 -- 172.21.15.117:0/216861168 wait complete. 2024-05-08T23:30:14.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441f59e700 1 Processor -- start 2024-05-08T23:30:14.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441f59e700 1 -- start start 2024-05-08T23:30:14.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441f59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4408004870 0x7f44200c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441f59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4420150030 0x7f44201504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441f59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44201fadc0 0x7f44201fb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:14.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441f59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200b0dc0 con 0x7f4408004870 2024-05-08T23:30:14.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441f59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44201509f0 con 0x7f44201fadc0 2024-05-08T23:30:14.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441f59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44201624f0 con 0x7f4420150030 2024-05-08T23:30:14.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441dd9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4420150030 0x7f44201504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44201fadc0 0x7f44201fb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441dd9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4420150030 0x7f44201504b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:35622/0 (socket says 172.21.15.117:35622) 2024-05-08T23:30:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441dd9b700 1 -- 172.21.15.117:0/4166557693 learned_addr learned my addr 172.21.15.117:0/4166557693 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-05-08T23:30:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441dd9b700 1 -- 172.21.15.117:0/4166557693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4420150030 msgr2=0x7f44201504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-08T23:30:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441e59c700 1 --2- 172.21.15.117:0/4166557693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4408004870 0x7f44200c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441dd9b700 1 -- 172.21.15.117:0/4166557693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4420150030 msgr2=0x7f44201504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:30:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441dd9b700 1 --2- 172.21.15.117:0/4166557693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4420150030 0x7f44201504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441dd9b700 1 --2- 172.21.15.117:0/4166557693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4420150030 0x7f44201504b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441ed9d700 1 -- 172.21.15.117:0/4166557693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4420150030 msgr2=0x7f44201504b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:30:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441ed9d700 1 --2- 172.21.15.117:0/4166557693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4420150030 0x7f44201504b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441ed9d700 1 -- 172.21.15.117:0/4166557693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4408004870 msgr2=0x7f44200c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441ed9d700 1 --2- 172.21.15.117:0/4166557693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4408004870 0x7f44200c7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441ed9d700 1 -- 172.21.15.117:0/4166557693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44200b0ae0 con 0x7f44201fadc0 2024-05-08T23:30:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441ed9d700 1 --2- 172.21.15.117:0/4166557693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44201fadc0 0x7f44201fb240 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4420071ae0 tx=0x7f4418001a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f440effd700 1 -- 172.21.15.117:0/4166557693 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441800a900 con 0x7f44201fadc0 2024-05-08T23:30:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f440effd700 1 -- 172.21.15.117:0/4166557693 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441800aaa0 con 0x7f44201fadc0 2024-05-08T23:30:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441f59e700 1 -- 172.21.15.117:0/4166557693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4420161ea0 con 0x7f44201fadc0 2024-05-08T23:30:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f440effd700 1 -- 172.21.15.117:0/4166557693 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441800ae50 con 0x7f44201fadc0 2024-05-08T23:30:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441f59e700 1 -- 172.21.15.117:0/4166557693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44201fb780 con 0x7f44201fadc0 2024-05-08T23:30:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f440effd700 1 -- 172.21.15.117:0/4166557693 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f441801d760 con 0x7f44201fadc0 2024-05-08T23:30:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f440effd700 1 -- 172.21.15.117:0/4166557693 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f4418013b40 con 0x7f44201fadc0 2024-05-08T23:30:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.899+0000 7f441f59e700 1 -- 172.21.15.117:0/4166557693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4420161d00 con 0x7f44201fadc0 2024-05-08T23:30:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.903+0000 7f440effd700 1 -- 172.21.15.117:0/4166557693 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f4420161d00 con 0x7f44201fadc0 2024-05-08T23:30:14.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.979+0000 7f441f59e700 1 -- 172.21.15.117:0/4166557693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "32ab34f3-4f16-44dd-9f62-91dc5e94cf83", "id": 1} v 0) v1 -- 0x7f4420162660 con 0x7f44201fadc0 2024-05-08T23:30:14.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.987+0000 7f440effd700 1 -- 172.21.15.117:0/4166557693 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "32ab34f3-4f16-44dd-9f62-91dc5e94cf83", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f441801db50 con 0x7f44201fadc0 2024-05-08T23:30:14.992 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-05-08T23:30:14.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.991+0000 7f441f59e700 1 -- 172.21.15.117:0/4166557693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44201fadc0 msgr2=0x7f44201fb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:14.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.991+0000 7f441f59e700 1 --2- 172.21.15.117:0/4166557693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44201fadc0 0x7f44201fb240 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4420071ae0 tx=0x7f4418001a20 comp rx=0 tx=0).stop 2024-05-08T23:30:14.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.991+0000 7f441f59e700 1 -- 172.21.15.117:0/4166557693 shutdown_connections 2024-05-08T23:30:14.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.991+0000 7f441f59e700 1 --2- 172.21.15.117:0/4166557693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44201fadc0 0x7f44201fb240 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.991+0000 7f441f59e700 1 --2- 172.21.15.117:0/4166557693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4420150030 0x7f44201504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.991+0000 7f441f59e700 1 --2- 172.21.15.117:0/4166557693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4408004870 0x7f44200c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:14.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.991+0000 7f441f59e700 1 -- 172.21.15.117:0/4166557693 >> 172.21.15.117:0/4166557693 conn(0x7f44200b2970 msgr2=0x7f44200cb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:14.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.991+0000 7f441f59e700 1 -- 172.21.15.117:0/4166557693 shutdown_connections 2024-05-08T23:30:14.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:14.991+0000 7f441f59e700 1 -- 172.21.15.117:0/4166557693 wait complete. 2024-05-08T23:30:15.010 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47 2 2024-05-08T23:30:15.095 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:15.091+0000 7fae133c22c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:30:15.101 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:15.098+0000 7f937ff502c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:30:15.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.111+0000 7f454e2ff700 1 Processor -- start 2024-05-08T23:30:15.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.111+0000 7f454e2ff700 1 -- start start 2024-05-08T23:30:15.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.111+0000 7f454e2ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4548159fb0 0x7f45480b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.111+0000 7f454e2ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45480ae7d0 0x7f4548071f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.111+0000 7f454e2ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45480ae340 con 0x7f4548071150 2024-05-08T23:30:15.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.111+0000 7f454e2ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454815a390 con 0x7f4548159fb0 2024-05-08T23:30:15.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454e2ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45480b3730 con 0x7f45480ae7d0 2024-05-08T23:30:15.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454d2fd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4548159fb0 0x7f45480b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454cafc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45480ae7d0 0x7f4548071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454dafe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f4548071150 0x7f454806d110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:34378/0 (socket says 172.21.15.117:34378) 2024-05-08T23:30:15.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454d2fd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4548159fb0 0x7f45480b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41074/0 (socket says 172.21.15.117:41074) 2024-05-08T23:30:15.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454dafe700 1 -- 172.21.15.117:0/2948768159 learned_addr learned my addr 172.21.15.117:0/2948768159 (peer_addr_for_me v1:172.21.15.117:0/0) 2024-05-08T23:30:15.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454cafc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45480ae7d0 0x7f4548071f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:35638/0 (socket says 172.21.15.117:35638) 2024-05-08T23:30:15.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454cafc700 1 -- 172.21.15.117:0/2948768159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45480ae7d0 msgr2=0x7f4548071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:30:15.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454cafc700 1 -- 172.21.15.117:0/2948768159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45480ae7d0 msgr2=0x7f4548071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:15.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454cafc700 1 --2- 172.21.15.117:0/2948768159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45480ae7d0 0x7f4548071f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:15.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454cafc700 1 --2- 172.21.15.117:0/2948768159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45480ae7d0 0x7f4548071f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:15.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454d2fd700 1 -- 172.21.15.117:0/2948768159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45480ae7d0 msgr2=0x7f4548071f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:15.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454d2fd700 1 --2- 172.21.15.117:0/2948768159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45480ae7d0 0x7f4548071f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454d2fd700 1 -- 172.21.15.117:0/2948768159 >> v1:172.21.15.117:6789/0 conn(0x7f4548071150 legacy=0x7f454806d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f453ffff700 1 -- 172.21.15.117:0/2948768159 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127772696 0 0) 0x7f45480ae340 con 0x7f4548071150 2024-05-08T23:30:15.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454d2fd700 1 -- 172.21.15.117:0/2948768159 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45480ad770 con 0x7f4548159fb0 2024-05-08T23:30:15.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454d2fd700 1 --2- 172.21.15.117:0/2948768159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4548159fb0 0x7f45480b2cd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4544009850 tx=0x7f4544009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f777fd97ef90c5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:15.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f453ffff700 1 -- 172.21.15.117:0/2948768159 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544011830 con 0x7f4548159fb0 2024-05-08T23:30:15.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f453ffff700 1 -- 172.21.15.117:0/2948768159 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4544011990 con 0x7f4548159fb0 2024-05-08T23:30:15.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f453ffff700 1 -- 172.21.15.117:0/2948768159 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544011d20 con 0x7f4548159fb0 2024-05-08T23:30:15.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454e2ff700 1 -- 172.21.15.117:0/2948768159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4548159fb0 msgr2=0x7f45480b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454e2ff700 1 --2- 172.21.15.117:0/2948768159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4548159fb0 0x7f45480b2cd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4544009850 tx=0x7f4544009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:15.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454e2ff700 1 -- 172.21.15.117:0/2948768159 shutdown_connections 2024-05-08T23:30:15.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454e2ff700 1 --2- 172.21.15.117:0/2948768159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45480ae7d0 0x7f4548071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454e2ff700 1 --2- 172.21.15.117:0/2948768159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4548159fb0 0x7f45480b2cd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454e2ff700 1 -- 172.21.15.117:0/2948768159 >> 172.21.15.117:0/2948768159 conn(0x7f45480cfd20 msgr2=0x7f45480d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:15.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454e2ff700 1 -- 172.21.15.117:0/2948768159 shutdown_connections 2024-05-08T23:30:15.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454e2ff700 1 -- 172.21.15.117:0/2948768159 wait complete. 2024-05-08T23:30:15.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454e2ff700 1 Processor -- start 2024-05-08T23:30:15.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.115+0000 7f454e2ff700 1 -- start start 2024-05-08T23:30:15.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454e2ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4548071150 0x7f454806a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454e2ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45480ae7d0 0x7f4548143e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454e2ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4548159fb0 0x7f45481443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454e2ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4548144910 con 0x7f45480ae7d0 2024-05-08T23:30:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454e2ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4548158ed0 con 0x7f4548071150 2024-05-08T23:30:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454e2ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45481591b0 con 0x7f4548159fb0 2024-05-08T23:30:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454d2fd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4548071150 0x7f454806a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454d2fd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4548071150 0x7f454806a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41086/0 (socket says 172.21.15.117:41086) 2024-05-08T23:30:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454d2fd700 1 -- 172.21.15.117:0/2751303359 learned_addr learned my addr 172.21.15.117:0/2751303359 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-05-08T23:30:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454dafe700 1 --2- 172.21.15.117:0/2751303359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4548159fb0 0x7f45481443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454dafe700 1 -- 172.21.15.117:0/2751303359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4548159fb0 msgr2=0x7f45481443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-08T23:30:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454dafe700 1 -- 172.21.15.117:0/2751303359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4548159fb0 msgr2=0x7f45481443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:30:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454dafe700 1 --2- 172.21.15.117:0/2751303359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4548159fb0 0x7f45481443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454cafc700 1 --2- 172.21.15.117:0/2751303359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45480ae7d0 0x7f4548143e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454dafe700 1 --2- 172.21.15.117:0/2751303359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4548159fb0 0x7f45481443d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454d2fd700 1 -- 172.21.15.117:0/2751303359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4548159fb0 msgr2=0x7f45481443d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:30:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454d2fd700 1 --2- 172.21.15.117:0/2751303359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4548159fb0 0x7f45481443d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454d2fd700 1 -- 172.21.15.117:0/2751303359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45480ae7d0 msgr2=0x7f4548143e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454d2fd700 1 --2- 172.21.15.117:0/2751303359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45480ae7d0 0x7f4548143e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454d2fd700 1 -- 172.21.15.117:0/2751303359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45440094b0 con 0x7f4548071150 2024-05-08T23:30:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454d2fd700 1 --2- 172.21.15.117:0/2751303359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4548071150 0x7f454806a5d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4544009fd0 tx=0x7f4544004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f453dffb700 1 -- 172.21.15.117:0/2751303359 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440036c0 con 0x7f4548071150 2024-05-08T23:30:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f453dffb700 1 -- 172.21.15.117:0/2751303359 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454401e070 con 0x7f4548071150 2024-05-08T23:30:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454e2ff700 1 -- 172.21.15.117:0/2751303359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4548144c40 con 0x7f4548071150 2024-05-08T23:30:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f453dffb700 1 -- 172.21.15.117:0/2751303359 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544003a40 con 0x7f4548071150 2024-05-08T23:30:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454e2ff700 1 -- 172.21.15.117:0/2751303359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4548145200 con 0x7f4548071150 2024-05-08T23:30:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f453dffb700 1 -- 172.21.15.117:0/2751303359 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f454402d430 con 0x7f4548071150 2024-05-08T23:30:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f453dffb700 1 -- 172.21.15.117:0/2751303359 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f454402db00 con 0x7f4548071150 2024-05-08T23:30:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.119+0000 7f454e2ff700 1 -- 172.21.15.117:0/2751303359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45481453f0 con 0x7f4548071150 2024-05-08T23:30:15.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.123+0000 7f453dffb700 1 -- 172.21.15.117:0/2751303359 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f4544020070 con 0x7f4548071150 2024-05-08T23:30:15.150 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:15.147+0000 7fae133c22c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:30:15.156 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:15.154+0000 7f937ff502c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:30:15.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.199+0000 7f454e2ff700 1 -- 172.21.15.117:0/2751303359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47", "id": 2} v 0) v1 -- 0x7f45480672a0 con 0x7f4548071150 2024-05-08T23:30:15.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.207+0000 7f453dffb700 1 -- 172.21.15.117:0/2751303359 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4544015020 con 0x7f4548071150 2024-05-08T23:30:15.211 INFO:teuthology.orchestra.run.smithi117.stdout:2 2024-05-08T23:30:15.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.207+0000 7f454e2ff700 1 -- 172.21.15.117:0/2751303359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4548071150 msgr2=0x7f454806a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.207+0000 7f454e2ff700 1 --2- 172.21.15.117:0/2751303359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4548071150 0x7f454806a5d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4544009fd0 tx=0x7f4544004770 comp rx=0 tx=0).stop 2024-05-08T23:30:15.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.207+0000 7f454e2ff700 1 -- 172.21.15.117:0/2751303359 shutdown_connections 2024-05-08T23:30:15.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.207+0000 7f454e2ff700 1 --2- 172.21.15.117:0/2751303359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4548159fb0 0x7f45481443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.207+0000 7f454e2ff700 1 --2- 172.21.15.117:0/2751303359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45480ae7d0 0x7f4548143e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.207+0000 7f454e2ff700 1 --2- 172.21.15.117:0/2751303359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4548071150 0x7f454806a5d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.207+0000 7f454e2ff700 1 -- 172.21.15.117:0/2751303359 >> 172.21.15.117:0/2751303359 conn(0x7f45480cfd20 msgr2=0x7f45480afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:15.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.207+0000 7f454e2ff700 1 -- 172.21.15.117:0/2751303359 shutdown_connections 2024-05-08T23:30:15.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.207+0000 7f454e2ff700 1 -- 172.21.15.117:0/2751303359 wait complete. 2024-05-08T23:30:15.224 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new caedb186-c021-440e-aff2-889bdaf1f137 3 2024-05-08T23:30:15.313 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:15.310+0000 7f937ff502c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:30:15.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.327+0000 7f0c5e6cc700 1 Processor -- start 2024-05-08T23:30:15.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.327+0000 7f0c5e6cc700 1 -- start start 2024-05-08T23:30:15.332 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:15.327+0000 7fae133c22c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:30:15.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.327+0000 7f0c5e6cc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 0x7f0c5806a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.327+0000 7f0c5e6cc700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0c5815bc90 0x7f0c580ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.327+0000 7f0c5e6cc700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5806ac80 con 0x7f0c5815f7b0 2024-05-08T23:30:15.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.327+0000 7f0c5e6cc700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c5815c120 con 0x7f0c5815bc90 2024-05-08T23:30:15.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.327+0000 7f0c5e6cc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5815ecf0 con 0x7f0c5806a360 2024-05-08T23:30:15.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.327+0000 7f0c5d6ca700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 0x7f0c5806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.327+0000 7f0c5cec9700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f0c5815f7b0 0x7f0c5815b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:34394/0 (socket says 172.21.15.117:34394) 2024-05-08T23:30:15.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.327+0000 7f0c5cec9700 1 -- 172.21.15.117:0/3100101709 learned_addr learned my addr 172.21.15.117:0/3100101709 (peer_addr_for_me v1:172.21.15.117:0/0) 2024-05-08T23:30:15.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.327+0000 7f0c5d6ca700 1 -- 172.21.15.117:0/3100101709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 msgr2=0x7f0c5806a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:30:15.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.327+0000 7f0c5d6ca700 1 -- 172.21.15.117:0/3100101709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 msgr2=0x7f0c5806a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:15.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.327+0000 7f0c5d6ca700 1 --2- 172.21.15.117:0/3100101709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 0x7f0c5806a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:15.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5decb700 1 --2- 172.21.15.117:0/3100101709 >> v2:172.21.15.81:3300/0 conn(0x7f0c5815bc90 0x7f0c580ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5d6ca700 1 --2- 172.21.15.117:0/3100101709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 0x7f0c5806a740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:15.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c4bfff700 1 -- 172.21.15.117:0/3100101709 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339040285 0 0) 0x7f0c5806ac80 con 0x7f0c5815f7b0 2024-05-08T23:30:15.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c4bfff700 1 -- 172.21.15.117:0/3100101709 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c440035d0 con 0x7f0c5815f7b0 2024-05-08T23:30:15.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5decb700 1 -- 172.21.15.117:0/3100101709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 msgr2=0x7f0c5806a740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:15.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5decb700 1 --2- 172.21.15.117:0/3100101709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 0x7f0c5806a740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5decb700 1 -- 172.21.15.117:0/3100101709 >> v1:172.21.15.117:6789/0 conn(0x7f0c5815f7b0 legacy=0x7f0c5815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5decb700 1 -- 172.21.15.117:0/3100101709 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5815c320 con 0x7f0c5815bc90 2024-05-08T23:30:15.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5decb700 1 --2- 172.21.15.117:0/3100101709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c5815bc90 0x7f0c580ce3b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009d90 tx=0x7f0c5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cf9ac80c3860f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:15.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c4bfff700 1 -- 172.21.15.117:0/3100101709 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5000e070 con 0x7f0c5815bc90 2024-05-08T23:30:15.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c4bfff700 1 -- 172.21.15.117:0/3100101709 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5000bb20 con 0x7f0c5815bc90 2024-05-08T23:30:15.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c4bfff700 1 -- 172.21.15.117:0/3100101709 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50012430 con 0x7f0c5815bc90 2024-05-08T23:30:15.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 -- 172.21.15.117:0/3100101709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c5815bc90 msgr2=0x7f0c580ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 --2- 172.21.15.117:0/3100101709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c5815bc90 0x7f0c580ce3b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009d90 tx=0x7f0c5000b330 comp rx=0 tx=0).stop 2024-05-08T23:30:15.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 -- 172.21.15.117:0/3100101709 shutdown_connections 2024-05-08T23:30:15.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 --2- 172.21.15.117:0/3100101709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c5815bc90 0x7f0c580ce3b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 --2- 172.21.15.117:0/3100101709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 0x7f0c5806a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 -- 172.21.15.117:0/3100101709 >> 172.21.15.117:0/3100101709 conn(0x7f0c580b18c0 msgr2=0x7f0c580b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:15.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 -- 172.21.15.117:0/3100101709 shutdown_connections 2024-05-08T23:30:15.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 -- 172.21.15.117:0/3100101709 wait complete. 2024-05-08T23:30:15.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 Processor -- start 2024-05-08T23:30:15.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 -- start start 2024-05-08T23:30:15.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 0x7f0c5812a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c5815bc90 0x7f0c580c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5815f7b0 0x7f0c5806d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580cb4f0 con 0x7f0c5815bc90 2024-05-08T23:30:15.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580cb1e0 con 0x7f0c5815f7b0 2024-05-08T23:30:15.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5e6cc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580c54a0 con 0x7f0c5806a360 2024-05-08T23:30:15.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.331+0000 7f0c5d6ca700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 0x7f0c5812a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5decb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5815f7b0 0x7f0c5806d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5d6ca700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 0x7f0c5812a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:35658/0 (socket says 172.21.15.117:35658) 2024-05-08T23:30:15.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5d6ca700 1 -- 172.21.15.117:0/2480574466 learned_addr learned my addr 172.21.15.117:0/2480574466 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-05-08T23:30:15.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5d6ca700 1 -- 172.21.15.117:0/2480574466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 msgr2=0x7f0c5812a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-08T23:30:15.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5d6ca700 1 -- 172.21.15.117:0/2480574466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 msgr2=0x7f0c5812a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:30:15.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5d6ca700 1 --2- 172.21.15.117:0/2480574466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 0x7f0c5812a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:15.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5d6ca700 1 --2- 172.21.15.117:0/2480574466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 0x7f0c5812a5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:15.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5cec9700 1 --2- 172.21.15.117:0/2480574466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c5815bc90 0x7f0c580c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5decb700 1 -- 172.21.15.117:0/2480574466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 msgr2=0x7f0c5812a5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:30:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5decb700 1 --2- 172.21.15.117:0/2480574466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 0x7f0c5812a5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5decb700 1 -- 172.21.15.117:0/2480574466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c5815bc90 msgr2=0x7f0c580c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5decb700 1 --2- 172.21.15.117:0/2480574466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c5815bc90 0x7f0c580c4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5decb700 1 -- 172.21.15.117:0/2480574466 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c50009a70 con 0x7f0c5815f7b0 2024-05-08T23:30:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5decb700 1 --2- 172.21.15.117:0/2480574466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5815f7b0 0x7f0c5806d2e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0c50016000 tx=0x7f0c50004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c49ffb700 1 -- 172.21.15.117:0/2480574466 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50012430 con 0x7f0c5815f7b0 2024-05-08T23:30:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5e6cc700 1 -- 172.21.15.117:0/2480574466 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c580c5780 con 0x7f0c5815f7b0 2024-05-08T23:30:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c49ffb700 1 -- 172.21.15.117:0/2480574466 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c50020070 con 0x7f0c5815f7b0 2024-05-08T23:30:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c49ffb700 1 -- 172.21.15.117:0/2480574466 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50004280 con 0x7f0c5815f7b0 2024-05-08T23:30:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5e6cc700 1 -- 172.21.15.117:0/2480574466 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c58128c60 con 0x7f0c5815f7b0 2024-05-08T23:30:15.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c49ffb700 1 -- 172.21.15.117:0/2480574466 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0c500046e0 con 0x7f0c5815f7b0 2024-05-08T23:30:15.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c49ffb700 1 -- 172.21.15.117:0/2480574466 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f0c50022070 con 0x7f0c5815f7b0 2024-05-08T23:30:15.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.335+0000 7f0c5e6cc700 1 -- 172.21.15.117:0/2480574466 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c58128df0 con 0x7f0c5815f7b0 2024-05-08T23:30:15.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.339+0000 7f0c49ffb700 1 -- 172.21.15.117:0/2480574466 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f0c50046dc0 con 0x7f0c5815f7b0 2024-05-08T23:30:15.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.415+0000 7f0c5e6cc700 1 -- 172.21.15.117:0/2480574466 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "caedb186-c021-440e-aff2-889bdaf1f137", "id": 3} v 0) v1 -- 0x7f0c580c5ca0 con 0x7f0c5815f7b0 2024-05-08T23:30:15.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.459+0000 7f0c49ffb700 1 -- 172.21.15.117:0/2480574466 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "caedb186-c021-440e-aff2-889bdaf1f137", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0c50035030 con 0x7f0c5815f7b0 2024-05-08T23:30:15.464 INFO:teuthology.orchestra.run.smithi117.stdout:3 2024-05-08T23:30:15.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.459+0000 7f0c5e6cc700 1 -- 172.21.15.117:0/2480574466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5815f7b0 msgr2=0x7f0c5806d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.459+0000 7f0c5e6cc700 1 --2- 172.21.15.117:0/2480574466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5815f7b0 0x7f0c5806d2e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0c50016000 tx=0x7f0c50004af0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.459+0000 7f0c5e6cc700 1 -- 172.21.15.117:0/2480574466 shutdown_connections 2024-05-08T23:30:15.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.459+0000 7f0c5e6cc700 1 --2- 172.21.15.117:0/2480574466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c5815f7b0 0x7f0c5806d2e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.459+0000 7f0c5e6cc700 1 --2- 172.21.15.117:0/2480574466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c5815bc90 0x7f0c580c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.459+0000 7f0c5e6cc700 1 --2- 172.21.15.117:0/2480574466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5806a360 0x7f0c5812a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.459+0000 7f0c5e6cc700 1 -- 172.21.15.117:0/2480574466 >> 172.21.15.117:0/2480574466 conn(0x7f0c580b18c0 msgr2=0x7f0c5806e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:15.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.459+0000 7f0c5e6cc700 1 -- 172.21.15.117:0/2480574466 shutdown_connections 2024-05-08T23:30:15.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.459+0000 7f0c5e6cc700 1 -- 172.21.15.117:0/2480574466 wait complete. 2024-05-08T23:30:15.480 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 9aaee44f-bb9e-4014-af3a-a8b421885b7b 4 2024-05-08T23:30:15.525 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:15.522+0000 7f937ff502c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:30:15.565 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:15.563+0000 7fae133c22c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:30:15.574 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:15.570+0000 7f937ff502c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:30:15.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facbdf1a700 1 Processor -- start 2024-05-08T23:30:15.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facbdf1a700 1 -- start start 2024-05-08T23:30:15.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facbdf1a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7facb806d220 0x7facb806e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facbdf1a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7facb8161bc0 0x7facb8163fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facbdf1a700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb80aed00 con 0x7facb806e880 2024-05-08T23:30:15.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facbdf1a700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb80adc60 con 0x7facb806d220 2024-05-08T23:30:15.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facbdf1a700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb80adf40 con 0x7facb8161bc0 2024-05-08T23:30:15.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facb7fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7facb8161bc0 0x7facb8163fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facb77fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7facb806e880 0x7facb806ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.117:51110/0 (socket says 172.21.15.117:51110) 2024-05-08T23:30:15.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facb7fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7facb8161bc0 0x7facb8163fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41114/0 (socket says 172.21.15.117:41114) 2024-05-08T23:30:15.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facb77fe700 1 -- 172.21.15.117:0/216860915 learned_addr learned my addr 172.21.15.117:0/216860915 (peer_addr_for_me v1:172.21.15.117:0/0) 2024-05-08T23:30:15.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facb6ffd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7facb806d220 0x7facb806e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facb7fff700 1 -- 172.21.15.117:0/216860915 >> v2:172.21.15.81:3300/0 conn(0x7facb806d220 msgr2=0x7facb806e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:15.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facb7fff700 1 --2- 172.21.15.117:0/216860915 >> v2:172.21.15.81:3300/0 conn(0x7facb806d220 0x7facb806e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facb7fff700 1 -- 172.21.15.117:0/216860915 >> v1:172.21.15.81:6789/0 conn(0x7facb806e880 legacy=0x7facb806ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facb7fff700 1 -- 172.21.15.117:0/216860915 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb80ae220 con 0x7facb8161bc0 2024-05-08T23:30:15.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facb7fff700 1 --2- 172.21.15.117:0/216860915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facb8161bc0 0x7facb8163fb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7facb80d0fa0 tx=0x7faca8009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74374f37e5884da4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:15.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facb67fc700 1 -- 172.21.15.117:0/216860915 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca8011830 con 0x7facb8161bc0 2024-05-08T23:30:15.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facb67fc700 1 -- 172.21.15.117:0/216860915 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca8011990 con 0x7facb8161bc0 2024-05-08T23:30:15.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facb67fc700 1 -- 172.21.15.117:0/216860915 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca8011d20 con 0x7facb8161bc0 2024-05-08T23:30:15.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facbdf1a700 1 -- 172.21.15.117:0/216860915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facb8161bc0 msgr2=0x7facb8163fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facbdf1a700 1 --2- 172.21.15.117:0/216860915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facb8161bc0 0x7facb8163fb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7facb80d0fa0 tx=0x7faca8009b30 comp rx=0 tx=0).stop 2024-05-08T23:30:15.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facbdf1a700 1 -- 172.21.15.117:0/216860915 shutdown_connections 2024-05-08T23:30:15.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facbdf1a700 1 --2- 172.21.15.117:0/216860915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facb8161bc0 0x7facb8163fb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facbdf1a700 1 --2- 172.21.15.117:0/216860915 >> v2:172.21.15.81:3300/0 conn(0x7facb806d220 0x7facb806e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facbdf1a700 1 -- 172.21.15.117:0/216860915 >> 172.21.15.117:0/216860915 conn(0x7facb80cfec0 msgr2=0x7facb80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:15.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facbdf1a700 1 -- 172.21.15.117:0/216860915 shutdown_connections 2024-05-08T23:30:15.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.583+0000 7facbdf1a700 1 -- 172.21.15.117:0/216860915 wait complete. 2024-05-08T23:30:15.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facbdf1a700 1 Processor -- start 2024-05-08T23:30:15.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facbdf1a700 1 -- start start 2024-05-08T23:30:15.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facbdf1a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facb806d220 0x7facb814b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facbdf1a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facb806e880 0x7facb815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facbdf1a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facb8161bc0 0x7facb815d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facbdf1a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb815d5d0 con 0x7facb806d220 2024-05-08T23:30:15.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facbdf1a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb814a6d0 con 0x7facb806e880 2024-05-08T23:30:15.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facbdf1a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb814a9b0 con 0x7facb8161bc0 2024-05-08T23:30:15.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facb7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facb8161bc0 0x7facb815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facb7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facb8161bc0 0x7facb815d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:35674/0 (socket says 172.21.15.117:35674) 2024-05-08T23:30:15.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facb6ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facb806e880 0x7facb815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facb7fff700 1 -- 172.21.15.117:0/4282150028 learned_addr learned my addr 172.21.15.117:0/4282150028 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-05-08T23:30:15.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facb77fe700 1 --2- 172.21.15.117:0/4282150028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facb806d220 0x7facb814b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facb7fff700 1 -- 172.21.15.117:0/4282150028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facb8161bc0 msgr2=0x7facb815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-08T23:30:15.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facb7fff700 1 -- 172.21.15.117:0/4282150028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facb8161bc0 msgr2=0x7facb815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:30:15.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facb7fff700 1 --2- 172.21.15.117:0/4282150028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facb8161bc0 0x7facb815d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:15.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facb7fff700 1 --2- 172.21.15.117:0/4282150028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facb8161bc0 0x7facb815d090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facb6ffd700 1 -- 172.21.15.117:0/4282150028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facb8161bc0 msgr2=0x7facb815d090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:30:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facb6ffd700 1 --2- 172.21.15.117:0/4282150028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facb8161bc0 0x7facb815d090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facb6ffd700 1 -- 172.21.15.117:0/4282150028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facb806d220 msgr2=0x7facb814b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facb6ffd700 1 --2- 172.21.15.117:0/4282150028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facb806d220 0x7facb814b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facb6ffd700 1 -- 172.21.15.117:0/4282150028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca80094b0 con 0x7facb806e880 2024-05-08T23:30:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facb6ffd700 1 --2- 172.21.15.117:0/4282150028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facb806e880 0x7facb815cb50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faca000e870 tx=0x7faca000eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facaffff700 1 -- 172.21.15.117:0/4282150028 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca0013680 con 0x7facb806e880 2024-05-08T23:30:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facaffff700 1 -- 172.21.15.117:0/4282150028 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca00137e0 con 0x7facb806e880 2024-05-08T23:30:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facbdf1a700 1 -- 172.21.15.117:0/4282150028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb814ac90 con 0x7facb806e880 2024-05-08T23:30:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facaffff700 1 -- 172.21.15.117:0/4282150028 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca0013b70 con 0x7facb806e880 2024-05-08T23:30:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.587+0000 7facbdf1a700 1 -- 172.21.15.117:0/4282150028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb815a580 con 0x7facb806e880 2024-05-08T23:30:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.591+0000 7facaffff700 1 -- 172.21.15.117:0/4282150028 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7faca0005a40 con 0x7facb806e880 2024-05-08T23:30:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.591+0000 7facaffff700 1 -- 172.21.15.117:0/4282150028 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7faca0006750 con 0x7facb806e880 2024-05-08T23:30:15.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.591+0000 7facbdf1a700 1 -- 172.21.15.117:0/4282150028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb8073b50 con 0x7facb806e880 2024-05-08T23:30:15.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.591+0000 7facaffff700 1 -- 172.21.15.117:0/4282150028 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7faca0021060 con 0x7facb806e880 2024-05-08T23:30:15.620 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:15.615+0000 7fae133c22c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:30:15.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.671+0000 7facbdf1a700 1 -- 172.21.15.117:0/4282150028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9aaee44f-bb9e-4014-af3a-a8b421885b7b", "id": 4} v 0) v1 -- 0x7facb80672a0 con 0x7facb806e880 2024-05-08T23:30:15.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.679+0000 7facaffff700 1 -- 172.21.15.117:0/4282150028 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9aaee44f-bb9e-4014-af3a-a8b421885b7b", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7faca0017070 con 0x7facb806e880 2024-05-08T23:30:15.684 INFO:teuthology.orchestra.run.smithi117.stdout:4 2024-05-08T23:30:15.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.679+0000 7facbdf1a700 1 -- 172.21.15.117:0/4282150028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facb806e880 msgr2=0x7facb815cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.679+0000 7facbdf1a700 1 --2- 172.21.15.117:0/4282150028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facb806e880 0x7facb815cb50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faca000e870 tx=0x7faca000eb80 comp rx=0 tx=0).stop 2024-05-08T23:30:15.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.679+0000 7facbdf1a700 1 -- 172.21.15.117:0/4282150028 shutdown_connections 2024-05-08T23:30:15.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.679+0000 7facbdf1a700 1 --2- 172.21.15.117:0/4282150028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facb8161bc0 0x7facb815d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.679+0000 7facbdf1a700 1 --2- 172.21.15.117:0/4282150028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facb806e880 0x7facb815cb50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.679+0000 7facbdf1a700 1 --2- 172.21.15.117:0/4282150028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facb806d220 0x7facb814b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.679+0000 7facbdf1a700 1 -- 172.21.15.117:0/4282150028 >> 172.21.15.117:0/4282150028 conn(0x7facb80cfec0 msgr2=0x7facb80b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:15.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.679+0000 7facbdf1a700 1 -- 172.21.15.117:0/4282150028 shutdown_connections 2024-05-08T23:30:15.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.683+0000 7facbdf1a700 1 -- 172.21.15.117:0/4282150028 wait complete. 2024-05-08T23:30:15.700 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 98cce22f-822c-43fc-a4de-c9f990847a0b 5 2024-05-08T23:30:15.752 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:15.750+0000 7f937ff502c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:30:15.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0f1ed700 1 Processor -- start 2024-05-08T23:30:15.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0f1ed700 1 -- start start 2024-05-08T23:30:15.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0f1ed700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6e0815d670 0x7f6e080d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0f1ed700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e080ce8d0 0x7f6e080ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0f1ed700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e080ce440 con 0x7f6e080b40f0 2024-05-08T23:30:15.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0f1ed700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e080cf2a0 con 0x7f6e0815d670 2024-05-08T23:30:15.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0f1ed700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e080d3b50 con 0x7f6e080ce8d0 2024-05-08T23:30:15.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0e1eb700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6e0815d670 0x7f6e080d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0e9ec700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e080ce8d0 0x7f6e080ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0e1eb700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6e0815d670 0x7f6e080d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41136/0 (socket says 172.21.15.117:41136) 2024-05-08T23:30:15.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0e1eb700 1 -- 172.21.15.117:0/3639082739 learned_addr learned my addr 172.21.15.117:0/3639082739 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-05-08T23:30:15.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0e9ec700 1 -- 172.21.15.117:0/3639082739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e080ce8d0 msgr2=0x7f6e080ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:30:15.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0e9ec700 1 -- 172.21.15.117:0/3639082739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e080ce8d0 msgr2=0x7f6e080ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:15.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0e9ec700 1 --2- 172.21.15.117:0/3639082739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e080ce8d0 0x7f6e080ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:15.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0e9ec700 1 --2- 172.21.15.117:0/3639082739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e080ce8d0 0x7f6e080ced60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:15.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0e1eb700 1 -- 172.21.15.117:0/3639082739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e080ce8d0 msgr2=0x7f6e080ced60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:15.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0e1eb700 1 --2- 172.21.15.117:0/3639082739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e080ce8d0 0x7f6e080ced60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0e1eb700 1 -- 172.21.15.117:0/3639082739 >> v1:172.21.15.81:6789/0 conn(0x7f6e080b40f0 legacy=0x7f6e080b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.803+0000 7f6e0e1eb700 1 -- 172.21.15.117:0/3639082739 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e080d3e30 con 0x7f6e0815d670 2024-05-08T23:30:15.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0e1eb700 1 --2- 172.21.15.117:0/3639082739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e0815d670 0x7f6e080d0e10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009e50 tx=0x7f6e0000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2074ad5116953f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:15.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0d1e9700 1 -- 172.21.15.117:0/3639082739 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0000e070 con 0x7f6e0815d670 2024-05-08T23:30:15.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0d1e9700 1 -- 172.21.15.117:0/3639082739 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0000b9c0 con 0x7f6e0815d670 2024-05-08T23:30:15.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0d1e9700 1 -- 172.21.15.117:0/3639082739 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0000bd70 con 0x7f6e0815d670 2024-05-08T23:30:15.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 -- 172.21.15.117:0/3639082739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e0815d670 msgr2=0x7f6e080d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 --2- 172.21.15.117:0/3639082739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e0815d670 0x7f6e080d0e10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009e50 tx=0x7f6e0000b330 comp rx=0 tx=0).stop 2024-05-08T23:30:15.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 -- 172.21.15.117:0/3639082739 shutdown_connections 2024-05-08T23:30:15.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 --2- 172.21.15.117:0/3639082739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e080ce8d0 0x7f6e080ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 --2- 172.21.15.117:0/3639082739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e0815d670 0x7f6e080d0e10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 -- 172.21.15.117:0/3639082739 >> 172.21.15.117:0/3639082739 conn(0x7f6e080d3030 msgr2=0x7f6e080d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:15.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 -- 172.21.15.117:0/3639082739 shutdown_connections 2024-05-08T23:30:15.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 -- 172.21.15.117:0/3639082739 wait complete. 2024-05-08T23:30:15.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 Processor -- start 2024-05-08T23:30:15.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 -- start start 2024-05-08T23:30:15.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e080b40f0 0x7f6e081f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e080ce8d0 0x7f6e081f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e0815d670 0x7f6e081f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:15.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0815e190 con 0x7f6e080b40f0 2024-05-08T23:30:15.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0815e470 con 0x7f6e080ce8d0 2024-05-08T23:30:15.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0f1ed700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0815e750 con 0x7f6e0815d670 2024-05-08T23:30:15.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0e9ec700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e0815d670 0x7f6e081f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0d9ea700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e080ce8d0 0x7f6e081f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0e9ec700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e0815d670 0x7f6e081f3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:35694/0 (socket says 172.21.15.117:35694) 2024-05-08T23:30:15.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0e9ec700 1 -- 172.21.15.117:0/4263331363 learned_addr learned my addr 172.21.15.117:0/4263331363 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-05-08T23:30:15.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0e1eb700 1 --2- 172.21.15.117:0/4263331363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e080b40f0 0x7f6e081f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:15.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0e9ec700 1 -- 172.21.15.117:0/4263331363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e0815d670 msgr2=0x7f6e081f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-08T23:30:15.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0e9ec700 1 -- 172.21.15.117:0/4263331363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e0815d670 msgr2=0x7f6e081f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:30:15.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0e9ec700 1 --2- 172.21.15.117:0/4263331363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e0815d670 0x7f6e081f3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:15.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0e9ec700 1 --2- 172.21.15.117:0/4263331363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e0815d670 0x7f6e081f3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:15.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0d9ea700 1 -- 172.21.15.117:0/4263331363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e0815d670 msgr2=0x7f6e081f3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:30:15.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0d9ea700 1 --2- 172.21.15.117:0/4263331363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e0815d670 0x7f6e081f3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0d9ea700 1 -- 172.21.15.117:0/4263331363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e080b40f0 msgr2=0x7f6e081f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0d9ea700 1 --2- 172.21.15.117:0/4263331363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e080b40f0 0x7f6e081f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.807+0000 7f6e0d9ea700 1 -- 172.21.15.117:0/4263331363 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e00009b30 con 0x7f6e080ce8d0 2024-05-08T23:30:15.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.811+0000 7f6e0d9ea700 1 --2- 172.21.15.117:0/4263331363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e080ce8d0 0x7f6e081f41e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00c410 tx=0x7f6dfc001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:15.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.811+0000 7f6e06ffd700 1 -- 172.21.15.117:0/4263331363 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc0052d0 con 0x7f6e080ce8d0 2024-05-08T23:30:15.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.811+0000 7f6e06ffd700 1 -- 172.21.15.117:0/4263331363 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc005430 con 0x7f6e080ce8d0 2024-05-08T23:30:15.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.811+0000 7f6e06ffd700 1 -- 172.21.15.117:0/4263331363 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc0055a0 con 0x7f6e080ce8d0 2024-05-08T23:30:15.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.811+0000 7f6e0f1ed700 1 -- 172.21.15.117:0/4263331363 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0815ea30 con 0x7f6e080ce8d0 2024-05-08T23:30:15.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.811+0000 7f6e0f1ed700 1 -- 172.21.15.117:0/4263331363 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e081f2e50 con 0x7f6e080ce8d0 2024-05-08T23:30:15.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.811+0000 7f6e06ffd700 1 -- 172.21.15.117:0/4263331363 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f6dfc003be0 con 0x7f6e080ce8d0 2024-05-08T23:30:15.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.811+0000 7f6e06ffd700 1 -- 172.21.15.117:0/4263331363 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f6dfc011cc0 con 0x7f6e080ce8d0 2024-05-08T23:30:15.819 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:15.811+0000 7fae133c22c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:30:15.820 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:15.814+0000 7f937ff502c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:30:15.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.811+0000 7f6e0f1ed700 1 -- 172.21.15.117:0/4263331363 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e081f3370 con 0x7f6e080ce8d0 2024-05-08T23:30:15.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.815+0000 7f6e06ffd700 1 -- 172.21.15.117:0/4263331363 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f6e081f3370 con 0x7f6e080ce8d0 2024-05-08T23:30:15.880 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:15.874+0000 7f937ff502c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:30:15.881 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:15.879+0000 7fae133c22c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:30:15.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.891+0000 7f6e0f1ed700 1 -- 172.21.15.117:0/4263331363 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "98cce22f-822c-43fc-a4de-c9f990847a0b", "id": 5} v 0) v1 -- 0x7f6e080672a0 con 0x7f6e080ce8d0 2024-05-08T23:30:15.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.899+0000 7f6e06ffd700 1 -- 172.21.15.117:0/4263331363 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "98cce22f-822c-43fc-a4de-c9f990847a0b", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6dfc017070 con 0x7f6e080ce8d0 2024-05-08T23:30:15.903 INFO:teuthology.orchestra.run.smithi117.stdout:5 2024-05-08T23:30:15.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.899+0000 7f6e0f1ed700 1 -- 172.21.15.117:0/4263331363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e080ce8d0 msgr2=0x7f6e081f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:15.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.899+0000 7f6e0f1ed700 1 --2- 172.21.15.117:0/4263331363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e080ce8d0 0x7f6e081f41e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00c410 tx=0x7f6dfc001e70 comp rx=0 tx=0).stop 2024-05-08T23:30:15.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.899+0000 7f6e0f1ed700 1 -- 172.21.15.117:0/4263331363 shutdown_connections 2024-05-08T23:30:15.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.899+0000 7f6e0f1ed700 1 --2- 172.21.15.117:0/4263331363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e0815d670 0x7f6e081f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.899+0000 7f6e0f1ed700 1 --2- 172.21.15.117:0/4263331363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e080ce8d0 0x7f6e081f41e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.899+0000 7f6e0f1ed700 1 --2- 172.21.15.117:0/4263331363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e080b40f0 0x7f6e081f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:15.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.899+0000 7f6e0f1ed700 1 -- 172.21.15.117:0/4263331363 >> 172.21.15.117:0/4263331363 conn(0x7f6e080d3030 msgr2=0x7f6e080d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:15.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.899+0000 7f6e0f1ed700 1 -- 172.21.15.117:0/4263331363 shutdown_connections 2024-05-08T23:30:15.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:15.899+0000 7f6e0f1ed700 1 -- 172.21.15.117:0/4263331363 wait complete. 2024-05-08T23:30:15.919 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 0b353abc-6eed-4974-8d15-f9d1e8e9c352 6 2024-05-08T23:30:15.949 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:15.943+0000 7fae133c22c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:30:16.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fffff700 1 Processor -- start 2024-05-08T23:30:16.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fffff700 1 -- start start 2024-05-08T23:30:16.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fffff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa30006aa70 0x7fa3000ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:16.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3000aee90 0x7fa3000b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:16.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fffff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3000ad420 con 0x7fa30015d160 2024-05-08T23:30:16.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fffff700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3000af390 con 0x7fa30006aa70 2024-05-08T23:30:16.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30006d560 con 0x7fa3000aee90 2024-05-08T23:30:16.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2ff7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3000aee90 0x7fa3000b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:16.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2ff7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3000aee90 0x7fa3000b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:35702/0 (socket says 172.21.15.117:35702) 2024-05-08T23:30:16.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2ff7fe700 1 -- 172.21.15.117:0/3445350432 learned_addr learned my addr 172.21.15.117:0/3445350432 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-05-08T23:30:16.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2ff7fe700 1 -- 172.21.15.117:0/3445350432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3000aee90 msgr2=0x7fa3000b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:30:16.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2ff7fe700 1 -- 172.21.15.117:0/3445350432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3000aee90 msgr2=0x7fa3000b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:16.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2ff7fe700 1 --2- 172.21.15.117:0/3445350432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3000aee90 0x7fa3000b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:16.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2ff7fe700 1 --2- 172.21.15.117:0/3445350432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3000aee90 0x7fa3000b2dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:16.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fe7fc700 1 --2- 172.21.15.117:0/3445350432 >> v2:172.21.15.81:3300/0 conn(0x7fa30006aa70 0x7fa3000ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:16.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fe7fc700 1 -- 172.21.15.117:0/3445350432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3000aee90 msgr2=0x7fa3000b2dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:16.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fe7fc700 1 --2- 172.21.15.117:0/3445350432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3000aee90 0x7fa3000b2dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fe7fc700 1 -- 172.21.15.117:0/3445350432 >> v1:172.21.15.81:6789/0 conn(0x7fa30015d160 legacy=0x7fa30006a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:16.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fe7fc700 1 -- 172.21.15.117:0/3445350432 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3000ad650 con 0x7fa30006aa70 2024-05-08T23:30:16.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fe7fc700 1 --2- 172.21.15.117:0/3445350432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa30006aa70 0x7fa3000ae410 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009850 tx=0x7fa2e8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d5530e312b762c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:16.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fdffb700 1 -- 172.21.15.117:0/3445350432 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8011830 con 0x7fa30006aa70 2024-05-08T23:30:16.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fdffb700 1 -- 172.21.15.117:0/3445350432 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e8011990 con 0x7fa30006aa70 2024-05-08T23:30:16.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.015+0000 7fa2fdffb700 1 -- 172.21.15.117:0/3445350432 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8011d20 con 0x7fa30006aa70 2024-05-08T23:30:16.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 -- 172.21.15.117:0/3445350432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa30006aa70 msgr2=0x7fa3000ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:16.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 --2- 172.21.15.117:0/3445350432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa30006aa70 0x7fa3000ae410 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009850 tx=0x7fa2e8009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:16.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 -- 172.21.15.117:0/3445350432 shutdown_connections 2024-05-08T23:30:16.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 --2- 172.21.15.117:0/3445350432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3000aee90 0x7fa3000b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 --2- 172.21.15.117:0/3445350432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa30006aa70 0x7fa3000ae410 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 -- 172.21.15.117:0/3445350432 >> 172.21.15.117:0/3445350432 conn(0x7fa30006b470 msgr2=0x7fa30006b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:16.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 -- 172.21.15.117:0/3445350432 shutdown_connections 2024-05-08T23:30:16.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 -- 172.21.15.117:0/3445350432 wait complete. 2024-05-08T23:30:16.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 Processor -- start 2024-05-08T23:30:16.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 -- start start 2024-05-08T23:30:16.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa30006aa70 0x7fa300156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:16.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3000aee90 0x7fa3001e9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:16.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa30015d160 0x7fa300156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:16.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3001e9580 con 0x7fa3000aee90 2024-05-08T23:30:16.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3001e7230 con 0x7fa30006aa70 2024-05-08T23:30:16.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2fffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3001e7510 con 0x7fa30015d160 2024-05-08T23:30:16.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2ff7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa30015d160 0x7fa300156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:16.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2feffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa30006aa70 0x7fa300156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:16.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2ff7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa30015d160 0x7fa300156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:35712/0 (socket says 172.21.15.117:35712) 2024-05-08T23:30:16.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.019+0000 7fa2ff7fe700 1 -- 172.21.15.117:0/1958670639 learned_addr learned my addr 172.21.15.117:0/1958670639 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-05-08T23:30:16.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2fe7fc700 1 --2- 172.21.15.117:0/1958670639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3000aee90 0x7fa3001e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:16.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2ff7fe700 1 -- 172.21.15.117:0/1958670639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa30015d160 msgr2=0x7fa300156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-08T23:30:16.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2ff7fe700 1 -- 172.21.15.117:0/1958670639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa30015d160 msgr2=0x7fa300156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:30:16.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2ff7fe700 1 --2- 172.21.15.117:0/1958670639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa30015d160 0x7fa300156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:16.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2ff7fe700 1 --2- 172.21.15.117:0/1958670639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa30015d160 0x7fa300156410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:16.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2feffd700 1 -- 172.21.15.117:0/1958670639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa30015d160 msgr2=0x7fa300156410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:30:16.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2feffd700 1 --2- 172.21.15.117:0/1958670639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa30015d160 0x7fa300156410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2feffd700 1 -- 172.21.15.117:0/1958670639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3000aee90 msgr2=0x7fa3001e9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:16.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2feffd700 1 --2- 172.21.15.117:0/1958670639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3000aee90 0x7fa3001e9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2feffd700 1 -- 172.21.15.117:0/1958670639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e80094b0 con 0x7fa30006aa70 2024-05-08T23:30:16.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2feffd700 1 --2- 172.21.15.117:0/1958670639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa30006aa70 0x7fa300156c30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000c3c0 tx=0x7fa2f000c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:16.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2f77fe700 1 -- 172.21.15.117:0/1958670639 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f00052d0 con 0x7fa30006aa70 2024-05-08T23:30:16.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2f77fe700 1 -- 172.21.15.117:0/1958670639 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f0005450 con 0x7fa30006aa70 2024-05-08T23:30:16.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2fffff700 1 -- 172.21.15.117:0/1958670639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3001e77f0 con 0x7fa30006aa70 2024-05-08T23:30:16.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2f77fe700 1 -- 172.21.15.117:0/1958670639 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0006ac0 con 0x7fa30006aa70 2024-05-08T23:30:16.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2fffff700 1 -- 172.21.15.117:0/1958670639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa300156950 con 0x7fa30006aa70 2024-05-08T23:30:16.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2f77fe700 1 -- 172.21.15.117:0/1958670639 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa2f00039a0 con 0x7fa30006aa70 2024-05-08T23:30:16.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2f77fe700 1 -- 172.21.15.117:0/1958670639 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fa2f0012420 con 0x7fa30006aa70 2024-05-08T23:30:16.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.023+0000 7fa2fffff700 1 -- 172.21.15.117:0/1958670639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3000672a0 con 0x7fa30006aa70 2024-05-08T23:30:16.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.027+0000 7fa2f77fe700 1 -- 172.21.15.117:0/1958670639 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fa2f0020060 con 0x7fa30006aa70 2024-05-08T23:30:16.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.103+0000 7fa2fffff700 1 -- 172.21.15.117:0/1958670639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0b353abc-6eed-4974-8d15-f9d1e8e9c352", "id": 6} v 0) v1 -- 0x7fa3000af820 con 0x7fa30006aa70 2024-05-08T23:30:16.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.111+0000 7fa2f77fe700 1 -- 172.21.15.117:0/1958670639 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0b353abc-6eed-4974-8d15-f9d1e8e9c352", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa2f0003e30 con 0x7fa30006aa70 2024-05-08T23:30:16.117 INFO:teuthology.orchestra.run.smithi117.stdout:6 2024-05-08T23:30:16.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.115+0000 7fa2fffff700 1 -- 172.21.15.117:0/1958670639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa30006aa70 msgr2=0x7fa300156c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:16.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.115+0000 7fa2fffff700 1 --2- 172.21.15.117:0/1958670639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa30006aa70 0x7fa300156c30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000c3c0 tx=0x7fa2f000c6d0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.115+0000 7fa2fffff700 1 -- 172.21.15.117:0/1958670639 shutdown_connections 2024-05-08T23:30:16.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.115+0000 7fa2fffff700 1 --2- 172.21.15.117:0/1958670639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa30015d160 0x7fa300156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.115+0000 7fa2fffff700 1 --2- 172.21.15.117:0/1958670639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3000aee90 0x7fa3001e9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.115+0000 7fa2fffff700 1 --2- 172.21.15.117:0/1958670639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa30006aa70 0x7fa300156c30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.115+0000 7fa2fffff700 1 -- 172.21.15.117:0/1958670639 >> 172.21.15.117:0/1958670639 conn(0x7fa30006b470 msgr2=0x7fa3000ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:16.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.115+0000 7fa2fffff700 1 -- 172.21.15.117:0/1958670639 shutdown_connections 2024-05-08T23:30:16.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.115+0000 7fa2fffff700 1 -- 172.21.15.117:0/1958670639 wait complete. 2024-05-08T23:30:16.137 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new c2c70905-4d88-4004-81c2-1475901fd9dc 7 2024-05-08T23:30:16.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.247+0000 7fefa3109700 1 Processor -- start 2024-05-08T23:30:16.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.247+0000 7fefa3109700 1 -- start start 2024-05-08T23:30:16.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.247+0000 7fefa3109700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fef9c15d9d0 0x7fef9c15de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:16.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.247+0000 7fefa3109700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 0x7fef9c15b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:16.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.247+0000 7fefa3109700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9c0b3c30 con 0x7fef9c154430 2024-05-08T23:30:16.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.247+0000 7fefa3109700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9c15e390 con 0x7fef9c15d9d0 2024-05-08T23:30:16.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.247+0000 7fefa3109700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c154810 con 0x7fef9c15b4f0 2024-05-08T23:30:16.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa1906700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fef9c15d9d0 0x7fef9c15de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:16.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa2908700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 0x7fef9c15b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:16.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa2107700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fef9c154430 0x7fef9c0b2580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:34410/0 (socket says 172.21.15.117:34410) 2024-05-08T23:30:16.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa1906700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fef9c15d9d0 0x7fef9c15de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41156/0 (socket says 172.21.15.117:41156) 2024-05-08T23:30:16.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa2107700 1 -- 172.21.15.117:0/1191859452 learned_addr learned my addr 172.21.15.117:0/1191859452 (peer_addr_for_me v1:172.21.15.117:0/0) 2024-05-08T23:30:16.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa2908700 1 -- 172.21.15.117:0/1191859452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 msgr2=0x7fef9c15b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:30:16.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa2908700 1 -- 172.21.15.117:0/1191859452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 msgr2=0x7fef9c15b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:16.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa2908700 1 --2- 172.21.15.117:0/1191859452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 0x7fef9c15b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:16.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa2908700 1 --2- 172.21.15.117:0/1191859452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 0x7fef9c15b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:16.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa1906700 1 -- 172.21.15.117:0/1191859452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 msgr2=0x7fef9c15b980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:16.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa1906700 1 --2- 172.21.15.117:0/1191859452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 0x7fef9c15b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa1906700 1 -- 172.21.15.117:0/1191859452 >> v1:172.21.15.117:6789/0 conn(0x7fef9c154430 legacy=0x7fef9c0b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:16.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa1105700 1 -- 172.21.15.117:0/1191859452 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431751464 0 0) 0x7fef9c0b3c30 con 0x7fef9c154430 2024-05-08T23:30:16.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa1906700 1 -- 172.21.15.117:0/1191859452 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c0b3020 con 0x7fef9c15d9d0 2024-05-08T23:30:16.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa1906700 1 --2- 172.21.15.117:0/1191859452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef9c15d9d0 0x7fef9c15de50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009850 tx=0x7fef8c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99c16f126dc2fc1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:16.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa1105700 1 -- 172.21.15.117:0/1191859452 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c011830 con 0x7fef9c15d9d0 2024-05-08T23:30:16.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa1105700 1 -- 172.21.15.117:0/1191859452 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c011990 con 0x7fef9c15d9d0 2024-05-08T23:30:16.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa1105700 1 -- 172.21.15.117:0/1191859452 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c011d20 con 0x7fef9c15d9d0 2024-05-08T23:30:16.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa3109700 1 -- 172.21.15.117:0/1191859452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef9c15d9d0 msgr2=0x7fef9c15de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:16.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa3109700 1 --2- 172.21.15.117:0/1191859452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef9c15d9d0 0x7fef9c15de50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009850 tx=0x7fef8c009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:16.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa3109700 1 -- 172.21.15.117:0/1191859452 shutdown_connections 2024-05-08T23:30:16.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa3109700 1 --2- 172.21.15.117:0/1191859452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 0x7fef9c15b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa3109700 1 --2- 172.21.15.117:0/1191859452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef9c15d9d0 0x7fef9c15de50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa3109700 1 -- 172.21.15.117:0/1191859452 >> 172.21.15.117:0/1191859452 conn(0x7fef9c0cf510 msgr2=0x7fef9c0cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:16.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa3109700 1 -- 172.21.15.117:0/1191859452 shutdown_connections 2024-05-08T23:30:16.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.251+0000 7fefa3109700 1 -- 172.21.15.117:0/1191859452 wait complete. 2024-05-08T23:30:16.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa3109700 1 Processor -- start 2024-05-08T23:30:16.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa3109700 1 -- start start 2024-05-08T23:30:16.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa3109700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef9c154430 0x7fef9c144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:16.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa3109700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 0x7fef9c1586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:16.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa3109700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef9c15d9d0 0x7fef9c14f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:16.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa3109700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c1549e0 con 0x7fef9c15d9d0 2024-05-08T23:30:16.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa3109700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c159160 con 0x7fef9c154430 2024-05-08T23:30:16.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa3109700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c159440 con 0x7fef9c15b4f0 2024-05-08T23:30:16.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa1906700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 0x7fef9c1586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:16.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa2107700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef9c154430 0x7fef9c144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:16.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa2107700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef9c154430 0x7fef9c144e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41162/0 (socket says 172.21.15.117:41162) 2024-05-08T23:30:16.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa2107700 1 -- 172.21.15.117:0/3567359534 learned_addr learned my addr 172.21.15.117:0/3567359534 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-05-08T23:30:16.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa1906700 1 -- 172.21.15.117:0/3567359534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 msgr2=0x7fef9c1586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-08T23:30:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa2908700 1 --2- 172.21.15.117:0/3567359534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef9c15d9d0 0x7fef9c14f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa1906700 1 -- 172.21.15.117:0/3567359534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 msgr2=0x7fef9c1586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:30:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa1906700 1 --2- 172.21.15.117:0/3567359534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 0x7fef9c1586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa1906700 1 --2- 172.21.15.117:0/3567359534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 0x7fef9c1586c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa2107700 1 -- 172.21.15.117:0/3567359534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 msgr2=0x7fef9c1586c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:30:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa2107700 1 --2- 172.21.15.117:0/3567359534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 0x7fef9c1586c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa2107700 1 -- 172.21.15.117:0/3567359534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef9c15d9d0 msgr2=0x7fef9c14f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa2107700 1 --2- 172.21.15.117:0/3567359534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef9c15d9d0 0x7fef9c14f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa2107700 1 -- 172.21.15.117:0/3567359534 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0094b0 con 0x7fef9c154430 2024-05-08T23:30:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa2908700 1 --2- 172.21.15.117:0/3567359534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef9c15d9d0 0x7fef9c14f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa2107700 1 --2- 172.21.15.117:0/3567359534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef9c154430 0x7fef9c144e40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef9800c580 tx=0x7fef9800c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fef92ffd700 1 -- 172.21.15.117:0/3567359534 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef98004ca0 con 0x7fef9c154430 2024-05-08T23:30:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa3109700 1 -- 172.21.15.117:0/3567359534 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c159720 con 0x7fef9c154430 2024-05-08T23:30:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fefa3109700 1 -- 172.21.15.117:0/3567359534 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c145380 con 0x7fef9c154430 2024-05-08T23:30:16.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fef92ffd700 1 -- 172.21.15.117:0/3567359534 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef98004e20 con 0x7fef9c154430 2024-05-08T23:30:16.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fef92ffd700 1 -- 172.21.15.117:0/3567359534 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef98007310 con 0x7fef9c154430 2024-05-08T23:30:16.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fef92ffd700 1 -- 172.21.15.117:0/3567359534 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fef9801d3f0 con 0x7fef9c154430 2024-05-08T23:30:16.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.255+0000 7fef92ffd700 1 -- 172.21.15.117:0/3567359534 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fef98013a20 con 0x7fef9c154430 2024-05-08T23:30:16.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.259+0000 7fefa3109700 1 -- 172.21.15.117:0/3567359534 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef84005310 con 0x7fef9c154430 2024-05-08T23:30:16.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.259+0000 7fef92ffd700 1 -- 172.21.15.117:0/3567359534 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fef98013d30 con 0x7fef9c154430 2024-05-08T23:30:16.280 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:16.278+0000 7f937ff502c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:30:16.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.331+0000 7fefa3109700 1 -- 172.21.15.117:0/3567359534 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c2c70905-4d88-4004-81c2-1475901fd9dc", "id": 7} v 0) v1 -- 0x7fef84005660 con 0x7fef9c154430 2024-05-08T23:30:16.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.347+0000 7fef92ffd700 1 -- 172.21.15.117:0/3567359534 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c2c70905-4d88-4004-81c2-1475901fd9dc", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fef980133c0 con 0x7fef9c154430 2024-05-08T23:30:16.350 INFO:teuthology.orchestra.run.smithi117.stdout:7 2024-05-08T23:30:16.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.347+0000 7fefa3109700 1 -- 172.21.15.117:0/3567359534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef9c154430 msgr2=0x7fef9c144e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:16.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.347+0000 7fefa3109700 1 --2- 172.21.15.117:0/3567359534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef9c154430 0x7fef9c144e40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef9800c580 tx=0x7fef9800c890 comp rx=0 tx=0).stop 2024-05-08T23:30:16.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.347+0000 7fefa3109700 1 -- 172.21.15.117:0/3567359534 shutdown_connections 2024-05-08T23:30:16.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.347+0000 7fefa3109700 1 --2- 172.21.15.117:0/3567359534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef9c15d9d0 0x7fef9c14f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.347+0000 7fefa3109700 1 --2- 172.21.15.117:0/3567359534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef9c15b4f0 0x7fef9c1586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.347+0000 7fefa3109700 1 --2- 172.21.15.117:0/3567359534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef9c154430 0x7fef9c144e40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.347+0000 7fefa3109700 1 -- 172.21.15.117:0/3567359534 >> 172.21.15.117:0/3567359534 conn(0x7fef9c0cf510 msgr2=0x7fef9c15cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:16.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.347+0000 7fefa3109700 1 -- 172.21.15.117:0/3567359534 shutdown_connections 2024-05-08T23:30:16.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-05-08T23:30:16.347+0000 7fefa3109700 1 -- 172.21.15.117:0/3567359534 wait complete. 2024-05-08T23:30:16.367 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-08T23:30:16.367 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-08T23:30:16.369 INFO:tasks.ceph.osd.0:Started 2024-05-08T23:30:16.369 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-08T23:30:16.369 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-08T23:30:16.371 INFO:tasks.ceph.osd.1:Started 2024-05-08T23:30:16.371 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-08T23:30:16.371 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-08T23:30:16.373 INFO:tasks.ceph.osd.2:Started 2024-05-08T23:30:16.373 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-08T23:30:16.373 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-08T23:30:16.376 INFO:tasks.ceph.osd.3:Started 2024-05-08T23:30:16.376 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-08T23:30:16.376 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-08T23:30:16.378 INFO:tasks.ceph.osd.4:Started 2024-05-08T23:30:16.378 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-08T23:30:16.378 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-08T23:30:16.380 INFO:tasks.ceph.osd.5:Started 2024-05-08T23:30:16.380 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-08T23:30:16.380 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-08T23:30:16.382 INFO:tasks.ceph.osd.6:Started 2024-05-08T23:30:16.383 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-08T23:30:16.383 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-08T23:30:16.385 INFO:tasks.ceph.osd.7:Started 2024-05-08T23:30:16.385 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:30:16.404 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:16.399+0000 7fae133c22c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:30:16.408 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:16.406+0000 7f937ff502c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:30:16.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f6611b71700 1 Processor -- start 2024-05-08T23:30:16.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f6611b71700 1 -- start start 2024-05-08T23:30:16.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f6611b71700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f660c0adbf0 0x7f660c0cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:16.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f6611b71700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 0x7f660c06b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:16.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f6611b71700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660c15c6a0 con 0x7f660c0aedc0 2024-05-08T23:30:16.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f6611b71700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660c15ba90 con 0x7f660c0adbf0 2024-05-08T23:30:16.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f6611b71700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660c15bd70 con 0x7f660c0adfd0 2024-05-08T23:30:16.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f6611370700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f660c0aedc0 0x7f660c06cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52902/0 (socket says 172.21.15.81:52902) 2024-05-08T23:30:16.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f6611370700 1 -- 172.21.15.81:0/3030803847 learned_addr learned my addr 172.21.15.81:0/3030803847 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:16.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f660bfff700 1 --2- 172.21.15.81:0/3030803847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 0x7f660c06b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:16.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f6610b6f700 1 --2- 172.21.15.81:0/3030803847 >> v2:172.21.15.117:3300/0 conn(0x7f660c0adbf0 0x7f660c0cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:16.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f660bfff700 1 -- 172.21.15.81:0/3030803847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 msgr2=0x7f660c06b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:30:16.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f660bfff700 1 -- 172.21.15.81:0/3030803847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 msgr2=0x7f660c06b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:30:16.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f660bfff700 1 --2- 172.21.15.81:0/3030803847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 0x7f660c06b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:16.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f660bfff700 1 --2- 172.21.15.81:0/3030803847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 0x7f660c06b370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:16.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f660b7fe700 1 -- 172.21.15.81:0/3030803847 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693216908 0 0) 0x7f660c15c6a0 con 0x7f660c0aedc0 2024-05-08T23:30:16.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f660b7fe700 1 -- 172.21.15.81:0/3030803847 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65f40035d0 con 0x7f660c0aedc0 2024-05-08T23:30:16.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f6610b6f700 1 -- 172.21.15.81:0/3030803847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 msgr2=0x7f660c06b370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:16.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f6610b6f700 1 --2- 172.21.15.81:0/3030803847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 0x7f660c06b370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f6610b6f700 1 -- 172.21.15.81:0/3030803847 >> v1:172.21.15.81:6789/0 conn(0x7f660c0aedc0 legacy=0x7f660c06cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:16.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f6610b6f700 1 -- 172.21.15.81:0/3030803847 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c15c050 con 0x7f660c0adbf0 2024-05-08T23:30:16.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f6610b6f700 1 --2- 172.21.15.81:0/3030803847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f660c0adbf0 0x7f660c0cedd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6604009d90 tx=0x7f660400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23ecea351e557a78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:16.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f660b7fe700 1 -- 172.21.15.81:0/3030803847 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660400e070 con 0x7f660c0adbf0 2024-05-08T23:30:16.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f660b7fe700 1 -- 172.21.15.81:0/3030803847 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660400bb20 con 0x7f660c0adbf0 2024-05-08T23:30:16.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.486+0000 7f660b7fe700 1 -- 172.21.15.81:0/3030803847 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660400be90 con 0x7f660c0adbf0 2024-05-08T23:30:16.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 -- 172.21.15.81:0/3030803847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f660c0adbf0 msgr2=0x7f660c0cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:16.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 --2- 172.21.15.81:0/3030803847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f660c0adbf0 0x7f660c0cedd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6604009d90 tx=0x7f660400b330 comp rx=0 tx=0).stop 2024-05-08T23:30:16.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 -- 172.21.15.81:0/3030803847 shutdown_connections 2024-05-08T23:30:16.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 --2- 172.21.15.81:0/3030803847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 0x7f660c06b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 --2- 172.21.15.81:0/3030803847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f660c0adbf0 0x7f660c0cedd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 -- 172.21.15.81:0/3030803847 >> 172.21.15.81:0/3030803847 conn(0x7f660c070960 msgr2=0x7f660c070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:16.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 -- 172.21.15.81:0/3030803847 shutdown_connections 2024-05-08T23:30:16.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 -- 172.21.15.81:0/3030803847 wait complete. 2024-05-08T23:30:16.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 Processor -- start 2024-05-08T23:30:16.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 -- start start 2024-05-08T23:30:16.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f660c0adbf0 0x7f660c147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:16.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 0x7f660c0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:16.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f660c0aedc0 0x7f660c0ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:16.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c153630 con 0x7f660c0aedc0 2024-05-08T23:30:16.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c148410 con 0x7f660c0adbf0 2024-05-08T23:30:16.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611b71700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660c1529b0 con 0x7f660c0adfd0 2024-05-08T23:30:16.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611370700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f660c0aedc0 0x7f660c0ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:16.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611370700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f660c0aedc0 0x7f660c0ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36880/0 (socket says 172.21.15.81:36880) 2024-05-08T23:30:16.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611370700 1 -- 172.21.15.81:0/1645676780 learned_addr learned my addr 172.21.15.81:0/1645676780 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:16.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6610b6f700 1 --2- 172.21.15.81:0/1645676780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f660c0adbf0 0x7f660c147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:16.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f660bfff700 1 --2- 172.21.15.81:0/1645676780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 0x7f660c0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:16.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f660bfff700 1 -- 172.21.15.81:0/1645676780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 msgr2=0x7f660c0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-08T23:30:16.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f660bfff700 1 -- 172.21.15.81:0/1645676780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 msgr2=0x7f660c0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:30:16.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f660bfff700 1 --2- 172.21.15.81:0/1645676780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 0x7f660c0ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:16.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f660bfff700 1 --2- 172.21.15.81:0/1645676780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 0x7f660c0ac8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:16.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611370700 1 -- 172.21.15.81:0/1645676780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 msgr2=0x7f660c0ac8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:30:16.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611370700 1 --2- 172.21.15.81:0/1645676780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 0x7f660c0ac8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611370700 1 -- 172.21.15.81:0/1645676780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f660c0adbf0 msgr2=0x7f660c147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:16.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611370700 1 --2- 172.21.15.81:0/1645676780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f660c0adbf0 0x7f660c147ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.490+0000 7f6611370700 1 -- 172.21.15.81:0/1645676780 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6604009a70 con 0x7f660c0aedc0 2024-05-08T23:30:16.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.494+0000 7f6611370700 1 --2- 172.21.15.81:0/1645676780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f660c0aedc0 0x7f660c0ace20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f65fc005930 tx=0x7f65fc00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:16.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.494+0000 7f66097fa700 1 -- 172.21.15.81:0/1645676780 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc005d90 con 0x7f660c0aedc0 2024-05-08T23:30:16.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.494+0000 7f66097fa700 1 -- 172.21.15.81:0/1645676780 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc0070d0 con 0x7f660c0aedc0 2024-05-08T23:30:16.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.494+0000 7f6611b71700 1 -- 172.21.15.81:0/1645676780 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660c152c90 con 0x7f660c0aedc0 2024-05-08T23:30:16.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.494+0000 7f66097fa700 1 -- 172.21.15.81:0/1645676780 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc007440 con 0x7f660c0aedc0 2024-05-08T23:30:16.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.494+0000 7f6611b71700 1 -- 172.21.15.81:0/1645676780 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660c153250 con 0x7f660c0aedc0 2024-05-08T23:30:16.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.494+0000 7f66097fa700 1 -- 172.21.15.81:0/1645676780 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f65fc01c440 con 0x7f660c0aedc0 2024-05-08T23:30:16.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.494+0000 7f66097fa700 1 -- 172.21.15.81:0/1645676780 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f65fc012ad0 con 0x7f660c0aedc0 2024-05-08T23:30:16.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.494+0000 7f6611b71700 1 -- 172.21.15.81:0/1645676780 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660c153770 con 0x7f660c0aedc0 2024-05-08T23:30:16.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.498+0000 7f66097fa700 1 -- 172.21.15.81:0/1645676780 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f660c153770 con 0x7f660c0aedc0 2024-05-08T23:30:16.538 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:16.535+0000 7fae133c22c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:30:16.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.562+0000 7f6611b71700 1 -- 172.21.15.81:0/1645676780 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f660c0672a0 con 0x7f660c0aedc0 2024-05-08T23:30:16.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.566+0000 7f66097fa700 1 -- 172.21.15.81:0/1645676780 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f65fc021060 con 0x7f660c0aedc0 2024-05-08T23:30:16.569 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:16.569 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":10,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:16.339862+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:30:16.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.566+0000 7f6611b71700 1 -- 172.21.15.81:0/1645676780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f660c0aedc0 msgr2=0x7f660c0ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:16.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.566+0000 7f6611b71700 1 --2- 172.21.15.81:0/1645676780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f660c0aedc0 0x7f660c0ace20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f65fc005930 tx=0x7f65fc00c7d0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.566+0000 7f6611b71700 1 -- 172.21.15.81:0/1645676780 shutdown_connections 2024-05-08T23:30:16.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.566+0000 7f6611b71700 1 --2- 172.21.15.81:0/1645676780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f660c0aedc0 0x7f660c0ace20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.566+0000 7f6611b71700 1 --2- 172.21.15.81:0/1645676780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f660c0adfd0 0x7f660c0ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.566+0000 7f6611b71700 1 --2- 172.21.15.81:0/1645676780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f660c0adbf0 0x7f660c147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:16.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.566+0000 7f6611b71700 1 -- 172.21.15.81:0/1645676780 >> 172.21.15.81:0/1645676780 conn(0x7f660c070960 msgr2=0x7f660c159af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:16.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.566+0000 7f6611b71700 1 -- 172.21.15.81:0/1645676780 shutdown_connections 2024-05-08T23:30:16.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:16.566+0000 7f6611b71700 1 -- 172.21.15.81:0/1645676780 wait complete. 2024-05-08T23:30:16.581 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-08T23:30:16.581 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-08T23:30:16.739 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-05-08T23:30:16.735+0000 7f30d60d4880 -1 Falling back to public interface 2024-05-08T23:30:16.749 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-05-08T23:30:16.746+0000 7f58a2f27880 -1 Falling back to public interface 2024-05-08T23:30:16.750 INFO:tasks.ceph.osd.3.smithi081.stderr:2024-05-08T23:30:16.746+0000 7f570b265880 -1 Falling back to public interface 2024-05-08T23:30:16.750 INFO:tasks.ceph.osd.3.smithi081.stderr:2024-05-08T23:30:16.746+0000 7f570a230700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6804/0: (98) Address already in use 2024-05-08T23:30:16.763 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-05-08T23:30:16.759+0000 7fd9ba567880 -1 Falling back to public interface 2024-05-08T23:30:16.774 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-05-08T23:30:16.771+0000 7f5d7ea75880 -1 Falling back to public interface 2024-05-08T23:30:16.782 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:16.778+0000 7f937ff502c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:30:16.901 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:16.895+0000 7fae133c22c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:30:16.968 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:16.966+0000 7f937ff502c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:30:17.013 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-05-08T23:30:17.010+0000 7fed4322b880 -1 Falling back to public interface 2024-05-08T23:30:17.014 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-05-08T23:30:17.010+0000 7f40be681880 -1 Falling back to public interface 2024-05-08T23:30:17.023 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-05-08T23:30:17.019+0000 7f699966d880 -1 Falling back to public interface 2024-05-08T23:30:17.035 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:17.030+0000 7f937ff502c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:30:17.089 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:17.083+0000 7fae133c22c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:30:17.158 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:17.155+0000 7fae133c22c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:30:17.195 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:17.190+0000 7f937ff502c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:30:17.318 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:17.315+0000 7fae133c22c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:30:17.386 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T23:30:17.486 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.482+0000 7f155d985700 1 Processor -- start 2024-05-08T23:30:17.486 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.482+0000 7f155d985700 1 -- start start 2024-05-08T23:30:17.490 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.482+0000 7f155d985700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f15500e0250 0x7f155004f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:17.490 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.482+0000 7f155d985700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15500540b0 con 0x7f15500e3b40 2024-05-08T23:30:17.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.482+0000 7f155d985700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15500532f0 con 0x7f1550006150 2024-05-08T23:30:17.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.482+0000 7f155d985700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15500535d0 con 0x7f15500e0250 2024-05-08T23:30:17.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.482+0000 7f155c983700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f15500e3b40 0x7f15500e3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52914/0 (socket says 172.21.15.81:52914) 2024-05-08T23:30:17.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.482+0000 7f155c983700 1 -- 172.21.15.81:0/3173839432 learned_addr learned my addr 172.21.15.81:0/3173839432 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:17.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.482+0000 7f1557fff700 1 --2- 172.21.15.81:0/3173839432 >> v2:172.21.15.117:3300/0 conn(0x7f15500e0250 0x7f155004f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:17.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 -- 172.21.15.81:0/3173839432 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953629025 0 0) 0x7f15500540b0 con 0x7f15500e3b40 2024-05-08T23:30:17.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 -- 172.21.15.81:0/3173839432 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15480035d0 con 0x7f15500e3b40 2024-05-08T23:30:17.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 -- 172.21.15.81:0/3173839432 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1762113670 0 0) 0x7f15480035d0 con 0x7f15500e3b40 2024-05-08T23:30:17.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 -- 172.21.15.81:0/3173839432 >> v2:172.21.15.117:3300/0 conn(0x7f15500e0250 msgr2=0x7f155004f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:17.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 --2- 172.21.15.81:0/3173839432 >> v2:172.21.15.117:3300/0 conn(0x7f15500e0250 0x7f155004f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:17.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 -- 172.21.15.81:0/3173839432 >> v1:172.21.15.117:6789/0 conn(0x7f1550006150 legacy=0x7f155004f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:17.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 -- 172.21.15.81:0/3173839432 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15500538b0 con 0x7f15500e3b40 2024-05-08T23:30:17.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 -- 172.21.15.81:0/3173839432 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f1544001670 con 0x7f15500e3b40 2024-05-08T23:30:17.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 -- 172.21.15.81:0/3173839432 >> v1:172.21.15.81:6789/0 conn(0x7f15500e3b40 legacy=0x7f15500e3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:17.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 --2- 172.21.15.81:0/3173839432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1548003a20 0x7f1548003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:17.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 --2- 172.21.15.81:0/3173839432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1548004870 0x7f1548004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:17.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 --2- 172.21.15.81:0/3173839432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1548005230 0x7f154800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:17.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 -- 172.21.15.81:0/3173839432 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15500535d0 con 0x7f1548003a20 2024-05-08T23:30:17.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 -- 172.21.15.81:0/3173839432 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154800c130 con 0x7f1548005230 2024-05-08T23:30:17.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 -- 172.21.15.81:0/3173839432 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154800c2f0 con 0x7f1548004870 2024-05-08T23:30:17.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155c983700 1 --2- 172.21.15.81:0/3173839432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1548003a20 0x7f1548003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:17.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f1557fff700 1 --2- 172.21.15.81:0/3173839432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1548004870 0x7f1548004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:17.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155d184700 1 --2- 172.21.15.81:0/3173839432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1548005230 0x7f154800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:17.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155c983700 1 -- 172.21.15.81:0/3173839432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1548004870 msgr2=0x7f1548004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:17.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155c983700 1 --2- 172.21.15.81:0/3173839432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1548004870 0x7f1548004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:17.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155c983700 1 -- 172.21.15.81:0/3173839432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1548005230 msgr2=0x7f154800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:17.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155c983700 1 --2- 172.21.15.81:0/3173839432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1548005230 0x7f154800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:17.493 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155c983700 1 -- 172.21.15.81:0/3173839432 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15480035d0 con 0x7f1548003a20 2024-05-08T23:30:17.493 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155d184700 1 --2- 172.21.15.81:0/3173839432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1548005230 0x7f154800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:17.493 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155d184700 1 -- 172.21.15.81:0/3173839432 reap_dead start 2024-05-08T23:30:17.493 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155c983700 1 --2- 172.21.15.81:0/3173839432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1548003a20 0x7f1548003e40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f154400c3a0 tx=0x7f15480058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9071aefff140d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:17.493 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 -- 172.21.15.81:0/3173839432 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544007ca0 con 0x7f1548003a20 2024-05-08T23:30:17.493 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 -- 172.21.15.81:0/3173839432 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1544005980 con 0x7f1548003a20 2024-05-08T23:30:17.493 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f15577fe700 1 -- 172.21.15.81:0/3173839432 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544005d30 con 0x7f1548003a20 2024-05-08T23:30:17.493 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155d985700 1 -- 172.21.15.81:0/3173839432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1548003a20 msgr2=0x7f1548003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:17.493 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155d985700 1 --2- 172.21.15.81:0/3173839432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1548003a20 0x7f1548003e40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f154400c3a0 tx=0x7f15480058a0 comp rx=0 tx=0).stop 2024-05-08T23:30:17.493 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155d985700 1 -- 172.21.15.81:0/3173839432 shutdown_connections 2024-05-08T23:30:17.493 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155d985700 1 --2- 172.21.15.81:0/3173839432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1548003a20 0x7f1548003e40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:17.493 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155d985700 1 -- 172.21.15.81:0/3173839432 >> 172.21.15.81:0/3173839432 conn(0x7f1550054b70 msgr2=0x7f1550054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:17.493 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155d985700 1 -- 172.21.15.81:0/3173839432 shutdown_connections 2024-05-08T23:30:17.493 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.486+0000 7f155d985700 1 -- 172.21.15.81:0/3173839432 wait complete. 2024-05-08T23:30:17.496 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155d985700 1 Processor -- start 2024-05-08T23:30:17.496 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155d985700 1 -- start start 2024-05-08T23:30:17.496 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155d985700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1548003a20 0x7f15500e1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:17.496 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155d985700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15500e3360 0x7f15500df390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:17.496 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155d985700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15500379d0 0x7f15500d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:17.496 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155d985700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15500db130 con 0x7f15500e3360 2024-05-08T23:30:17.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155d985700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15500e22c0 con 0x7f15500379d0 2024-05-08T23:30:17.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155d985700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15500e25a0 con 0x7f1548003a20 2024-05-08T23:30:17.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f1557fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15500e3360 0x7f15500df390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:17.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f1557fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15500e3360 0x7f15500df390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36886/0 (socket says 172.21.15.81:36886) 2024-05-08T23:30:17.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155c983700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1548003a20 0x7f15500e1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:17.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f1557fff700 1 -- 172.21.15.81:0/740549616 learned_addr learned my addr 172.21.15.81:0/740549616 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:17.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155d184700 1 --2- 172.21.15.81:0/740549616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15500379d0 0x7f15500d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:17.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155c983700 1 -- 172.21.15.81:0/740549616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1548003a20 msgr2=0x7f15500e1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-08T23:30:17.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155c983700 1 -- 172.21.15.81:0/740549616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1548003a20 msgr2=0x7f15500e1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:30:17.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155c983700 1 --2- 172.21.15.81:0/740549616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1548003a20 0x7f15500e1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:30:17.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155c983700 1 --2- 172.21.15.81:0/740549616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1548003a20 0x7f15500e1bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:30:17.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f1557fff700 1 -- 172.21.15.81:0/740549616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1548003a20 msgr2=0x7f15500e1bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:30:17.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f1557fff700 1 --2- 172.21.15.81:0/740549616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1548003a20 0x7f15500e1bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:17.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f1557fff700 1 -- 172.21.15.81:0/740549616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15500379d0 msgr2=0x7f15500d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:17.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f1557fff700 1 --2- 172.21.15.81:0/740549616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15500379d0 0x7f15500d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:17.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f1557fff700 1 -- 172.21.15.81:0/740549616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15500e2880 con 0x7f15500e3360 2024-05-08T23:30:17.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f1557fff700 1 --2- 172.21.15.81:0/740549616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15500e3360 0x7f15500df390 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f154c011aa0 tx=0x7f154c011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:17.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f15557fa700 1 -- 172.21.15.81:0/740549616 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c00f940 con 0x7f15500e3360 2024-05-08T23:30:17.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f15557fa700 1 -- 172.21.15.81:0/740549616 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c00faa0 con 0x7f15500e3360 2024-05-08T23:30:17.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155d985700 1 -- 172.21.15.81:0/740549616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15500e2b60 con 0x7f15500e3360 2024-05-08T23:30:17.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f15557fa700 1 -- 172.21.15.81:0/740549616 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c00fe30 con 0x7f15500e3360 2024-05-08T23:30:17.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f155d985700 1 -- 172.21.15.81:0/740549616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15500df8d0 con 0x7f15500e3360 2024-05-08T23:30:17.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f15557fa700 1 -- 172.21.15.81:0/740549616 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f154c014740 con 0x7f15500e3360 2024-05-08T23:30:17.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.490+0000 7f15557fa700 1 -- 172.21.15.81:0/740549616 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f154c01dc60 con 0x7f15500e3360 2024-05-08T23:30:17.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.494+0000 7f155d985700 1 -- 172.21.15.81:0/740549616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1550005440 con 0x7f15500e3360 2024-05-08T23:30:17.499 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.494+0000 7f15557fa700 1 -- 172.21.15.81:0/740549616 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f154c022020 con 0x7f15500e3360 2024-05-08T23:30:17.560 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.558+0000 7f155d985700 1 -- 172.21.15.81:0/740549616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1550054280 con 0x7f15500e3360 2024-05-08T23:30:17.560 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.558+0000 7f15557fa700 1 -- 172.21.15.81:0/740549616 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f154c014b30 con 0x7f15500e3360 2024-05-08T23:30:17.560 INFO:teuthology.misc.health.smithi081.stdout: 2024-05-08T23:30:17.561 INFO:teuthology.misc.health.smithi081.stdout:{"epoch":10,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:16.339862+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:30:17.562 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.558+0000 7f155d985700 1 -- 172.21.15.81:0/740549616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15500e3360 msgr2=0x7f15500df390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:17.562 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.558+0000 7f155d985700 1 --2- 172.21.15.81:0/740549616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15500e3360 0x7f15500df390 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f154c011aa0 tx=0x7f154c011db0 comp rx=0 tx=0).stop 2024-05-08T23:30:17.562 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.558+0000 7f155d985700 1 -- 172.21.15.81:0/740549616 shutdown_connections 2024-05-08T23:30:17.562 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.558+0000 7f155d985700 1 --2- 172.21.15.81:0/740549616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15500379d0 0x7f15500d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:17.562 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.558+0000 7f155d985700 1 --2- 172.21.15.81:0/740549616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15500e3360 0x7f15500df390 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:17.562 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.558+0000 7f155d985700 1 --2- 172.21.15.81:0/740549616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1548003a20 0x7f15500e1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:17.562 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.558+0000 7f155d985700 1 -- 172.21.15.81:0/740549616 >> 172.21.15.81:0/740549616 conn(0x7f1550054b70 msgr2=0x7f155004f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:17.562 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.558+0000 7f155d985700 1 -- 172.21.15.81:0/740549616 shutdown_connections 2024-05-08T23:30:17.562 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:17.558+0000 7f155d985700 1 -- 172.21.15.81:0/740549616 wait complete. 2024-05-08T23:30:17.575 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-08T23:30:17.899 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:17.894+0000 7f937ff502c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:30:17.967 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:17.962+0000 7f937ff502c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:30:18.021 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:18.015+0000 7fae133c22c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:30:18.089 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:18.083+0000 7fae133c22c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:30:18.147 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:18.142+0000 7f937ff502c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:30:18.212 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:18.210+0000 7f937ff502c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:30:18.254 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:18.251+0000 7fae133c22c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:30:18.281 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:18.278+0000 7f937ff502c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:30:18.320 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:18.315+0000 7fae133c22c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:30:18.386 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:18.383+0000 7fae133c22c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:30:18.567 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:18.562+0000 7f937ff502c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:30:18.631 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:18.626+0000 7f937ff502c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:30:18.678 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:18.675+0000 7fae133c22c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:30:18.750 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:18.747+0000 7fae133c22c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:30:19.290 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:19.286+0000 7f937ff502c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:30:19.418 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-05-08T23:30:19.415+0000 7fae133c22c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:30:20.149 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-05-08T23:30:20.146+0000 7f58a2f27880 -1 osd.0 0 log_to_monitors true 2024-05-08T23:30:20.329 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:20.326+0000 7f9366ac9700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T23:30:20.330 INFO:tasks.ceph.mgr.y.smithi081.stderr:2024-05-08T23:30:20.326+0000 7f9366ac9700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-08T23:30:20.340 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-05-08T23:30:20.335+0000 7f30d60d4880 -1 osd.7 0 log_to_monitors true 2024-05-08T23:30:20.631 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-05-08T23:30:20.627+0000 7f5d7ea75880 -1 osd.5 0 log_to_monitors true 2024-05-08T23:30:20.638 INFO:tasks.ceph.osd.3.smithi081.stderr:2024-05-08T23:30:20.634+0000 7f570b265880 -1 osd.3 0 log_to_monitors true 2024-05-08T23:30:20.640 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-05-08T23:30:20.638+0000 7fed4322b880 -1 osd.1 0 log_to_monitors true 2024-05-08T23:30:20.649 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-05-08T23:30:20.643+0000 7fd9ba567880 -1 osd.4 0 log_to_monitors true 2024-05-08T23:30:20.830 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-05-08T23:30:20.827+0000 7f699966d880 -1 osd.6 0 log_to_monitors true 2024-05-08T23:30:20.845 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-05-08T23:30:20.842+0000 7f40be681880 -1 osd.2 0 log_to_monitors true 2024-05-08T23:30:22.951 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-05-08T23:30:22.947+0000 7f30d16f0700 -1 osd.7 0 waiting for initial osdmap 2024-05-08T23:30:23.021 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-05-08T23:30:23.018+0000 7f589e543700 -1 osd.0 0 waiting for initial osdmap 2024-05-08T23:30:23.109 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-05-08T23:30:23.103+0000 7f30c99c5700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:30:23.209 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-05-08T23:30:23.206+0000 7f5896818700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:30:24.188 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-05-08T23:30:24.183+0000 7fd9b5b83700 -1 osd.4 0 waiting for initial osdmap 2024-05-08T23:30:24.199 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-05-08T23:30:24.195+0000 7fd9ade58700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:30:24.205 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-05-08T23:30:24.199+0000 7f6994c89700 -1 osd.6 0 waiting for initial osdmap 2024-05-08T23:30:24.211 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-05-08T23:30:24.207+0000 7f698cf5e700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:30:24.230 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-05-08T23:30:24.226+0000 7f40b9c9d700 -1 osd.2 0 waiting for initial osdmap 2024-05-08T23:30:24.234 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-05-08T23:30:24.230+0000 7f40b1f72700 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:30:24.237 INFO:tasks.ceph.osd.3.smithi081.stderr:2024-05-08T23:30:24.234+0000 7f5706881700 -1 osd.3 0 waiting for initial osdmap 2024-05-08T23:30:24.242 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-05-08T23:30:24.239+0000 7f5d7a091700 -1 osd.5 0 waiting for initial osdmap 2024-05-08T23:30:24.244 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-05-08T23:30:24.242+0000 7fed3e847700 -1 osd.1 0 waiting for initial osdmap 2024-05-08T23:30:24.246 INFO:tasks.ceph.osd.3.smithi081.stderr:2024-05-08T23:30:24.242+0000 7f56feb56700 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:30:24.250 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-05-08T23:30:24.247+0000 7f5d72366700 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:30:24.257 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-05-08T23:30:24.254+0000 7fed36b1c700 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:30:24.381 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T23:30:24.488 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42fb184700 1 Processor -- start 2024-05-08T23:30:24.488 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42fb184700 1 -- start start 2024-05-08T23:30:24.489 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42fb184700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f42f40c6670 0x7f42f40cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:24.489 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42fb184700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f415b570 0x7f42f40c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:24.489 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42fb184700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f40cb670 con 0x7f42f40b2760 2024-05-08T23:30:24.489 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42fb184700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f40cbc30 con 0x7f42f40c6670 2024-05-08T23:30:24.489 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42fb184700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f40c6f90 con 0x7f42f415b570 2024-05-08T23:30:24.489 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42f9721700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f42f40b2760 0x7f42f4099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52924/0 (socket says 172.21.15.81:52924) 2024-05-08T23:30:24.489 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42f9721700 1 -- 172.21.15.81:0/2151549009 learned_addr learned my addr 172.21.15.81:0/2151549009 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:24.489 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42f8f20700 1 --2- 172.21.15.81:0/2151549009 >> v2:172.21.15.117:3300/0 conn(0x7f42f40c6670 0x7f42f40cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:24.489 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42ebfff700 1 --2- 172.21.15.81:0/2151549009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f415b570 0x7f42f40c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:24.490 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42f8f20700 1 -- 172.21.15.81:0/2151549009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f415b570 msgr2=0x7f42f40c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:24.490 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42eb7fe700 1 -- 172.21.15.81:0/2151549009 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450407169 0 0) 0x7f42f40cb670 con 0x7f42f40b2760 2024-05-08T23:30:24.490 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42f8f20700 1 --2- 172.21.15.81:0/2151549009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f415b570 0x7f42f40c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:24.490 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42f8f20700 1 -- 172.21.15.81:0/2151549009 >> v1:172.21.15.81:6789/0 conn(0x7f42f40b2760 legacy=0x7f42f4099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:24.490 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42f8f20700 1 -- 172.21.15.81:0/2151549009 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f40d0200 con 0x7f42f40c6670 2024-05-08T23:30:24.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42f8f20700 1 --2- 172.21.15.81:0/2151549009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42f40c6670 0x7f42f40cf0b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009850 tx=0x7f42f0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a190daf453ecd972 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:24.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42eb7fe700 1 -- 172.21.15.81:0/2151549009 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0011830 con 0x7f42f40c6670 2024-05-08T23:30:24.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42eb7fe700 1 -- 172.21.15.81:0/2151549009 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f0011990 con 0x7f42f40c6670 2024-05-08T23:30:24.491 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42eb7fe700 1 -- 172.21.15.81:0/2151549009 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0011d40 con 0x7f42f40c6670 2024-05-08T23:30:24.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42fb184700 1 -- 172.21.15.81:0/2151549009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42f40c6670 msgr2=0x7f42f40cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:24.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.486+0000 7f42fb184700 1 --2- 172.21.15.81:0/2151549009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42f40c6670 0x7f42f40cf0b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009850 tx=0x7f42f0009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:24.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.490+0000 7f42fb184700 1 -- 172.21.15.81:0/2151549009 shutdown_connections 2024-05-08T23:30:24.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.490+0000 7f42fb184700 1 --2- 172.21.15.81:0/2151549009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f415b570 0x7f42f40c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:24.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.490+0000 7f42fb184700 1 --2- 172.21.15.81:0/2151549009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42f40c6670 0x7f42f40cf0b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:24.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.490+0000 7f42fb184700 1 -- 172.21.15.81:0/2151549009 >> 172.21.15.81:0/2151549009 conn(0x7f42f40d1020 msgr2=0x7f42f40d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:24.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.490+0000 7f42fb184700 1 -- 172.21.15.81:0/2151549009 shutdown_connections 2024-05-08T23:30:24.492 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.490+0000 7f42fb184700 1 -- 172.21.15.81:0/2151549009 wait complete. 2024-05-08T23:30:24.495 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.490+0000 7f42fb184700 1 Processor -- start 2024-05-08T23:30:24.495 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.490+0000 7f42fb184700 1 -- start start 2024-05-08T23:30:24.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42fb184700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f40b2760 0x7f42f4154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:24.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42fb184700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42f40c6670 0x7f42f4154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:24.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42fb184700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f415b570 0x7f42f40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:24.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42fb184700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f406b470 con 0x7f42f40b2760 2024-05-08T23:30:24.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42fb184700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f40c8f70 con 0x7f42f40c6670 2024-05-08T23:30:24.497 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42fb184700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f4153f80 con 0x7f42f415b570 2024-05-08T23:30:24.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42f8f20700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f40b2760 0x7f42f4154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:24.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42f8f20700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f40b2760 0x7f42f4154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36970/0 (socket says 172.21.15.81:36970) 2024-05-08T23:30:24.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42f8f20700 1 -- 172.21.15.81:0/1353381806 learned_addr learned my addr 172.21.15.81:0/1353381806 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:24.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42f9721700 1 --2- 172.21.15.81:0/1353381806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f415b570 0x7f42f40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:24.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42ebfff700 1 --2- 172.21.15.81:0/1353381806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42f40c6670 0x7f42f4154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:24.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42f8f20700 1 -- 172.21.15.81:0/1353381806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f415b570 msgr2=0x7f42f40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:24.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42f8f20700 1 --2- 172.21.15.81:0/1353381806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f415b570 0x7f42f40ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:24.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42f8f20700 1 -- 172.21.15.81:0/1353381806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42f40c6670 msgr2=0x7f42f4154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:24.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42f8f20700 1 --2- 172.21.15.81:0/1353381806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42f40c6670 0x7f42f4154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:24.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42f8f20700 1 -- 172.21.15.81:0/1353381806 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f00094b0 con 0x7f42f40b2760 2024-05-08T23:30:24.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42f8f20700 1 --2- 172.21.15.81:0/1353381806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f40b2760 0x7f42f4154510 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f42f0000c00 tx=0x7f42f0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:24.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42e97fa700 1 -- 172.21.15.81:0/1353381806 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f00036c0 con 0x7f42f40b2760 2024-05-08T23:30:24.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42e97fa700 1 -- 172.21.15.81:0/1353381806 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f001f070 con 0x7f42f40b2760 2024-05-08T23:30:24.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42fb184700 1 -- 172.21.15.81:0/1353381806 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f4155060 con 0x7f42f40b2760 2024-05-08T23:30:24.498 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42e97fa700 1 -- 172.21.15.81:0/1353381806 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0003a80 con 0x7f42f40b2760 2024-05-08T23:30:24.499 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42fb184700 1 -- 172.21.15.81:0/1353381806 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f406b650 con 0x7f42f40b2760 2024-05-08T23:30:24.499 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.494+0000 7f42fb184700 1 -- 172.21.15.81:0/1353381806 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f4155580 con 0x7f42f40b2760 2024-05-08T23:30:24.500 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.498+0000 7f42e97fa700 1 -- 172.21.15.81:0/1353381806 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f42f0003be0 con 0x7f42f40b2760 2024-05-08T23:30:24.500 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.498+0000 7f42e97fa700 1 --2- 172.21.15.81:0/1353381806 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42e00666a0 0x7f42e0068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:24.500 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.498+0000 7f42e97fa700 1 -- 172.21.15.81:0/1353381806 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2791+0+0 (secure 0 0 0) 0x7f42f0021070 con 0x7f42f40b2760 2024-05-08T23:30:24.500 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.498+0000 7f42ebfff700 1 --2- 172.21.15.81:0/1353381806 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42e00666a0 0x7f42e0068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:24.501 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.498+0000 7f42ebfff700 1 --2- 172.21.15.81:0/1353381806 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42e00666a0 0x7f42e0068b60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f42f4158350 tx=0x7f42e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:24.505 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.502+0000 7f42e97fa700 1 -- 172.21.15.81:0/1353381806 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f42f4155580 con 0x7f42f40b2760 2024-05-08T23:30:24.676 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.674+0000 7f42fb184700 1 -- 172.21.15.81:0/1353381806 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42f4155580 con 0x7f42f40b2760 2024-05-08T23:30:24.678 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.674+0000 7f42e97fa700 1 -- 172.21.15.81:0/1353381806 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+7920 (secure 0 0 0) 0x7f42f4155580 con 0x7f42f40b2760 2024-05-08T23:30:24.678 INFO:teuthology.misc.health.smithi081.stdout: 2024-05-08T23:30:24.678 INFO:teuthology.misc.health.smithi081.stdout:{"epoch":14,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:24.197712+0000","last_up_change":"2024-05-08T23:30:24.197712+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:30:24.681 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.678+0000 7f42fb184700 1 -- 172.21.15.81:0/1353381806 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42e00666a0 msgr2=0x7f42e0068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:24.681 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.678+0000 7f42fb184700 1 --2- 172.21.15.81:0/1353381806 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42e00666a0 0x7f42e0068b60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f42f4158350 tx=0x7f42e4008040 comp rx=0 tx=0).stop 2024-05-08T23:30:24.681 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.678+0000 7f42fb184700 1 -- 172.21.15.81:0/1353381806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f40b2760 msgr2=0x7f42f4154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:24.681 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.678+0000 7f42fb184700 1 --2- 172.21.15.81:0/1353381806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f40b2760 0x7f42f4154510 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f42f0000c00 tx=0x7f42f0004770 comp rx=0 tx=0).stop 2024-05-08T23:30:24.681 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.678+0000 7f42fb184700 1 -- 172.21.15.81:0/1353381806 shutdown_connections 2024-05-08T23:30:24.681 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.678+0000 7f42fb184700 1 --2- 172.21.15.81:0/1353381806 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42e00666a0 0x7f42e0068b60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:24.681 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.678+0000 7f42fb184700 1 --2- 172.21.15.81:0/1353381806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f415b570 0x7f42f40ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:24.681 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.678+0000 7f42fb184700 1 --2- 172.21.15.81:0/1353381806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42f40c6670 0x7f42f4154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:24.681 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.678+0000 7f42fb184700 1 --2- 172.21.15.81:0/1353381806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f40b2760 0x7f42f4154510 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:24.681 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.678+0000 7f42fb184700 1 -- 172.21.15.81:0/1353381806 >> 172.21.15.81:0/1353381806 conn(0x7f42f40d1020 msgr2=0x7f42f40cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:24.681 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.678+0000 7f42fb184700 1 -- 172.21.15.81:0/1353381806 shutdown_connections 2024-05-08T23:30:24.681 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:24.678+0000 7f42fb184700 1 -- 172.21.15.81:0/1353381806 wait complete. 2024-05-08T23:30:24.692 DEBUG:teuthology.misc:2 of 8 OSDs are up 2024-05-08T23:30:31.499 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T23:30:31.611 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.605+0000 7fa37276b700 1 Processor -- start 2024-05-08T23:30:31.611 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.605+0000 7fa37276b700 1 -- start start 2024-05-08T23:30:31.613 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa37276b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa36c0cd7b0 0x7fa36c15eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:31.613 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa37276b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa36c06aa70 0x7fa36c15b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:31.613 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa37276b700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36c0cdb90 con 0x7fa36c071b80 2024-05-08T23:30:31.613 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa37276b700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa36c0adb70 con 0x7fa36c06aa70 2024-05-08T23:30:31.613 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa37276b700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa36c0ade50 con 0x7fa36c0cd7b0 2024-05-08T23:30:31.614 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa371f6a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa36c06aa70 0x7fa36c15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:31.614 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa370f68700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa36c071b80 0x7fa36c06a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52226/0 (socket says 172.21.15.81:52226) 2024-05-08T23:30:31.614 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa370f68700 1 -- 172.21.15.81:0/278158471 learned_addr learned my addr 172.21.15.81:0/278158471 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:31.614 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa371769700 1 --2- 172.21.15.81:0/278158471 >> v2:172.21.15.117:3300/0 conn(0x7fa36c0cd7b0 0x7fa36c15eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:31.614 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa363fff700 1 -- 172.21.15.81:0/278158471 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850771885 0 0) 0x7fa36c0cdb90 con 0x7fa36c071b80 2024-05-08T23:30:31.614 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa363fff700 1 -- 172.21.15.81:0/278158471 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3580035d0 con 0x7fa36c071b80 2024-05-08T23:30:31.614 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa371f6a700 1 -- 172.21.15.81:0/278158471 >> v2:172.21.15.117:3300/0 conn(0x7fa36c0cd7b0 msgr2=0x7fa36c15eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:31.614 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa371f6a700 1 --2- 172.21.15.81:0/278158471 >> v2:172.21.15.117:3300/0 conn(0x7fa36c0cd7b0 0x7fa36c15eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:31.614 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa371f6a700 1 -- 172.21.15.81:0/278158471 >> v1:172.21.15.81:6789/0 conn(0x7fa36c071b80 legacy=0x7fa36c06a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:31.614 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa371f6a700 1 -- 172.21.15.81:0/278158471 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c0ac0f0 con 0x7fa36c06aa70 2024-05-08T23:30:31.614 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa371f6a700 1 --2- 172.21.15.81:0/278158471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa36c06aa70 0x7fa36c15b800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa36c06e240 tx=0x7fa36400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27297b620c217bcc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:31.614 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa363fff700 1 -- 172.21.15.81:0/278158471 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36400e070 con 0x7fa36c06aa70 2024-05-08T23:30:31.614 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa363fff700 1 -- 172.21.15.81:0/278158471 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36400bb20 con 0x7fa36c06aa70 2024-05-08T23:30:31.614 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa363fff700 1 -- 172.21.15.81:0/278158471 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364012430 con 0x7fa36c06aa70 2024-05-08T23:30:31.615 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa37276b700 1 -- 172.21.15.81:0/278158471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa36c06aa70 msgr2=0x7fa36c15b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:31.615 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa37276b700 1 --2- 172.21.15.81:0/278158471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa36c06aa70 0x7fa36c15b800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa36c06e240 tx=0x7fa36400b330 comp rx=0 tx=0).stop 2024-05-08T23:30:31.615 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa37276b700 1 -- 172.21.15.81:0/278158471 shutdown_connections 2024-05-08T23:30:31.615 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa37276b700 1 --2- 172.21.15.81:0/278158471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa36c06aa70 0x7fa36c15b800 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:31.615 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa37276b700 1 --2- 172.21.15.81:0/278158471 >> v2:172.21.15.117:3300/0 conn(0x7fa36c0cd7b0 0x7fa36c15eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:31.616 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa37276b700 1 -- 172.21.15.81:0/278158471 >> 172.21.15.81:0/278158471 conn(0x7fa36c0cee60 msgr2=0x7fa36c0cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:31.616 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa37276b700 1 -- 172.21.15.81:0/278158471 shutdown_connections 2024-05-08T23:30:31.616 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.609+0000 7fa37276b700 1 -- 172.21.15.81:0/278158471 wait complete. 2024-05-08T23:30:31.616 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa37276b700 1 Processor -- start 2024-05-08T23:30:31.616 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa37276b700 1 -- start start 2024-05-08T23:30:31.618 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa37276b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa36c06aa70 0x7fa36c15df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:31.619 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa37276b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa36c071b80 0x7fa36c15c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:31.619 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa37276b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa36c0cd7b0 0x7fa36c141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:31.619 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa37276b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c0b18b0 con 0x7fa36c071b80 2024-05-08T23:30:31.619 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa37276b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c0b1b50 con 0x7fa36c06aa70 2024-05-08T23:30:31.619 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa37276b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c0b1e30 con 0x7fa36c0cd7b0 2024-05-08T23:30:31.619 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa370f68700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa36c071b80 0x7fa36c15c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:31.619 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa370f68700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa36c071b80 0x7fa36c15c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36266/0 (socket says 172.21.15.81:36266) 2024-05-08T23:30:31.619 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa370f68700 1 -- 172.21.15.81:0/2926070709 learned_addr learned my addr 172.21.15.81:0/2926070709 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:31.619 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa371769700 1 --2- 172.21.15.81:0/2926070709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa36c06aa70 0x7fa36c15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:31.619 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa371f6a700 1 --2- 172.21.15.81:0/2926070709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa36c0cd7b0 0x7fa36c141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:31.619 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa370f68700 1 -- 172.21.15.81:0/2926070709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa36c0cd7b0 msgr2=0x7fa36c141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:31.619 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa370f68700 1 --2- 172.21.15.81:0/2926070709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa36c0cd7b0 0x7fa36c141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:31.619 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa370f68700 1 -- 172.21.15.81:0/2926070709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa36c06aa70 msgr2=0x7fa36c15df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:31.619 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa370f68700 1 --2- 172.21.15.81:0/2926070709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa36c06aa70 0x7fa36c15df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:31.620 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa370f68700 1 -- 172.21.15.81:0/2926070709 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa364009a70 con 0x7fa36c071b80 2024-05-08T23:30:31.620 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa371769700 1 --2- 172.21.15.81:0/2926070709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa36c06aa70 0x7fa36c15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:31.620 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa370f68700 1 --2- 172.21.15.81:0/2926070709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa36c071b80 0x7fa36c15c360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa36800c700 tx=0x7fa36800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:31.620 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa361ffb700 1 -- 172.21.15.81:0/2926070709 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368005d90 con 0x7fa36c071b80 2024-05-08T23:30:31.620 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa361ffb700 1 -- 172.21.15.81:0/2926070709 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3680070d0 con 0x7fa36c071b80 2024-05-08T23:30:31.620 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa37276b700 1 -- 172.21.15.81:0/2926070709 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36c0b2110 con 0x7fa36c071b80 2024-05-08T23:30:31.620 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa361ffb700 1 -- 172.21.15.81:0/2926070709 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368007460 con 0x7fa36c071b80 2024-05-08T23:30:31.620 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.613+0000 7fa37276b700 1 -- 172.21.15.81:0/2926070709 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36c143810 con 0x7fa36c071b80 2024-05-08T23:30:31.620 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.617+0000 7fa37276b700 1 -- 172.21.15.81:0/2926070709 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36c0731d0 con 0x7fa36c071b80 2024-05-08T23:30:31.621 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.617+0000 7fa361ffb700 1 -- 172.21.15.81:0/2926070709 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3680075c0 con 0x7fa36c071b80 2024-05-08T23:30:31.622 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.617+0000 7fa361ffb700 1 --2- 172.21.15.81:0/2926070709 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa350066730 0x7fa350068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:31.622 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.617+0000 7fa361ffb700 1 -- 172.21.15.81:0/2926070709 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa368083420 con 0x7fa36c071b80 2024-05-08T23:30:31.622 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.617+0000 7fa371769700 1 --2- 172.21.15.81:0/2926070709 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa350066730 0x7fa350068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:31.623 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.617+0000 7fa371769700 1 --2- 172.21.15.81:0/2926070709 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa350066730 0x7fa350068bf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa36c14dbf0 tx=0x7fa35c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:31.632 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.629+0000 7fa361ffb700 1 -- 172.21.15.81:0/2926070709 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa368050af0 con 0x7fa36c071b80 2024-05-08T23:30:31.807 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.801+0000 7fa37276b700 1 -- 172.21.15.81:0/2926070709 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa36c06d790 con 0x7fa36c071b80 2024-05-08T23:30:31.808 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.805+0000 7fa361ffb700 1 -- 172.21.15.81:0/2926070709 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12382 (secure 0 0 0) 0x7fa368054110 con 0x7fa36c071b80 2024-05-08T23:30:31.808 INFO:teuthology.misc.health.smithi081.stdout: 2024-05-08T23:30:31.809 INFO:teuthology.misc.health.smithi081.stdout:{"epoch":18,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:28.221779+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:30:31.811 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.805+0000 7fa37276b700 1 -- 172.21.15.81:0/2926070709 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa350066730 msgr2=0x7fa350068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:31.811 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.805+0000 7fa37276b700 1 --2- 172.21.15.81:0/2926070709 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa350066730 0x7fa350068bf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa36c14dbf0 tx=0x7fa35c008040 comp rx=0 tx=0).stop 2024-05-08T23:30:31.811 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.805+0000 7fa37276b700 1 -- 172.21.15.81:0/2926070709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa36c071b80 msgr2=0x7fa36c15c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:31.811 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.805+0000 7fa37276b700 1 --2- 172.21.15.81:0/2926070709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa36c071b80 0x7fa36c15c360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa36800c700 tx=0x7fa36800cac0 comp rx=0 tx=0).stop 2024-05-08T23:30:31.812 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.805+0000 7fa37276b700 1 -- 172.21.15.81:0/2926070709 shutdown_connections 2024-05-08T23:30:31.812 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.805+0000 7fa37276b700 1 --2- 172.21.15.81:0/2926070709 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa350066730 0x7fa350068bf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:31.812 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.805+0000 7fa37276b700 1 --2- 172.21.15.81:0/2926070709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa36c0cd7b0 0x7fa36c141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:31.812 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.805+0000 7fa37276b700 1 --2- 172.21.15.81:0/2926070709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa36c071b80 0x7fa36c15c360 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:31.812 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.805+0000 7fa37276b700 1 --2- 172.21.15.81:0/2926070709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa36c06aa70 0x7fa36c15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:31.812 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.805+0000 7fa37276b700 1 -- 172.21.15.81:0/2926070709 >> 172.21.15.81:0/2926070709 conn(0x7fa36c0cee60 msgr2=0x7fa36c06a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:31.812 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.805+0000 7fa37276b700 1 -- 172.21.15.81:0/2926070709 shutdown_connections 2024-05-08T23:30:31.812 INFO:teuthology.misc.health.smithi081.stderr:2024-05-08T23:30:31.805+0000 7fa37276b700 1 -- 172.21.15.81:0/2926070709 wait complete. 2024-05-08T23:30:31.822 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-08T23:30:31.822 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-08T23:30:31.823 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T23:30:31.823 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T23:30:31.825 INFO:tasks.ceph.mds.a:Started 2024-05-08T23:30:31.825 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T23:30:31.825 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T23:30:31.827 INFO:tasks.ceph.mds.c:Started 2024-05-08T23:30:31.827 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T23:30:31.827 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T23:30:31.830 INFO:tasks.ceph.mds.b:Started 2024-05-08T23:30:31.830 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T23:30:31.830 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T23:30:31.832 INFO:tasks.ceph.mds.d:Started 2024-05-08T23:30:31.832 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-08T23:30:31.835 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:30:31.907 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2024-05-08T23:30:31.912 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2024-05-08T23:30:31.920 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-05-08T23:30:31.928 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-05-08T23:30:31.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.933+0000 7f8f5ffff700 1 Processor -- start 2024-05-08T23:30:31.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.933+0000 7f8f5ffff700 1 -- start start 2024-05-08T23:30:31.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f601622f0 0x7f8f601626d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:31.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5ffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8f600b2de0 0x7f8f600b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:31.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5ffff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6006cab0 con 0x7f8f6006b370 2024-05-08T23:30:31.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5ffff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f600b3880 con 0x7f8f600b2de0 2024-05-08T23:30:31.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600b3b60 con 0x7f8f601622f0 2024-05-08T23:30:31.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5f7fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f8f6006b370 0x7f8f60157e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52242/0 (socket says 172.21.15.81:52242) 2024-05-08T23:30:31.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5f7fe700 1 -- 172.21.15.81:0/680178784 learned_addr learned my addr 172.21.15.81:0/680178784 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:31.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5e7fc700 1 --2- 172.21.15.81:0/680178784 >> v2:172.21.15.117:3300/0 conn(0x7f8f600b2de0 0x7f8f600b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:31.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5effd700 1 --2- 172.21.15.81:0/680178784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f601622f0 0x7f8f601626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:31.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5dffb700 1 -- 172.21.15.81:0/680178784 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712053617 0 0) 0x7f8f6006cab0 con 0x7f8f6006b370 2024-05-08T23:30:31.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5dffb700 1 -- 172.21.15.81:0/680178784 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f480035d0 con 0x7f8f6006b370 2024-05-08T23:30:31.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5e7fc700 1 -- 172.21.15.81:0/680178784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f601622f0 msgr2=0x7f8f601626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:31.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5e7fc700 1 --2- 172.21.15.81:0/680178784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f601622f0 0x7f8f601626d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:31.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5e7fc700 1 -- 172.21.15.81:0/680178784 >> v1:172.21.15.81:6789/0 conn(0x7f8f6006b370 legacy=0x7f8f60157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:31.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5e7fc700 1 -- 172.21.15.81:0/680178784 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f600b3e40 con 0x7f8f600b2de0 2024-05-08T23:30:31.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5effd700 1 --2- 172.21.15.81:0/680178784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f601622f0 0x7f8f601626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:31.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5e7fc700 1 --2- 172.21.15.81:0/680178784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600b2de0 0x7f8f600b3260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009850 tx=0x7f8f54009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebb6fd16466b2e47 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:31.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5dffb700 1 -- 172.21.15.81:0/680178784 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54011830 con 0x7f8f600b2de0 2024-05-08T23:30:31.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5dffb700 1 -- 172.21.15.81:0/680178784 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f54011990 con 0x7f8f600b2de0 2024-05-08T23:30:31.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5dffb700 1 -- 172.21.15.81:0/680178784 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54011d40 con 0x7f8f600b2de0 2024-05-08T23:30:31.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5ffff700 1 -- 172.21.15.81:0/680178784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600b2de0 msgr2=0x7f8f600b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:31.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5ffff700 1 --2- 172.21.15.81:0/680178784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600b2de0 0x7f8f600b3260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009850 tx=0x7f8f54009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:31.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5ffff700 1 -- 172.21.15.81:0/680178784 shutdown_connections 2024-05-08T23:30:31.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5ffff700 1 --2- 172.21.15.81:0/680178784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600b2de0 0x7f8f600b3260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:31.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5ffff700 1 --2- 172.21.15.81:0/680178784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f601622f0 0x7f8f601626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:31.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5ffff700 1 -- 172.21.15.81:0/680178784 >> 172.21.15.81:0/680178784 conn(0x7f8f60070960 msgr2=0x7f8f60070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:31.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5ffff700 1 -- 172.21.15.81:0/680178784 shutdown_connections 2024-05-08T23:30:31.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.937+0000 7f8f5ffff700 1 -- 172.21.15.81:0/680178784 wait complete. 2024-05-08T23:30:31.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5ffff700 1 Processor -- start 2024-05-08T23:30:31.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5ffff700 1 -- start start 2024-05-08T23:30:31.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f6006b370 0x7f8f601f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:31.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600b2de0 0x7f8f60146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:31.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f601622f0 0x7f8f600abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:31.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5ffff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60147420 con 0x7f8f6006b370 2024-05-08T23:30:31.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5ffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600ac1d0 con 0x7f8f600b2de0 2024-05-08T23:30:31.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f601440f0 con 0x7f8f601622f0 2024-05-08T23:30:31.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f6006b370 0x7f8f601f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:31.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f6006b370 0x7f8f601f6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36318/0 (socket says 172.21.15.81:36318) 2024-05-08T23:30:31.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5effd700 1 -- 172.21.15.81:0/3031552679 learned_addr learned my addr 172.21.15.81:0/3031552679 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:31.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5f7fe700 1 --2- 172.21.15.81:0/3031552679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f601622f0 0x7f8f600abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:31.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5e7fc700 1 --2- 172.21.15.81:0/3031552679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600b2de0 0x7f8f60146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:31.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5effd700 1 -- 172.21.15.81:0/3031552679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f601622f0 msgr2=0x7f8f600abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:31.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5effd700 1 --2- 172.21.15.81:0/3031552679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f601622f0 0x7f8f600abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:31.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5effd700 1 -- 172.21.15.81:0/3031552679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600b2de0 msgr2=0x7f8f60146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:31.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5effd700 1 --2- 172.21.15.81:0/3031552679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600b2de0 0x7f8f60146ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:31.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5effd700 1 -- 172.21.15.81:0/3031552679 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f500094b0 con 0x7f8f6006b370 2024-05-08T23:30:31.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5effd700 1 --2- 172.21.15.81:0/3031552679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f6006b370 0x7f8f601f6d80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000ebe0 tx=0x7f8f50009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:31.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f477fe700 1 -- 172.21.15.81:0/3031552679 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5000c910 con 0x7f8f6006b370 2024-05-08T23:30:31.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f477fe700 1 -- 172.21.15.81:0/3031552679 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5000ca70 con 0x7f8f6006b370 2024-05-08T23:30:31.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5ffff700 1 -- 172.21.15.81:0/3031552679 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f540094b0 con 0x7f8f6006b370 2024-05-08T23:30:31.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f477fe700 1 -- 172.21.15.81:0/3031552679 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5000ce00 con 0x7f8f6006b370 2024-05-08T23:30:31.948 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5ffff700 1 -- 172.21.15.81:0/3031552679 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f600ac6a0 con 0x7f8f6006b370 2024-05-08T23:30:31.948 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.941+0000 7f8f5ffff700 1 -- 172.21.15.81:0/3031552679 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f600672a0 con 0x7f8f6006b370 2024-05-08T23:30:31.948 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.945+0000 7f8f477fe700 1 -- 172.21.15.81:0/3031552679 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8f50013470 con 0x7f8f6006b370 2024-05-08T23:30:31.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.945+0000 7f8f477fe700 1 --2- 172.21.15.81:0/3031552679 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8f4c066730 0x7f8f4c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:31.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.945+0000 7f8f477fe700 1 -- 172.21.15.81:0/3031552679 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8f500840f0 con 0x7f8f6006b370 2024-05-08T23:30:31.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.945+0000 7f8f5e7fc700 1 --2- 172.21.15.81:0/3031552679 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8f4c066730 0x7f8f4c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:31.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.945+0000 7f8f5e7fc700 1 --2- 172.21.15.81:0/3031552679 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8f4c066730 0x7f8f4c068bf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8f540090b0 tx=0x7f8f54008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:31.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:31.949+0000 7f8f477fe700 1 -- 172.21.15.81:0/3031552679 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8f50051690 con 0x7f8f6006b370 2024-05-08T23:30:32.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.117+0000 7f8f5ffff700 1 -- 172.21.15.81:0/3031552679 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f600aebf0 con 0x7f8f6006b370 2024-05-08T23:30:32.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.121+0000 7f8f477fe700 1 -- 172.21.15.81:0/3031552679 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12382 (secure 0 0 0) 0x7f8f50054cb0 con 0x7f8f6006b370 2024-05-08T23:30:32.125 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:32.125 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":18,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:28.221779+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:30:32.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.121+0000 7f8f5ffff700 1 -- 172.21.15.81:0/3031552679 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8f4c066730 msgr2=0x7f8f4c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:32.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.121+0000 7f8f5ffff700 1 --2- 172.21.15.81:0/3031552679 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8f4c066730 0x7f8f4c068bf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8f540090b0 tx=0x7f8f54008030 comp rx=0 tx=0).stop 2024-05-08T23:30:32.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.121+0000 7f8f5ffff700 1 -- 172.21.15.81:0/3031552679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f6006b370 msgr2=0x7f8f601f6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:32.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.121+0000 7f8f5ffff700 1 --2- 172.21.15.81:0/3031552679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f6006b370 0x7f8f601f6d80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000ebe0 tx=0x7f8f50009ed0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.121+0000 7f8f5ffff700 1 -- 172.21.15.81:0/3031552679 shutdown_connections 2024-05-08T23:30:32.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.121+0000 7f8f5ffff700 1 --2- 172.21.15.81:0/3031552679 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8f4c066730 0x7f8f4c068bf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.121+0000 7f8f5ffff700 1 --2- 172.21.15.81:0/3031552679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f601622f0 0x7f8f600abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.121+0000 7f8f5ffff700 1 --2- 172.21.15.81:0/3031552679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f600b2de0 0x7f8f60146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.121+0000 7f8f5ffff700 1 --2- 172.21.15.81:0/3031552679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f6006b370 0x7f8f601f6d80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.121+0000 7f8f5ffff700 1 -- 172.21.15.81:0/3031552679 >> 172.21.15.81:0/3031552679 conn(0x7f8f60070960 msgr2=0x7f8f6006dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:32.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.121+0000 7f8f5ffff700 1 -- 172.21.15.81:0/3031552679 shutdown_connections 2024-05-08T23:30:32.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.121+0000 7f8f5ffff700 1 -- 172.21.15.81:0/3031552679 wait complete. 2024-05-08T23:30:32.139 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:30:32.140 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:30:32.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfb59e700 1 Processor -- start 2024-05-08T23:30:32.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfb59e700 1 -- start start 2024-05-08T23:30:32.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfb59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1cfc15b860 0x7f1cfc0ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:32.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfb59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1cfc162bc0 0x7f1cfc164fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:32.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfb59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cfc155ad0 con 0x7f1cfc0aed90 2024-05-08T23:30:32.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfb59e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc0cf2d0 con 0x7f1cfc162bc0 2024-05-08T23:30:32.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfb59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc0b3df0 con 0x7f1cfc15b860 2024-05-08T23:30:32.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfad9d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1cfc162bc0 0x7f1cfc164fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:32.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfad9d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1cfc162bc0 0x7f1cfc164fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36334/0 (socket says 172.21.15.81:36334) 2024-05-08T23:30:32.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfad9d700 1 -- 172.21.15.81:0/3611277237 learned_addr learned my addr 172.21.15.81:0/3611277237 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:32.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfa59c700 1 --2- 172.21.15.81:0/3611277237 >> v2:172.21.15.117:3300/0 conn(0x7f1cfc15b860 0x7f1cfc0ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:32.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfad9d700 1 -- 172.21.15.81:0/3611277237 >> v2:172.21.15.117:3300/0 conn(0x7f1cfc15b860 msgr2=0x7f1cfc0ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:32.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfad9d700 1 --2- 172.21.15.81:0/3611277237 >> v2:172.21.15.117:3300/0 conn(0x7f1cfc15b860 0x7f1cfc0ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfad9d700 1 -- 172.21.15.81:0/3611277237 >> v1:172.21.15.117:6789/0 conn(0x7f1cfc0aed90 legacy=0x7f1cfc15bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:32.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfad9d700 1 -- 172.21.15.81:0/3611277237 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc15d780 con 0x7f1cfc162bc0 2024-05-08T23:30:32.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cfad9d700 1 --2- 172.21.15.81:0/3611277237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cfc162bc0 0x7f1cfc164fb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc070590 tx=0x7f1cf400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fc044199076dcc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:32.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cf959a700 1 -- 172.21.15.81:0/3611277237 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400e070 con 0x7f1cfc162bc0 2024-05-08T23:30:32.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cf959a700 1 -- 172.21.15.81:0/3611277237 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf400b910 con 0x7f1cfc162bc0 2024-05-08T23:30:32.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.257+0000 7f1cf959a700 1 -- 172.21.15.81:0/3611277237 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400bcc0 con 0x7f1cfc162bc0 2024-05-08T23:30:32.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.261+0000 7f1cfb59e700 1 -- 172.21.15.81:0/3611277237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cfc162bc0 msgr2=0x7f1cfc164fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:32.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.261+0000 7f1cfb59e700 1 --2- 172.21.15.81:0/3611277237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cfc162bc0 0x7f1cfc164fb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc070590 tx=0x7f1cf400b330 comp rx=0 tx=0).stop 2024-05-08T23:30:32.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.261+0000 7f1cfb59e700 1 -- 172.21.15.81:0/3611277237 shutdown_connections 2024-05-08T23:30:32.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.261+0000 7f1cfb59e700 1 --2- 172.21.15.81:0/3611277237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cfc162bc0 0x7f1cfc164fb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.261+0000 7f1cfb59e700 1 --2- 172.21.15.81:0/3611277237 >> v2:172.21.15.117:3300/0 conn(0x7f1cfc15b860 0x7f1cfc0ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.261+0000 7f1cfb59e700 1 -- 172.21.15.81:0/3611277237 >> 172.21.15.81:0/3611277237 conn(0x7f1cfc0717b0 msgr2=0x7f1cfc06a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:32.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.261+0000 7f1cfb59e700 1 -- 172.21.15.81:0/3611277237 shutdown_connections 2024-05-08T23:30:32.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.261+0000 7f1cfb59e700 1 -- 172.21.15.81:0/3611277237 wait complete. 2024-05-08T23:30:32.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.261+0000 7f1cfb59e700 1 Processor -- start 2024-05-08T23:30:32.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.261+0000 7f1cfb59e700 1 -- start start 2024-05-08T23:30:32.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cfb59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cfc0aed90 0x7f1cfc140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:32.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cfb59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cfc15b860 0x7f1cfc154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:32.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cfb59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cfc162bc0 0x7f1cfc154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:32.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cfb59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc14b2b0 con 0x7f1cfc15b860 2024-05-08T23:30:32.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cfb59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc14ba70 con 0x7f1cfc162bc0 2024-05-08T23:30:32.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cfb59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc14b4f0 con 0x7f1cfc0aed90 2024-05-08T23:30:32.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cf9d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cfc15b860 0x7f1cfc154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:32.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cf9d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cfc15b860 0x7f1cfc154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36350/0 (socket says 172.21.15.81:36350) 2024-05-08T23:30:32.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cf9d9b700 1 -- 172.21.15.81:0/1905496232 learned_addr learned my addr 172.21.15.81:0/1905496232 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:32.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cfa59c700 1 --2- 172.21.15.81:0/1905496232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cfc0aed90 0x7f1cfc140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:32.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cfad9d700 1 --2- 172.21.15.81:0/1905496232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cfc162bc0 0x7f1cfc154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:32.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cf9d9b700 1 -- 172.21.15.81:0/1905496232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cfc0aed90 msgr2=0x7f1cfc140980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:32.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cf9d9b700 1 --2- 172.21.15.81:0/1905496232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cfc0aed90 0x7f1cfc140980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cf9d9b700 1 -- 172.21.15.81:0/1905496232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cfc162bc0 msgr2=0x7f1cfc154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:32.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cf9d9b700 1 --2- 172.21.15.81:0/1905496232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cfc162bc0 0x7f1cfc154c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cf9d9b700 1 -- 172.21.15.81:0/1905496232 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf4009b30 con 0x7f1cfc15b860 2024-05-08T23:30:32.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cfad9d700 1 --2- 172.21.15.81:0/1905496232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cfc162bc0 0x7f1cfc154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:32.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cf9d9b700 1 --2- 172.21.15.81:0/1905496232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cfc15b860 0x7f1cfc154720 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000c4d0 tx=0x7f1cf000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:32.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1ceaffd700 1 -- 172.21.15.81:0/1905496232 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00052d0 con 0x7f1cfc15b860 2024-05-08T23:30:32.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1ceaffd700 1 -- 172.21.15.81:0/1905496232 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf0005430 con 0x7f1cfc15b860 2024-05-08T23:30:32.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cfb59e700 1 -- 172.21.15.81:0/1905496232 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cfc14c730 con 0x7f1cfc15b860 2024-05-08T23:30:32.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1ceaffd700 1 -- 172.21.15.81:0/1905496232 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00055a0 con 0x7f1cfc15b860 2024-05-08T23:30:32.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cfb59e700 1 -- 172.21.15.81:0/1905496232 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cfc14c210 con 0x7f1cfc15b860 2024-05-08T23:30:32.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.265+0000 7f1cfb59e700 1 -- 172.21.15.81:0/1905496232 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cfc0672a0 con 0x7f1cfc15b860 2024-05-08T23:30:32.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.269+0000 7f1ceaffd700 1 -- 172.21.15.81:0/1905496232 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1cf00038c0 con 0x7f1cfc15b860 2024-05-08T23:30:32.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.269+0000 7f1ceaffd700 1 --2- 172.21.15.81:0/1905496232 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1cdc066730 0x7f1cdc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:32.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.269+0000 7f1ceaffd700 1 -- 172.21.15.81:0/1905496232 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1cf0082ff0 con 0x7f1cfc15b860 2024-05-08T23:30:32.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.269+0000 7f1cfa59c700 1 --2- 172.21.15.81:0/1905496232 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1cdc066730 0x7f1cdc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:32.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.269+0000 7f1cfa59c700 1 --2- 172.21.15.81:0/1905496232 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1cdc066730 0x7f1cdc068bf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009bf0 tx=0x7f1cec009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:32.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.273+0000 7f1ceaffd700 1 -- 172.21.15.81:0/1905496232 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1cf0050740 con 0x7f1cfc15b860 2024-05-08T23:30:32.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.445+0000 7f1cfb59e700 1 -- 172.21.15.81:0/1905496232 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1cfc155ad0 con 0x7f1cfc15b860 2024-05-08T23:30:32.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.445+0000 7f1ceaffd700 1 -- 172.21.15.81:0/1905496232 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1cf0017020 con 0x7f1cfc15b860 2024-05-08T23:30:32.450 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:30:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.449+0000 7f1cfb59e700 1 -- 172.21.15.81:0/1905496232 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1cdc066730 msgr2=0x7f1cdc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.449+0000 7f1cfb59e700 1 --2- 172.21.15.81:0/1905496232 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1cdc066730 0x7f1cdc068bf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009bf0 tx=0x7f1cec009240 comp rx=0 tx=0).stop 2024-05-08T23:30:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.449+0000 7f1cfb59e700 1 -- 172.21.15.81:0/1905496232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cfc15b860 msgr2=0x7f1cfc154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.449+0000 7f1cfb59e700 1 --2- 172.21.15.81:0/1905496232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cfc15b860 0x7f1cfc154720 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000c4d0 tx=0x7f1cf000c7e0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.449+0000 7f1cfb59e700 1 -- 172.21.15.81:0/1905496232 shutdown_connections 2024-05-08T23:30:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.449+0000 7f1cfb59e700 1 --2- 172.21.15.81:0/1905496232 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1cdc066730 0x7f1cdc068bf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.449+0000 7f1cfb59e700 1 --2- 172.21.15.81:0/1905496232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cfc162bc0 0x7f1cfc154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.449+0000 7f1cfb59e700 1 --2- 172.21.15.81:0/1905496232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cfc15b860 0x7f1cfc154720 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.449+0000 7f1cfb59e700 1 --2- 172.21.15.81:0/1905496232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cfc0aed90 0x7f1cfc140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.449+0000 7f1cfb59e700 1 -- 172.21.15.81:0/1905496232 >> 172.21.15.81:0/1905496232 conn(0x7f1cfc0717b0 msgr2=0x7f1cfc0af430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:32.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.449+0000 7f1cfb59e700 1 -- 172.21.15.81:0/1905496232 shutdown_connections 2024-05-08T23:30:32.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.449+0000 7f1cfb59e700 1 -- 172.21.15.81:0/1905496232 wait complete. 2024-05-08T23:30:32.464 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:30:32.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.569+0000 7fe7bc878700 1 Processor -- start 2024-05-08T23:30:32.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.569+0000 7fe7bc878700 1 -- start start 2024-05-08T23:30:32.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7bc878700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe7b81596a0 0x7fe7b80b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:32.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7bc878700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b80b3590 con 0x7fe7b815a190 2024-05-08T23:30:32.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7bc878700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b80b24f0 con 0x7fe7b80b4e50 2024-05-08T23:30:32.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7bc878700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b80b27d0 con 0x7fe7b81596a0 2024-05-08T23:30:32.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b759e700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe7b815a190 0x7fe7b80709a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52258/0 (socket says 172.21.15.81:52258) 2024-05-08T23:30:32.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b759e700 1 -- 172.21.15.81:0/1328167193 learned_addr learned my addr 172.21.15.81:0/1328167193 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:32.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b6d9d700 1 --2- 172.21.15.81:0/1328167193 >> v2:172.21.15.117:3300/0 conn(0x7fe7b81596a0 0x7fe7b80b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:32.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 -- 172.21.15.81:0/1328167193 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736268792 0 0) 0x7fe7b80b3590 con 0x7fe7b815a190 2024-05-08T23:30:32.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 -- 172.21.15.81:0/1328167193 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a40057e0 con 0x7fe7b815a190 2024-05-08T23:30:32.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 -- 172.21.15.81:0/1328167193 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1492598042 0 0) 0x7fe7a40057e0 con 0x7fe7b815a190 2024-05-08T23:30:32.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 -- 172.21.15.81:0/1328167193 >> v2:172.21.15.117:3300/0 conn(0x7fe7b81596a0 msgr2=0x7fe7b80b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:32.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 --2- 172.21.15.81:0/1328167193 >> v2:172.21.15.117:3300/0 conn(0x7fe7b81596a0 0x7fe7b80b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 -- 172.21.15.81:0/1328167193 >> v1:172.21.15.117:6789/0 conn(0x7fe7b80b4e50 legacy=0x7fe7b8159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:32.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 -- 172.21.15.81:0/1328167193 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b80b2ab0 con 0x7fe7b815a190 2024-05-08T23:30:32.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 -- 172.21.15.81:0/1328167193 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7fe7b00019f0 con 0x7fe7b815a190 2024-05-08T23:30:32.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 -- 172.21.15.81:0/1328167193 >> v1:172.21.15.81:6789/0 conn(0x7fe7b815a190 legacy=0x7fe7b80709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:32.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 --2- 172.21.15.81:0/1328167193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7a4005c30 0x7fe7a4006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:32.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 --2- 172.21.15.81:0/1328167193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a4006a80 0x7fe7a4006f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:32.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 --2- 172.21.15.81:0/1328167193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a4007440 0x7fe7a400dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:32.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 -- 172.21.15.81:0/1328167193 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b80b27d0 con 0x7fe7a4005c30 2024-05-08T23:30:32.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 -- 172.21.15.81:0/1328167193 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a400e2e0 con 0x7fe7a4006a80 2024-05-08T23:30:32.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 -- 172.21.15.81:0/1328167193 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a400e4a0 con 0x7fe7a4007440 2024-05-08T23:30:32.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b6d9d700 1 --2- 172.21.15.81:0/1328167193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7a4005c30 0x7fe7a4006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:32.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b659c700 1 --2- 172.21.15.81:0/1328167193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a4006a80 0x7fe7a4006f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:32.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b759e700 1 --2- 172.21.15.81:0/1328167193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a4007440 0x7fe7a400dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:32.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b6d9d700 1 -- 172.21.15.81:0/1328167193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a4007440 msgr2=0x7fe7a400dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:32.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b6d9d700 1 --2- 172.21.15.81:0/1328167193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a4007440 0x7fe7a400dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b6d9d700 1 -- 172.21.15.81:0/1328167193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a4006a80 msgr2=0x7fe7a4006f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:32.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b6d9d700 1 --2- 172.21.15.81:0/1328167193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a4006a80 0x7fe7a4006f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b6d9d700 1 -- 172.21.15.81:0/1328167193 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a40057e0 con 0x7fe7a4005c30 2024-05-08T23:30:32.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b759e700 1 --2- 172.21.15.81:0/1328167193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a4007440 0x7fe7a400dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:32.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b759e700 1 -- 172.21.15.81:0/1328167193 reap_dead start 2024-05-08T23:30:32.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b6d9d700 1 --2- 172.21.15.81:0/1328167193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7a4005c30 0x7fe7a4006050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009b70 tx=0x7fe7a4007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe46339fecb9a335 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:32.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 -- 172.21.15.81:0/1328167193 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800e6c0 con 0x7fe7a4005c30 2024-05-08T23:30:32.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 -- 172.21.15.81:0/1328167193 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a800e820 con 0x7fe7a4005c30 2024-05-08T23:30:32.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7b5d9b700 1 -- 172.21.15.81:0/1328167193 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800eb90 con 0x7fe7a4005c30 2024-05-08T23:30:32.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7bc878700 1 -- 172.21.15.81:0/1328167193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7a4005c30 msgr2=0x7fe7a4006050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:32.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7bc878700 1 --2- 172.21.15.81:0/1328167193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7a4005c30 0x7fe7a4006050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009b70 tx=0x7fe7a4007ab0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7bc878700 1 -- 172.21.15.81:0/1328167193 shutdown_connections 2024-05-08T23:30:32.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7bc878700 1 --2- 172.21.15.81:0/1328167193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7a4005c30 0x7fe7a4006050 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7bc878700 1 -- 172.21.15.81:0/1328167193 >> 172.21.15.81:0/1328167193 conn(0x7fe7b806c330 msgr2=0x7fe7b806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:32.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7bc878700 1 -- 172.21.15.81:0/1328167193 shutdown_connections 2024-05-08T23:30:32.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.573+0000 7fe7bc878700 1 -- 172.21.15.81:0/1328167193 wait complete. 2024-05-08T23:30:32.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.577+0000 7fe7bc878700 1 Processor -- start 2024-05-08T23:30:32.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.577+0000 7fe7bc878700 1 -- start start 2024-05-08T23:30:32.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.577+0000 7fe7bc878700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a4005c30 0x7fe7b81556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:32.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.577+0000 7fe7bc878700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b80b0120 0x7fe7b8155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:32.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.577+0000 7fe7bc878700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b81497c0 0x7fe7b814a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:32.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.577+0000 7fe7bc878700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b8156140 con 0x7fe7b81497c0 2024-05-08T23:30:32.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.577+0000 7fe7bc878700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b80af930 con 0x7fe7a4005c30 2024-05-08T23:30:32.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.577+0000 7fe7bc878700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b80afc10 con 0x7fe7b80b0120 2024-05-08T23:30:32.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.577+0000 7fe7b759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b81497c0 0x7fe7b814a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:32.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.577+0000 7fe7b759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b81497c0 0x7fe7b814a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36372/0 (socket says 172.21.15.81:36372) 2024-05-08T23:30:32.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.577+0000 7fe7b759e700 1 -- 172.21.15.81:0/610802840 learned_addr learned my addr 172.21.15.81:0/610802840 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:32.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.577+0000 7fe7b659c700 1 --2- 172.21.15.81:0/610802840 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b80b0120 0x7fe7b8155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:32.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.577+0000 7fe7b6d9d700 1 --2- 172.21.15.81:0/610802840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a4005c30 0x7fe7b81556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:32.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.581+0000 7fe7b759e700 1 -- 172.21.15.81:0/610802840 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b80b0120 msgr2=0x7fe7b8155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:32.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.581+0000 7fe7b759e700 1 --2- 172.21.15.81:0/610802840 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b80b0120 0x7fe7b8155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.581+0000 7fe7b759e700 1 -- 172.21.15.81:0/610802840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a4005c30 msgr2=0x7fe7b81556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:32.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.581+0000 7fe7b759e700 1 --2- 172.21.15.81:0/610802840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a4005c30 0x7fe7b81556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.581+0000 7fe7b759e700 1 -- 172.21.15.81:0/610802840 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b814bc00 con 0x7fe7b81497c0 2024-05-08T23:30:32.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.581+0000 7fe7b759e700 1 --2- 172.21.15.81:0/610802840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b81497c0 0x7fe7b814a110 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe7b806ec80 tx=0x7fe7b0007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:32.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.581+0000 7fe7af7fe700 1 -- 172.21.15.81:0/610802840 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b000e870 con 0x7fe7b81497c0 2024-05-08T23:30:32.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.581+0000 7fe7af7fe700 1 -- 172.21.15.81:0/610802840 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b000e9d0 con 0x7fe7b81497c0 2024-05-08T23:30:32.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.581+0000 7fe7bc878700 1 -- 172.21.15.81:0/610802840 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b814bee0 con 0x7fe7b81497c0 2024-05-08T23:30:32.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.581+0000 7fe7af7fe700 1 -- 172.21.15.81:0/610802840 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b000ed80 con 0x7fe7b81497c0 2024-05-08T23:30:32.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.581+0000 7fe7bc878700 1 -- 172.21.15.81:0/610802840 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b814b2e0 con 0x7fe7b81497c0 2024-05-08T23:30:32.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.581+0000 7fe7bc878700 1 -- 172.21.15.81:0/610802840 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b80af790 con 0x7fe7b81497c0 2024-05-08T23:30:32.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.581+0000 7fe7af7fe700 1 -- 172.21.15.81:0/610802840 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe7b001d460 con 0x7fe7b81497c0 2024-05-08T23:30:32.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.585+0000 7fe7af7fe700 1 --2- 172.21.15.81:0/610802840 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe798066760 0x7fe798068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:32.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.585+0000 7fe7af7fe700 1 -- 172.21.15.81:0/610802840 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe7b0083fe0 con 0x7fe7b81497c0 2024-05-08T23:30:32.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.585+0000 7fe7b6d9d700 1 --2- 172.21.15.81:0/610802840 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe798066760 0x7fe798068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:32.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.585+0000 7fe7b6d9d700 1 --2- 172.21.15.81:0/610802840 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe798066760 0x7fe798068c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80090b0 tx=0x7fe7a801f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:32.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.589+0000 7fe7af7fe700 1 -- 172.21.15.81:0/610802840 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe7b80af790 con 0x7fe7b81497c0 2024-05-08T23:30:32.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.797+0000 7fe7bc878700 1 -- 172.21.15.81:0/610802840 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7b80af790 con 0x7fe7b81497c0 2024-05-08T23:30:32.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.797+0000 7fe7af7fe700 1 -- 172.21.15.81:0/610802840 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1256 (secure 0 0 0) 0x7fe7b80af790 con 0x7fe7b81497c0 2024-05-08T23:30:32.803 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:32.803 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1475726755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1475726755},{"type":"v1","addr":"172.21.15.81:6835","nonce":1475726755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T23:30:32.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.801+0000 7fe7bc878700 1 -- 172.21.15.81:0/610802840 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe798066760 msgr2=0x7fe798068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:32.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.801+0000 7fe7bc878700 1 --2- 172.21.15.81:0/610802840 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe798066760 0x7fe798068c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80090b0 tx=0x7fe7a801f040 comp rx=0 tx=0).stop 2024-05-08T23:30:32.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.801+0000 7fe7bc878700 1 -- 172.21.15.81:0/610802840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b81497c0 msgr2=0x7fe7b814a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:32.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.801+0000 7fe7bc878700 1 --2- 172.21.15.81:0/610802840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b81497c0 0x7fe7b814a110 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe7b806ec80 tx=0x7fe7b0007cb0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.801+0000 7fe7bc878700 1 -- 172.21.15.81:0/610802840 shutdown_connections 2024-05-08T23:30:32.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.801+0000 7fe7bc878700 1 --2- 172.21.15.81:0/610802840 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe798066760 0x7fe798068c20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.801+0000 7fe7bc878700 1 --2- 172.21.15.81:0/610802840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b81497c0 0x7fe7b814a110 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.801+0000 7fe7bc878700 1 --2- 172.21.15.81:0/610802840 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b80b0120 0x7fe7b8155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.801+0000 7fe7bc878700 1 --2- 172.21.15.81:0/610802840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a4005c30 0x7fe7b81556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:32.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.801+0000 7fe7bc878700 1 -- 172.21.15.81:0/610802840 >> 172.21.15.81:0/610802840 conn(0x7fe7b806c330 msgr2=0x7fe7b8070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:32.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.801+0000 7fe7bc878700 1 -- 172.21.15.81:0/610802840 shutdown_connections 2024-05-08T23:30:32.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:32.801+0000 7fe7bc878700 1 -- 172.21.15.81:0/610802840 wait complete. 2024-05-08T23:30:32.807 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 2 2024-05-08T23:30:34.818 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:30:34.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.933+0000 7fa333fef700 1 Processor -- start 2024-05-08T23:30:34.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.933+0000 7fa333fef700 1 -- start start 2024-05-08T23:30:34.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa333fef700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa32c06ca60 0x7fa32c15a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:34.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa333fef700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa32c0b3380 con 0x7fa32c13cda0 2024-05-08T23:30:34.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa333fef700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa32c0b3660 con 0x7fa32c13d180 2024-05-08T23:30:34.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa333fef700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32c15b6a0 con 0x7fa32c06ca60 2024-05-08T23:30:34.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa331d8b700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa32c13cda0 0x7fa32c06c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52262/0 (socket says 172.21.15.81:52262) 2024-05-08T23:30:34.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa33158a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa32c06ca60 0x7fa32c15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:34.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa331d8b700 1 -- 172.21.15.81:0/1718918990 learned_addr learned my addr 172.21.15.81:0/1718918990 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:34.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa330d89700 1 -- 172.21.15.81:0/1718918990 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2549349587 0 0) 0x7fa32c0b3380 con 0x7fa32c13cda0 2024-05-08T23:30:34.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa330d89700 1 -- 172.21.15.81:0/1718918990 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3180035d0 con 0x7fa32c13cda0 2024-05-08T23:30:34.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa33158a700 1 -- 172.21.15.81:0/1718918990 >> v1:172.21.15.117:6789/0 conn(0x7fa32c13d180 legacy=0x7fa32c15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:34.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa33158a700 1 -- 172.21.15.81:0/1718918990 >> v1:172.21.15.81:6789/0 conn(0x7fa32c13cda0 legacy=0x7fa32c06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:34.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa33158a700 1 -- 172.21.15.81:0/1718918990 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c0b3940 con 0x7fa32c06ca60 2024-05-08T23:30:34.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa33158a700 1 --2- 172.21.15.81:0/1718918990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa32c06ca60 0x7fa32c15a950 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa320009850 tx=0x7fa320009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f8a3713585cfb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:34.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa330d89700 1 -- 172.21.15.81:0/1718918990 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320011830 con 0x7fa32c06ca60 2024-05-08T23:30:34.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa330d89700 1 -- 172.21.15.81:0/1718918990 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa320011990 con 0x7fa32c06ca60 2024-05-08T23:30:34.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa330d89700 1 -- 172.21.15.81:0/1718918990 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320011d40 con 0x7fa32c06ca60 2024-05-08T23:30:34.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa333fef700 1 -- 172.21.15.81:0/1718918990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa32c06ca60 msgr2=0x7fa32c15a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:34.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa333fef700 1 --2- 172.21.15.81:0/1718918990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa32c06ca60 0x7fa32c15a950 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa320009850 tx=0x7fa320009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:34.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa333fef700 1 -- 172.21.15.81:0/1718918990 shutdown_connections 2024-05-08T23:30:34.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa333fef700 1 --2- 172.21.15.81:0/1718918990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa32c06ca60 0x7fa32c15a950 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:34.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa333fef700 1 -- 172.21.15.81:0/1718918990 >> 172.21.15.81:0/1718918990 conn(0x7fa32c06a360 msgr2=0x7fa32c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:34.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa333fef700 1 -- 172.21.15.81:0/1718918990 shutdown_connections 2024-05-08T23:30:34.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.937+0000 7fa333fef700 1 -- 172.21.15.81:0/1718918990 wait complete. 2024-05-08T23:30:34.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa333fef700 1 Processor -- start 2024-05-08T23:30:34.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa333fef700 1 -- start start 2024-05-08T23:30:34.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa333fef700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa32c06ca60 0x7fa32c0ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:34.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa333fef700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa32c13cda0 0x7fa32c0adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:34.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa333fef700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa32c13d180 0x7fa32c13b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:34.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa333fef700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c13a0f0 con 0x7fa32c13cda0 2024-05-08T23:30:34.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa333fef700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c1f32c0 con 0x7fa32c06ca60 2024-05-08T23:30:34.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa333fef700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c1f3560 con 0x7fa32c13d180 2024-05-08T23:30:34.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa33158a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa32c13cda0 0x7fa32c0adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:34.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa33158a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa32c13cda0 0x7fa32c0adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36388/0 (socket says 172.21.15.81:36388) 2024-05-08T23:30:34.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa33158a700 1 -- 172.21.15.81:0/1476688328 learned_addr learned my addr 172.21.15.81:0/1476688328 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:34.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa33258c700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa32c13d180 0x7fa32c13b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:34.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa331d8b700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa32c06ca60 0x7fa32c0ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:34.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa33158a700 1 -- 172.21.15.81:0/1476688328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa32c13d180 msgr2=0x7fa32c13b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:34.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa33158a700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa32c13d180 0x7fa32c13b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:34.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa33158a700 1 -- 172.21.15.81:0/1476688328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa32c06ca60 msgr2=0x7fa32c0ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:34.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa33158a700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa32c06ca60 0x7fa32c0ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:34.948 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa33158a700 1 -- 172.21.15.81:0/1476688328 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3200094b0 con 0x7fa32c13cda0 2024-05-08T23:30:34.948 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa33258c700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa32c13d180 0x7fa32c13b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:34.948 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa331d8b700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa32c06ca60 0x7fa32c0ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:34.948 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa33158a700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa32c13cda0 0x7fa32c0adf20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa320000c00 tx=0x7fa320004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:34.948 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa31e7fc700 1 -- 172.21.15.81:0/1476688328 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32001aac0 con 0x7fa32c13cda0 2024-05-08T23:30:34.948 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa31e7fc700 1 -- 172.21.15.81:0/1476688328 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32001f070 con 0x7fa32c13cda0 2024-05-08T23:30:34.948 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa333fef700 1 -- 172.21.15.81:0/1476688328 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32c1f3840 con 0x7fa32c13cda0 2024-05-08T23:30:34.948 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa31e7fc700 1 -- 172.21.15.81:0/1476688328 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32001ae60 con 0x7fa32c13cda0 2024-05-08T23:30:34.948 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.941+0000 7fa333fef700 1 -- 172.21.15.81:0/1476688328 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c1f3e00 con 0x7fa32c13cda0 2024-05-08T23:30:34.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.945+0000 7fa333fef700 1 -- 172.21.15.81:0/1476688328 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32c1f3ff0 con 0x7fa32c13cda0 2024-05-08T23:30:34.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.945+0000 7fa31e7fc700 1 -- 172.21.15.81:0/1476688328 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa32001b430 con 0x7fa32c13cda0 2024-05-08T23:30:34.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.945+0000 7fa31e7fc700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa310066730 0x7fa310068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:34.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.945+0000 7fa31e7fc700 1 -- 172.21.15.81:0/1476688328 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa320084fc0 con 0x7fa32c13cda0 2024-05-08T23:30:34.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.945+0000 7fa331d8b700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa310066730 0x7fa310068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:34.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.945+0000 7fa331d8b700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa310066730 0x7fa310068bf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa3280011e0 tx=0x7fa328009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:34.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:34.953+0000 7fa31e7fc700 1 -- 172.21.15.81:0/1476688328 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa320052670 con 0x7fa32c13cda0 2024-05-08T23:30:35.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.161+0000 7fa333fef700 1 -- 172.21.15.81:0/1476688328 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa32c0684b0 con 0x7fa32c13cda0 2024-05-08T23:30:35.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.161+0000 7fa31e7fc700 1 -- 172.21.15.81:0/1476688328 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7fa320015020 con 0x7fa32c13cda0 2024-05-08T23:30:35.165 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:35.165 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1475726755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1475726755},{"type":"v1","addr":"172.21.15.81:6835","nonce":1475726755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3535820540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3535820540},{"type":"v1","addr":"172.21.15.81:6837","nonce":3535820540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1531019955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1531019955},{"type":"v1","addr":"172.21.15.117:6833","nonce":1531019955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/491911303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":491911303},{"type":"v1","addr":"172.21.15.117:6835","nonce":491911303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-05-08T23:30:35.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.161+0000 7fa333fef700 1 -- 172.21.15.81:0/1476688328 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa310066730 msgr2=0x7fa310068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.161+0000 7fa333fef700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa310066730 0x7fa310068bf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa3280011e0 tx=0x7fa328009040 comp rx=0 tx=0).stop 2024-05-08T23:30:35.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.161+0000 7fa333fef700 1 -- 172.21.15.81:0/1476688328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa32c13cda0 msgr2=0x7fa32c0adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.161+0000 7fa333fef700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa32c13cda0 0x7fa32c0adf20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa320000c00 tx=0x7fa320004770 comp rx=0 tx=0).stop 2024-05-08T23:30:35.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.161+0000 7fa333fef700 1 -- 172.21.15.81:0/1476688328 shutdown_connections 2024-05-08T23:30:35.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.161+0000 7fa333fef700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa310066730 0x7fa310068bf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.161+0000 7fa333fef700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa32c13d180 0x7fa32c13b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.161+0000 7fa333fef700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa32c13cda0 0x7fa32c0adf20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.161+0000 7fa333fef700 1 --2- 172.21.15.81:0/1476688328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa32c06ca60 0x7fa32c0ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.161+0000 7fa333fef700 1 -- 172.21.15.81:0/1476688328 >> 172.21.15.81:0/1476688328 conn(0x7fa32c06a360 msgr2=0x7fa32c15d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:35.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.161+0000 7fa333fef700 1 -- 172.21.15.81:0/1476688328 shutdown_connections 2024-05-08T23:30:35.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.161+0000 7fa333fef700 1 -- 172.21.15.81:0/1476688328 wait complete. 2024-05-08T23:30:35.171 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 3 2024-05-08T23:30:35.180 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:30:35.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.273+0000 7f84ad77c700 1 Processor -- start 2024-05-08T23:30:35.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.273+0000 7f84ad77c700 1 -- start start 2024-05-08T23:30:35.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.273+0000 7f84ad77c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84a815aed0 0x7f84a80cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.273+0000 7f84ad77c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f84a80cb970 0x7f84a80b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.273+0000 7f84ad77c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a80cd470 con 0x7f84a80b2e40 2024-05-08T23:30:35.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.273+0000 7f84ad77c700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a80ced50 con 0x7f84a80cb970 2024-05-08T23:30:35.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.273+0000 7f84ad77c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a80cceb0 con 0x7f84a815aed0 2024-05-08T23:30:35.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84a67fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f84a80cb970 0x7f84a80b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84a6ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84a815aed0 0x7f84a80cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84a77fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f84a80b2e40 0x7f84a806c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:60854/0 (socket says 172.21.15.81:60854) 2024-05-08T23:30:35.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84a77fe700 1 -- 172.21.15.81:0/3122618024 learned_addr learned my addr 172.21.15.81:0/3122618024 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:35.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84a6ffd700 1 -- 172.21.15.81:0/3122618024 >> v2:172.21.15.117:3300/0 conn(0x7f84a80cb970 msgr2=0x7f84a80b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:35.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84a6ffd700 1 --2- 172.21.15.81:0/3122618024 >> v2:172.21.15.117:3300/0 conn(0x7f84a80cb970 0x7f84a80b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84a6ffd700 1 -- 172.21.15.81:0/3122618024 >> v1:172.21.15.117:6789/0 conn(0x7f84a80b2e40 legacy=0x7f84a806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84a5ffb700 1 -- 172.21.15.81:0/3122618024 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064620543 0 0) 0x7f84a80cd470 con 0x7f84a80b2e40 2024-05-08T23:30:35.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84a6ffd700 1 -- 172.21.15.81:0/3122618024 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a80cd190 con 0x7f84a815aed0 2024-05-08T23:30:35.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84a67fc700 1 --2- 172.21.15.81:0/3122618024 >> v2:172.21.15.117:3300/0 conn(0x7f84a80cb970 0x7f84a80b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:35.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84a6ffd700 1 --2- 172.21.15.81:0/3122618024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84a815aed0 0x7f84a80cc7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8498009850 tx=0x7f8498009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2b4584f91558db0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:35.282 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84a5ffb700 1 -- 172.21.15.81:0/3122618024 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8498011830 con 0x7f84a815aed0 2024-05-08T23:30:35.282 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84a5ffb700 1 -- 172.21.15.81:0/3122618024 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8498011990 con 0x7f84a815aed0 2024-05-08T23:30:35.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84a5ffb700 1 -- 172.21.15.81:0/3122618024 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8498011d40 con 0x7f84a815aed0 2024-05-08T23:30:35.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84ad77c700 1 -- 172.21.15.81:0/3122618024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84a815aed0 msgr2=0x7f84a80cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84ad77c700 1 --2- 172.21.15.81:0/3122618024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84a815aed0 0x7f84a80cc7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8498009850 tx=0x7f8498009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:35.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84ad77c700 1 -- 172.21.15.81:0/3122618024 shutdown_connections 2024-05-08T23:30:35.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84ad77c700 1 --2- 172.21.15.81:0/3122618024 >> v2:172.21.15.117:3300/0 conn(0x7f84a80cb970 0x7f84a80b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84ad77c700 1 --2- 172.21.15.81:0/3122618024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84a815aed0 0x7f84a80cc7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84ad77c700 1 -- 172.21.15.81:0/3122618024 >> 172.21.15.81:0/3122618024 conn(0x7f84a806a360 msgr2=0x7f84a806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:35.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84ad77c700 1 -- 172.21.15.81:0/3122618024 shutdown_connections 2024-05-08T23:30:35.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.277+0000 7f84ad77c700 1 -- 172.21.15.81:0/3122618024 wait complete. 2024-05-08T23:30:35.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84ad77c700 1 Processor -- start 2024-05-08T23:30:35.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84ad77c700 1 -- start start 2024-05-08T23:30:35.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84ad77c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84a80b2e40 0x7f84a81f0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84ad77c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84a80cb970 0x7f84a814ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84ad77c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84a815aed0 0x7f84a814b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84ad77c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a813d0c0 con 0x7f84a80b2e40 2024-05-08T23:30:35.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84ad77c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a814b780 con 0x7f84a815aed0 2024-05-08T23:30:35.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84ad77c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a814ba60 con 0x7f84a80cb970 2024-05-08T23:30:35.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84a6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84a80b2e40 0x7f84a81f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.287 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84a6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84a80b2e40 0x7f84a81f0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36390/0 (socket says 172.21.15.81:36390) 2024-05-08T23:30:35.287 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84a6ffd700 1 -- 172.21.15.81:0/3399264897 learned_addr learned my addr 172.21.15.81:0/3399264897 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:35.287 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84a67fc700 1 --2- 172.21.15.81:0/3399264897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84a80cb970 0x7f84a814ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.287 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84a77fe700 1 --2- 172.21.15.81:0/3399264897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84a815aed0 0x7f84a814b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.288 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84a6ffd700 1 -- 172.21.15.81:0/3399264897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84a80cb970 msgr2=0x7f84a814ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.288 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84a6ffd700 1 --2- 172.21.15.81:0/3399264897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84a80cb970 0x7f84a814ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.288 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84a6ffd700 1 -- 172.21.15.81:0/3399264897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84a815aed0 msgr2=0x7f84a814b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.288 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84a6ffd700 1 --2- 172.21.15.81:0/3399264897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84a815aed0 0x7f84a814b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.288 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84a6ffd700 1 -- 172.21.15.81:0/3399264897 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849c0094b0 con 0x7f84a80b2e40 2024-05-08T23:30:35.288 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84a6ffd700 1 --2- 172.21.15.81:0/3399264897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84a80b2e40 0x7f84a81f0660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8498009ba0 tx=0x7f8498004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:35.288 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f848f7fe700 1 -- 172.21.15.81:0/3399264897 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8498004550 con 0x7f84a80b2e40 2024-05-08T23:30:35.288 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f848f7fe700 1 -- 172.21.15.81:0/3399264897 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849801e070 con 0x7f84a80b2e40 2024-05-08T23:30:35.289 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84ad77c700 1 -- 172.21.15.81:0/3399264897 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84980094b0 con 0x7f84a80b2e40 2024-05-08T23:30:35.289 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f848f7fe700 1 -- 172.21.15.81:0/3399264897 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8498003900 con 0x7f84a80b2e40 2024-05-08T23:30:35.289 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.281+0000 7f84ad77c700 1 -- 172.21.15.81:0/3399264897 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a814c020 con 0x7f84a80b2e40 2024-05-08T23:30:35.289 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.285+0000 7f84ad77c700 1 -- 172.21.15.81:0/3399264897 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a80cbf90 con 0x7f84a80b2e40 2024-05-08T23:30:35.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.285+0000 7f848f7fe700 1 -- 172.21.15.81:0/3399264897 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8498003a60 con 0x7f84a80b2e40 2024-05-08T23:30:35.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.285+0000 7f848f7fe700 1 --2- 172.21.15.81:0/3399264897 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8494066730 0x7f8494068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.285+0000 7f84a67fc700 1 --2- 172.21.15.81:0/3399264897 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8494066730 0x7f8494068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.285+0000 7f848f7fe700 1 -- 172.21.15.81:0/3399264897 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8498084bf0 con 0x7f84a80b2e40 2024-05-08T23:30:35.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.289+0000 7f84a67fc700 1 --2- 172.21.15.81:0/3399264897 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8494066730 0x7f8494068bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f84a8153640 tx=0x7f849c009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:35.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.293+0000 7f848f7fe700 1 -- 172.21.15.81:0/3399264897 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f84980522a0 con 0x7f84a80b2e40 2024-05-08T23:30:35.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.465+0000 7f84ad77c700 1 -- 172.21.15.81:0/3399264897 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84a80684b0 con 0x7f84a80b2e40 2024-05-08T23:30:35.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.469+0000 7f848f7fe700 1 -- 172.21.15.81:0/3399264897 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12382 (secure 0 0 0) 0x7f8498020090 con 0x7f84a80b2e40 2024-05-08T23:30:35.473 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:35.473 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":18,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:28.221779+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:30:35.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.469+0000 7f84ad77c700 1 -- 172.21.15.81:0/3399264897 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8494066730 msgr2=0x7f8494068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.469+0000 7f84ad77c700 1 --2- 172.21.15.81:0/3399264897 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8494066730 0x7f8494068bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f84a8153640 tx=0x7f849c009240 comp rx=0 tx=0).stop 2024-05-08T23:30:35.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.469+0000 7f84ad77c700 1 -- 172.21.15.81:0/3399264897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84a80b2e40 msgr2=0x7f84a81f0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.469+0000 7f84ad77c700 1 --2- 172.21.15.81:0/3399264897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84a80b2e40 0x7f84a81f0660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8498009ba0 tx=0x7f8498004770 comp rx=0 tx=0).stop 2024-05-08T23:30:35.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.469+0000 7f84ad77c700 1 -- 172.21.15.81:0/3399264897 shutdown_connections 2024-05-08T23:30:35.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.469+0000 7f84ad77c700 1 --2- 172.21.15.81:0/3399264897 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8494066730 0x7f8494068bf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.469+0000 7f84ad77c700 1 --2- 172.21.15.81:0/3399264897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84a815aed0 0x7f84a814b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.469+0000 7f84ad77c700 1 --2- 172.21.15.81:0/3399264897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84a80cb970 0x7f84a814ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.469+0000 7f84ad77c700 1 --2- 172.21.15.81:0/3399264897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84a80b2e40 0x7f84a81f0660 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.469+0000 7f84ad77c700 1 -- 172.21.15.81:0/3399264897 >> 172.21.15.81:0/3399264897 conn(0x7f84a806a360 msgr2=0x7f84a80b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:35.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.469+0000 7f84ad77c700 1 -- 172.21.15.81:0/3399264897 shutdown_connections 2024-05-08T23:30:35.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.469+0000 7f84ad77c700 1 -- 172.21.15.81:0/3399264897 wait complete. 2024-05-08T23:30:35.488 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:30:35.488 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:30:35.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.593+0000 7f3c93654700 1 Processor -- start 2024-05-08T23:30:35.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.593+0000 7f3c93654700 1 -- start start 2024-05-08T23:30:35.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.593+0000 7f3c93654700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3c8c15b3a0 0x7f3c8c0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.593+0000 7f3c93654700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c8c0cc950 0x7f3c8c06b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.593+0000 7f3c93654700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3c8c06b8b0 0x7f3c8c06c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.593+0000 7f3c93654700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c8c06bd40 con 0x7f3c8c15b3a0 2024-05-08T23:30:35.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.593+0000 7f3c93654700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c8c06e8e0 con 0x7f3c8c06b8b0 2024-05-08T23:30:35.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.593+0000 7f3c93654700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c0cc170 con 0x7f3c8c0cc950 2024-05-08T23:30:35.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.593+0000 7f3c913f0700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3c8c15b3a0 0x7f3c8c0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c913f0700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3c8c15b3a0 0x7f3c8c0cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36406/0 (socket says 172.21.15.81:36406) 2024-05-08T23:30:35.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c913f0700 1 -- 172.21.15.81:0/320535797 learned_addr learned my addr 172.21.15.81:0/320535797 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:35.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c90bef700 1 --2- 172.21.15.81:0/320535797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c8c0cc950 0x7f3c8c06b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c91bf1700 1 --2- 172.21.15.81:0/320535797 >> v2:172.21.15.117:3300/0 conn(0x7f3c8c06b8b0 0x7f3c8c06c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c913f0700 1 -- 172.21.15.81:0/320535797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c8c0cc950 msgr2=0x7f3c8c06b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:35.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c913f0700 1 --2- 172.21.15.81:0/320535797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c8c0cc950 0x7f3c8c06b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c913f0700 1 -- 172.21.15.81:0/320535797 >> v2:172.21.15.117:3300/0 conn(0x7f3c8c06b8b0 msgr2=0x7f3c8c06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:35.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c913f0700 1 --2- 172.21.15.81:0/320535797 >> v2:172.21.15.117:3300/0 conn(0x7f3c8c06b8b0 0x7f3c8c06c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c913f0700 1 -- 172.21.15.81:0/320535797 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c0cc410 con 0x7f3c8c15b3a0 2024-05-08T23:30:35.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c91bf1700 1 --2- 172.21.15.81:0/320535797 >> v2:172.21.15.117:3300/0 conn(0x7f3c8c06b8b0 0x7f3c8c06c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:35.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c913f0700 1 --2- 172.21.15.81:0/320535797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c8c15b3a0 0x7f3c8c0cd8b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c072270 tx=0x7f3c74009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f204650636e5b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:35.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c8bfff700 1 -- 172.21.15.81:0/320535797 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74011830 con 0x7f3c8c15b3a0 2024-05-08T23:30:35.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c8bfff700 1 -- 172.21.15.81:0/320535797 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c74011990 con 0x7f3c8c15b3a0 2024-05-08T23:30:35.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c8bfff700 1 -- 172.21.15.81:0/320535797 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74011d20 con 0x7f3c8c15b3a0 2024-05-08T23:30:35.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 -- 172.21.15.81:0/320535797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c8c15b3a0 msgr2=0x7f3c8c0cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 --2- 172.21.15.81:0/320535797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c8c15b3a0 0x7f3c8c0cd8b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c072270 tx=0x7f3c74009bf0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 -- 172.21.15.81:0/320535797 shutdown_connections 2024-05-08T23:30:35.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 --2- 172.21.15.81:0/320535797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c8c0cc950 0x7f3c8c06b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 --2- 172.21.15.81:0/320535797 >> v2:172.21.15.117:3300/0 conn(0x7f3c8c06b8b0 0x7f3c8c06c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 --2- 172.21.15.81:0/320535797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c8c15b3a0 0x7f3c8c0cd8b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 -- 172.21.15.81:0/320535797 >> 172.21.15.81:0/320535797 conn(0x7f3c8c070a70 msgr2=0x7f3c8c070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:35.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 -- 172.21.15.81:0/320535797 shutdown_connections 2024-05-08T23:30:35.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 -- 172.21.15.81:0/320535797 wait complete. 2024-05-08T23:30:35.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 Processor -- start 2024-05-08T23:30:35.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 -- start start 2024-05-08T23:30:35.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c8c06b8b0 0x7f3c8c0ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c8c0cc950 0x7f3c8c06a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c8c1524b0 0x7f3c8c14ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c1f6300 con 0x7f3c8c1524b0 2024-05-08T23:30:35.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c0ae3b0 con 0x7f3c8c06b8b0 2024-05-08T23:30:35.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.597+0000 7f3c93654700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c1f6550 con 0x7f3c8c0cc950 2024-05-08T23:30:35.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c91bf1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c8c1524b0 0x7f3c8c14ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c91bf1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c8c1524b0 0x7f3c8c14ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36420/0 (socket says 172.21.15.81:36420) 2024-05-08T23:30:35.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c91bf1700 1 -- 172.21.15.81:0/3411819492 learned_addr learned my addr 172.21.15.81:0/3411819492 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:35.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c90bef700 1 --2- 172.21.15.81:0/3411819492 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c8c0cc950 0x7f3c8c06a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c913f0700 1 --2- 172.21.15.81:0/3411819492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c8c06b8b0 0x7f3c8c0ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c91bf1700 1 -- 172.21.15.81:0/3411819492 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c8c0cc950 msgr2=0x7f3c8c06a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c91bf1700 1 --2- 172.21.15.81:0/3411819492 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c8c0cc950 0x7f3c8c06a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c91bf1700 1 -- 172.21.15.81:0/3411819492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c8c06b8b0 msgr2=0x7f3c8c0ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c91bf1700 1 --2- 172.21.15.81:0/3411819492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c8c06b8b0 0x7f3c8c0ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c91bf1700 1 -- 172.21.15.81:0/3411819492 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c74009570 con 0x7f3c8c1524b0 2024-05-08T23:30:35.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c913f0700 1 --2- 172.21.15.81:0/3411819492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c8c06b8b0 0x7f3c8c0ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:35.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c91bf1700 1 --2- 172.21.15.81:0/3411819492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c8c1524b0 0x7f3c8c14ac70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00d800 tx=0x7f3c7c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:35.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c89ffb700 1 -- 172.21.15.81:0/3411819492 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c004030 con 0x7f3c8c1524b0 2024-05-08T23:30:35.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c89ffb700 1 -- 172.21.15.81:0/3411819492 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7c004190 con 0x7f3c8c1524b0 2024-05-08T23:30:35.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c93654700 1 -- 172.21.15.81:0/3411819492 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8c1f6830 con 0x7f3c8c1524b0 2024-05-08T23:30:35.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c89ffb700 1 -- 172.21.15.81:0/3411819492 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c004520 con 0x7f3c8c1524b0 2024-05-08T23:30:35.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c93654700 1 -- 172.21.15.81:0/3411819492 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8c1f6df0 con 0x7f3c8c1524b0 2024-05-08T23:30:35.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c93654700 1 -- 172.21.15.81:0/3411819492 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c8c1f7010 con 0x7f3c8c1524b0 2024-05-08T23:30:35.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c89ffb700 1 -- 172.21.15.81:0/3411819492 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3c7c0036c0 con 0x7f3c8c1524b0 2024-05-08T23:30:35.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c89ffb700 1 --2- 172.21.15.81:0/3411819492 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3c6c066730 0x7f3c6c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c89ffb700 1 -- 172.21.15.81:0/3411819492 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3c7c084340 con 0x7f3c8c1524b0 2024-05-08T23:30:35.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.601+0000 7f3c913f0700 1 --2- 172.21.15.81:0/3411819492 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3c6c066730 0x7f3c6c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.605+0000 7f3c913f0700 1 --2- 172.21.15.81:0/3411819492 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3c6c066730 0x7f3c6c068bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0b4bc0 tx=0x7f3c74008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:35.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.609+0000 7f3c89ffb700 1 -- 172.21.15.81:0/3411819492 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3c7c051a00 con 0x7f3c8c1524b0 2024-05-08T23:30:35.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.773+0000 7f3c93654700 1 -- 172.21.15.81:0/3411819492 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c8c0684b0 con 0x7f3c8c1524b0 2024-05-08T23:30:35.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.773+0000 7f3c89ffb700 1 -- 172.21.15.81:0/3411819492 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c7c055020 con 0x7f3c8c1524b0 2024-05-08T23:30:35.780 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:30:35.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.777+0000 7f3c93654700 1 -- 172.21.15.81:0/3411819492 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3c6c066730 msgr2=0x7f3c6c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.777+0000 7f3c93654700 1 --2- 172.21.15.81:0/3411819492 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3c6c066730 0x7f3c6c068bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0b4bc0 tx=0x7f3c74008030 comp rx=0 tx=0).stop 2024-05-08T23:30:35.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.777+0000 7f3c93654700 1 -- 172.21.15.81:0/3411819492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c8c1524b0 msgr2=0x7f3c8c14ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.777+0000 7f3c93654700 1 --2- 172.21.15.81:0/3411819492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c8c1524b0 0x7f3c8c14ac70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00d800 tx=0x7f3c7c00dbc0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.777+0000 7f3c93654700 1 -- 172.21.15.81:0/3411819492 shutdown_connections 2024-05-08T23:30:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.777+0000 7f3c93654700 1 --2- 172.21.15.81:0/3411819492 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3c6c066730 0x7f3c6c068bf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.777+0000 7f3c93654700 1 --2- 172.21.15.81:0/3411819492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c8c1524b0 0x7f3c8c14ac70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.777+0000 7f3c93654700 1 --2- 172.21.15.81:0/3411819492 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c8c0cc950 0x7f3c8c06a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.777+0000 7f3c93654700 1 --2- 172.21.15.81:0/3411819492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c8c06b8b0 0x7f3c8c0ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.777+0000 7f3c93654700 1 -- 172.21.15.81:0/3411819492 >> 172.21.15.81:0/3411819492 conn(0x7f3c8c070a70 msgr2=0x7f3c8c15c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.781+0000 7f3c93654700 1 -- 172.21.15.81:0/3411819492 shutdown_connections 2024-05-08T23:30:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.781+0000 7f3c93654700 1 -- 172.21.15.81:0/3411819492 wait complete. 2024-05-08T23:30:35.799 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T23:30:35.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4012bea700 1 Processor -- start 2024-05-08T23:30:35.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4012bea700 1 -- start start 2024-05-08T23:30:35.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4012bea700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f400c15afa0 0x7f400c15b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4012bea700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f400c0ce990 0x7f400c0b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4012bea700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f400c15eda0 con 0x7f400c15b8c0 2024-05-08T23:30:35.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4012bea700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f400c0ced70 con 0x7f400c15afa0 2024-05-08T23:30:35.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4012bea700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f400c15ea70 con 0x7f400c0ce990 2024-05-08T23:30:35.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4011be8700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f400c15afa0 0x7f400c15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4011be8700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f400c15afa0 0x7f400c15b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32808/0 (socket says 172.21.15.81:32808) 2024-05-08T23:30:35.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4011be8700 1 -- 172.21.15.81:0/2225833167 learned_addr learned my addr 172.21.15.81:0/2225833167 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:35.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f40123e9700 1 --2- 172.21.15.81:0/2225833167 >> v2:172.21.15.117:3300/0 conn(0x7f400c0ce990 0x7f400c0b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4011be8700 1 -- 172.21.15.81:0/2225833167 >> v2:172.21.15.117:3300/0 conn(0x7f400c0ce990 msgr2=0x7f400c0b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:35.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4011be8700 1 --2- 172.21.15.81:0/2225833167 >> v2:172.21.15.117:3300/0 conn(0x7f400c0ce990 0x7f400c0b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4011be8700 1 -- 172.21.15.81:0/2225833167 >> v1:172.21.15.117:6789/0 conn(0x7f400c15b8c0 legacy=0x7f400c15f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4011be8700 1 -- 172.21.15.81:0/2225833167 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c15bd40 con 0x7f400c15afa0 2024-05-08T23:30:35.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f40123e9700 1 --2- 172.21.15.81:0/2225833167 >> v2:172.21.15.117:3300/0 conn(0x7f400c0ce990 0x7f400c0b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:35.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4011be8700 1 --2- 172.21.15.81:0/2225833167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f400c15afa0 0x7f400c15b380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f400c06e5f0 tx=0x7f400400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a095663c5e82f340 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:35.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4010be6700 1 -- 172.21.15.81:0/2225833167 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400400e070 con 0x7f400c15afa0 2024-05-08T23:30:35.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.905+0000 7f4010be6700 1 -- 172.21.15.81:0/2225833167 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400400b9c0 con 0x7f400c15afa0 2024-05-08T23:30:35.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.909+0000 7f4010be6700 1 -- 172.21.15.81:0/2225833167 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400400bd70 con 0x7f400c15afa0 2024-05-08T23:30:35.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.909+0000 7f4012bea700 1 -- 172.21.15.81:0/2225833167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f400c15afa0 msgr2=0x7f400c15b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.909+0000 7f4012bea700 1 --2- 172.21.15.81:0/2225833167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f400c15afa0 0x7f400c15b380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f400c06e5f0 tx=0x7f400400b330 comp rx=0 tx=0).stop 2024-05-08T23:30:35.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.909+0000 7f4012bea700 1 -- 172.21.15.81:0/2225833167 shutdown_connections 2024-05-08T23:30:35.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.909+0000 7f4012bea700 1 --2- 172.21.15.81:0/2225833167 >> v2:172.21.15.117:3300/0 conn(0x7f400c0ce990 0x7f400c0b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.909+0000 7f4012bea700 1 --2- 172.21.15.81:0/2225833167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f400c15afa0 0x7f400c15b380 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.909+0000 7f4012bea700 1 -- 172.21.15.81:0/2225833167 >> 172.21.15.81:0/2225833167 conn(0x7f400c06c330 msgr2=0x7f400c06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:35.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.909+0000 7f4012bea700 1 -- 172.21.15.81:0/2225833167 shutdown_connections 2024-05-08T23:30:35.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.909+0000 7f4012bea700 1 -- 172.21.15.81:0/2225833167 wait complete. 2024-05-08T23:30:35.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.909+0000 7f4012bea700 1 Processor -- start 2024-05-08T23:30:35.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.909+0000 7f4012bea700 1 -- start start 2024-05-08T23:30:35.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f4012bea700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f400c0ce990 0x7f400c14d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f4012bea700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f400c15afa0 0x7f400c14baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f4012bea700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f400c15b8c0 0x7f400c151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f4012bea700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c14c030 con 0x7f400c15b8c0 2024-05-08T23:30:35.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f4012bea700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c14c5d0 con 0x7f400c15afa0 2024-05-08T23:30:35.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f4012bea700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400c1f0cb0 con 0x7f400c0ce990 2024-05-08T23:30:35.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f40123e9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f400c15b8c0 0x7f400c151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f40123e9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f400c15b8c0 0x7f400c151460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32812/0 (socket says 172.21.15.81:32812) 2024-05-08T23:30:35.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f40123e9700 1 -- 172.21.15.81:0/4229139133 learned_addr learned my addr 172.21.15.81:0/4229139133 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:35.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f4011be8700 1 --2- 172.21.15.81:0/4229139133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f400c0ce990 0x7f400c14d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f40113e7700 1 --2- 172.21.15.81:0/4229139133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f400c15afa0 0x7f400c14baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f40123e9700 1 -- 172.21.15.81:0/4229139133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f400c0ce990 msgr2=0x7f400c14d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f40123e9700 1 --2- 172.21.15.81:0/4229139133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f400c0ce990 0x7f400c14d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f40123e9700 1 -- 172.21.15.81:0/4229139133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f400c15afa0 msgr2=0x7f400c14baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:35.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f40123e9700 1 --2- 172.21.15.81:0/4229139133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f400c15afa0 0x7f400c14baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:35.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f40123e9700 1 -- 172.21.15.81:0/4229139133 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4004009b30 con 0x7f400c15b8c0 2024-05-08T23:30:35.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f40123e9700 1 --2- 172.21.15.81:0/4229139133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f400c15b8c0 0x7f400c151460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400e810 tx=0x7f3ff400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:35.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f400a7fc700 1 -- 172.21.15.81:0/4229139133 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff400c070 con 0x7f400c15b8c0 2024-05-08T23:30:35.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f400a7fc700 1 -- 172.21.15.81:0/4229139133 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff4004020 con 0x7f400c15b8c0 2024-05-08T23:30:35.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f400a7fc700 1 -- 172.21.15.81:0/4229139133 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff40043b0 con 0x7f400c15b8c0 2024-05-08T23:30:35.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f4012bea700 1 -- 172.21.15.81:0/4229139133 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400c14d900 con 0x7f400c15b8c0 2024-05-08T23:30:35.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f4012bea700 1 -- 172.21.15.81:0/4229139133 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400c14dec0 con 0x7f400c15b8c0 2024-05-08T23:30:35.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.913+0000 7f4012bea700 1 -- 172.21.15.81:0/4229139133 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400c1f0e70 con 0x7f400c15b8c0 2024-05-08T23:30:35.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.917+0000 7f400a7fc700 1 -- 172.21.15.81:0/4229139133 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3ff40036c0 con 0x7f400c15b8c0 2024-05-08T23:30:35.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.917+0000 7f400a7fc700 1 --2- 172.21.15.81:0/4229139133 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3fec066760 0x7f3fec068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:35.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.917+0000 7f400a7fc700 1 -- 172.21.15.81:0/4229139133 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3ff4083690 con 0x7f400c15b8c0 2024-05-08T23:30:35.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.917+0000 7f4011be8700 1 --2- 172.21.15.81:0/4229139133 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3fec066760 0x7f3fec068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:35.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.917+0000 7f4011be8700 1 --2- 172.21.15.81:0/4229139133 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3fec066760 0x7f3fec068c20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4004000f80 tx=0x7f4004009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:35.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:35.925+0000 7f400a7fc700 1 -- 172.21.15.81:0/4229139133 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3ff4050d60 con 0x7f400c15b8c0 2024-05-08T23:30:35.963 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:30:35.965 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:30:36.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.097+0000 7f4012bea700 1 -- 172.21.15.81:0/4229139133 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f400c0684b0 con 0x7f400c15b8c0 2024-05-08T23:30:36.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.097+0000 7f400a7fc700 1 -- 172.21.15.81:0/4229139133 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3ff4054380 con 0x7f400c15b8c0 2024-05-08T23:30:36.101 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:36.101 INFO:teuthology.orchestra.run.smithi081.stdout:[ 2024-05-08T23:30:36.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:30:36.102 INFO:teuthology.orchestra.run.smithi081.stdout: "poolnum": 1, 2024-05-08T23:30:36.102 INFO:teuthology.orchestra.run.smithi081.stdout: "poolname": ".mgr" 2024-05-08T23:30:36.102 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:30:36.102 INFO:teuthology.orchestra.run.smithi081.stdout:] 2024-05-08T23:30:36.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.097+0000 7f4012bea700 1 -- 172.21.15.81:0/4229139133 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3fec066760 msgr2=0x7f3fec068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:36.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.097+0000 7f4012bea700 1 --2- 172.21.15.81:0/4229139133 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3fec066760 0x7f3fec068c20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4004000f80 tx=0x7f4004009000 comp rx=0 tx=0).stop 2024-05-08T23:30:36.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.097+0000 7f4012bea700 1 -- 172.21.15.81:0/4229139133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f400c15b8c0 msgr2=0x7f400c151460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:36.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.097+0000 7f4012bea700 1 --2- 172.21.15.81:0/4229139133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f400c15b8c0 0x7f400c151460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400e810 tx=0x7f3ff400eb20 comp rx=0 tx=0).stop 2024-05-08T23:30:36.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.101+0000 7f4012bea700 1 -- 172.21.15.81:0/4229139133 shutdown_connections 2024-05-08T23:30:36.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.101+0000 7f4012bea700 1 --2- 172.21.15.81:0/4229139133 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3fec066760 0x7f3fec068c20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:36.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.101+0000 7f4012bea700 1 --2- 172.21.15.81:0/4229139133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f400c15b8c0 0x7f400c151460 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:36.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.101+0000 7f4012bea700 1 --2- 172.21.15.81:0/4229139133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f400c15afa0 0x7f400c14baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:36.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.101+0000 7f4012bea700 1 --2- 172.21.15.81:0/4229139133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f400c0ce990 0x7f400c14d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:36.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.101+0000 7f4012bea700 1 -- 172.21.15.81:0/4229139133 >> 172.21.15.81:0/4229139133 conn(0x7f400c06c330 msgr2=0x7f400c0afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:36.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.101+0000 7f4012bea700 1 -- 172.21.15.81:0/4229139133 shutdown_connections 2024-05-08T23:30:36.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.101+0000 7f4012bea700 1 -- 172.21.15.81:0/4229139133 wait complete. 2024-05-08T23:30:36.116 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T23:30:36.116 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T23:30:36.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.233+0000 7f4f9359e700 1 Processor -- start 2024-05-08T23:30:36.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.233+0000 7f4f9359e700 1 -- start start 2024-05-08T23:30:36.240 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9359e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f940cc4a0 0x7f4f940d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:36.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9359e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9406e3b0 con 0x7f4f9409a180 2024-05-08T23:30:36.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9359e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f940cc980 con 0x7f4f9415b220 2024-05-08T23:30:36.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9359e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9415b600 con 0x7f4f940cc4a0 2024-05-08T23:30:36.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f91d9b700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f4f9409a180 0x7f4f9406ee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54966/0 (socket says 172.21.15.81:54966) 2024-05-08T23:30:36.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f91d9b700 1 -- 172.21.15.81:0/1258584258 learned_addr learned my addr 172.21.15.81:0/1258584258 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:36.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f92d9d700 1 --2- 172.21.15.81:0/1258584258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f940cc4a0 0x7f4f940d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:36.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 -- 172.21.15.81:0/1258584258 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769731682 0 0) 0x7f4f9406e3b0 con 0x7f4f9409a180 2024-05-08T23:30:36.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 -- 172.21.15.81:0/1258584258 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f7c0035d0 con 0x7f4f9409a180 2024-05-08T23:30:36.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 -- 172.21.15.81:0/1258584258 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2113047139 0 0) 0x7f4f7c0035d0 con 0x7f4f9409a180 2024-05-08T23:30:36.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 -- 172.21.15.81:0/1258584258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f940cc4a0 msgr2=0x7f4f940d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:36.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 --2- 172.21.15.81:0/1258584258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f940cc4a0 0x7f4f940d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:36.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 -- 172.21.15.81:0/1258584258 >> v1:172.21.15.117:6789/0 conn(0x7f4f9415b220 legacy=0x7f4f94099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:36.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 -- 172.21.15.81:0/1258584258 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f940ccc60 con 0x7f4f9409a180 2024-05-08T23:30:36.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 -- 172.21.15.81:0/1258584258 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f4f84001670 con 0x7f4f9409a180 2024-05-08T23:30:36.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 -- 172.21.15.81:0/1258584258 >> v1:172.21.15.81:6789/0 conn(0x7f4f9409a180 legacy=0x7f4f9406ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:36.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 --2- 172.21.15.81:0/1258584258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f7c003a50 0x7f4f7c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:36.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 --2- 172.21.15.81:0/1258584258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f940cc4a0 0x7f4f7c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:36.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 --2- 172.21.15.81:0/1258584258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7c004db0 0x7f4f7c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:36.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 -- 172.21.15.81:0/1258584258 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9415b600 con 0x7f4f7c004db0 2024-05-08T23:30:36.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 -- 172.21.15.81:0/1258584258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c00bae0 con 0x7f4f7c003a50 2024-05-08T23:30:36.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 -- 172.21.15.81:0/1258584258 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c00bca0 con 0x7f4f940cc4a0 2024-05-08T23:30:36.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f92d9d700 1 --2- 172.21.15.81:0/1258584258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7c004db0 0x7f4f7c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:36.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f91d9b700 1 --2- 172.21.15.81:0/1258584258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f940cc4a0 0x7f4f7c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:36.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9259c700 1 --2- 172.21.15.81:0/1258584258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f7c003a50 0x7f4f7c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:36.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f92d9d700 1 -- 172.21.15.81:0/1258584258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f940cc4a0 msgr2=0x7f4f7c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:36.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f92d9d700 1 --2- 172.21.15.81:0/1258584258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f940cc4a0 0x7f4f7c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:36.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f92d9d700 1 -- 172.21.15.81:0/1258584258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f7c003a50 msgr2=0x7f4f7c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:36.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f92d9d700 1 --2- 172.21.15.81:0/1258584258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f7c003a50 0x7f4f7c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:36.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f92d9d700 1 -- 172.21.15.81:0/1258584258 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c0035d0 con 0x7f4f7c004db0 2024-05-08T23:30:36.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f92d9d700 1 --2- 172.21.15.81:0/1258584258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7c004db0 0x7f4f7c005230 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4f940718d0 tx=0x7f4f7c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b87006d6a560d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:36.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 -- 172.21.15.81:0/1258584258 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c00eba0 con 0x7f4f7c004db0 2024-05-08T23:30:36.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 -- 172.21.15.81:0/1258584258 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c00f040 con 0x7f4f7c004db0 2024-05-08T23:30:36.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9159a700 1 -- 172.21.15.81:0/1258584258 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c014480 con 0x7f4f7c004db0 2024-05-08T23:30:36.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9359e700 1 -- 172.21.15.81:0/1258584258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7c004db0 msgr2=0x7f4f7c005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:36.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9359e700 1 --2- 172.21.15.81:0/1258584258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7c004db0 0x7f4f7c005230 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4f940718d0 tx=0x7f4f7c0058a0 comp rx=0 tx=0).stop 2024-05-08T23:30:36.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f92d9d700 1 -- 172.21.15.81:0/1258584258 reap_dead start 2024-05-08T23:30:36.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9359e700 1 -- 172.21.15.81:0/1258584258 shutdown_connections 2024-05-08T23:30:36.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9359e700 1 -- 172.21.15.81:0/1258584258 >> 172.21.15.81:0/1258584258 conn(0x7f4f940d0e80 msgr2=0x7f4f940d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:36.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9359e700 1 -- 172.21.15.81:0/1258584258 shutdown_connections 2024-05-08T23:30:36.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.237+0000 7f4f9359e700 1 -- 172.21.15.81:0/1258584258 wait complete. 2024-05-08T23:30:36.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.241+0000 7f4f9359e700 1 Processor -- start 2024-05-08T23:30:36.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.241+0000 7f4f9359e700 1 -- start start 2024-05-08T23:30:36.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.241+0000 7f4f9359e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7c004db0 0x7f4f9409ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:36.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9359e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f9409dbc0 0x7f4f940b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:36.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9359e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f940c7f50 0x7f4f940c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:36.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9359e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9409e150 con 0x7f4f7c004db0 2024-05-08T23:30:36.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9359e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9409e430 con 0x7f4f940c7f50 2024-05-08T23:30:36.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9359e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9409e710 con 0x7f4f9409dbc0 2024-05-08T23:30:36.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7c004db0 0x7f4f9409ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:36.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7c004db0 0x7f4f9409ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32824/0 (socket says 172.21.15.81:32824) 2024-05-08T23:30:36.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9259c700 1 -- 172.21.15.81:0/2619143234 learned_addr learned my addr 172.21.15.81:0/2619143234 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:36.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f91d9b700 1 --2- 172.21.15.81:0/2619143234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f9409dbc0 0x7f4f940b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:36.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f92d9d700 1 --2- 172.21.15.81:0/2619143234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f940c7f50 0x7f4f940c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:36.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9259c700 1 -- 172.21.15.81:0/2619143234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f9409dbc0 msgr2=0x7f4f940b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:36.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9259c700 1 --2- 172.21.15.81:0/2619143234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f9409dbc0 0x7f4f940b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:36.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9259c700 1 -- 172.21.15.81:0/2619143234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f940c7f50 msgr2=0x7f4f940c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:36.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9259c700 1 --2- 172.21.15.81:0/2619143234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f940c7f50 0x7f4f940c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:36.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9259c700 1 -- 172.21.15.81:0/2619143234 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9409e9f0 con 0x7f4f7c004db0 2024-05-08T23:30:36.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9259c700 1 --2- 172.21.15.81:0/2619143234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7c004db0 0x7f4f9409ae40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800f820 tx=0x7f4f8800fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:36.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f82ffd700 1 -- 172.21.15.81:0/2619143234 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f880052e0 con 0x7f4f7c004db0 2024-05-08T23:30:36.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f82ffd700 1 -- 172.21.15.81:0/2619143234 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f88005440 con 0x7f4f7c004db0 2024-05-08T23:30:36.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9359e700 1 -- 172.21.15.81:0/2619143234 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9409b380 con 0x7f4f7c004db0 2024-05-08T23:30:36.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f82ffd700 1 -- 172.21.15.81:0/2619143234 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f880038c0 con 0x7f4f7c004db0 2024-05-08T23:30:36.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.245+0000 7f4f9359e700 1 -- 172.21.15.81:0/2619143234 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f940b1b50 con 0x7f4f7c004db0 2024-05-08T23:30:36.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.249+0000 7f4f82ffd700 1 -- 172.21.15.81:0/2619143234 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4f88003a20 con 0x7f4f7c004db0 2024-05-08T23:30:36.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.249+0000 7f4f9359e700 1 -- 172.21.15.81:0/2619143234 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f940672a0 con 0x7f4f7c004db0 2024-05-08T23:30:36.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.249+0000 7f4f82ffd700 1 --2- 172.21.15.81:0/2619143234 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4f74066760 0x7f4f74068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:36.253 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.249+0000 7f4f91d9b700 1 --2- 172.21.15.81:0/2619143234 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4f74066760 0x7f4f74068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:36.253 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.249+0000 7f4f82ffd700 1 -- 172.21.15.81:0/2619143234 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4f880839e0 con 0x7f4f7c004db0 2024-05-08T23:30:36.253 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.249+0000 7f4f91d9b700 1 --2- 172.21.15.81:0/2619143234 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4f74066760 0x7f4f74068c20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4f940b1fa0 tx=0x7f4f84000f80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:36.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.253+0000 7f4f82ffd700 1 -- 172.21.15.81:0/2619143234 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4f880510a0 con 0x7f4f7c004db0 2024-05-08T23:30:36.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:36.425+0000 7f4f9359e700 1 -- 172.21.15.81:0/2619143234 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4f940b1b50 con 0x7f4f7c004db0 2024-05-08T23:30:37.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.281+0000 7f4f82ffd700 1 -- 172.21.15.81:0/2619143234 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4f940b1b50 con 0x7f4f7c004db0 2024-05-08T23:30:37.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.285+0000 7f4f9359e700 1 -- 172.21.15.81:0/2619143234 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4f74066760 msgr2=0x7f4f74068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:37.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.285+0000 7f4f9359e700 1 --2- 172.21.15.81:0/2619143234 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4f74066760 0x7f4f74068c20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4f940b1fa0 tx=0x7f4f84000f80 comp rx=0 tx=0).stop 2024-05-08T23:30:37.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.285+0000 7f4f9359e700 1 -- 172.21.15.81:0/2619143234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7c004db0 msgr2=0x7f4f9409ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:37.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.285+0000 7f4f9359e700 1 --2- 172.21.15.81:0/2619143234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7c004db0 0x7f4f9409ae40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800f820 tx=0x7f4f8800fb30 comp rx=0 tx=0).stop 2024-05-08T23:30:37.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.285+0000 7f4f9359e700 1 -- 172.21.15.81:0/2619143234 shutdown_connections 2024-05-08T23:30:37.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.285+0000 7f4f9359e700 1 --2- 172.21.15.81:0/2619143234 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4f74066760 0x7f4f74068c20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:37.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.285+0000 7f4f9359e700 1 --2- 172.21.15.81:0/2619143234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f940c7f50 0x7f4f940c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:37.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.285+0000 7f4f9359e700 1 --2- 172.21.15.81:0/2619143234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f9409dbc0 0x7f4f940b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:37.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.285+0000 7f4f9359e700 1 --2- 172.21.15.81:0/2619143234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7c004db0 0x7f4f9409ae40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:37.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.285+0000 7f4f9359e700 1 -- 172.21.15.81:0/2619143234 >> 172.21.15.81:0/2619143234 conn(0x7f4f940d0e80 msgr2=0x7f4f940afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:37.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.285+0000 7f4f9359e700 1 -- 172.21.15.81:0/2619143234 shutdown_connections 2024-05-08T23:30:37.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.285+0000 7f4f9359e700 1 -- 172.21.15.81:0/2619143234 wait complete. 2024-05-08T23:30:37.291 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' created 2024-05-08T23:30:37.305 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T23:30:37.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.425+0000 7ff4b7fff700 1 Processor -- start 2024-05-08T23:30:37.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.425+0000 7ff4b7fff700 1 -- start start 2024-05-08T23:30:37.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.425+0000 7ff4b7fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff4b806b890 0x7ff4b809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:37.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b7fff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b809a9c0 con 0x7ff4b806bfb0 2024-05-08T23:30:37.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b7fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b80b4c50 con 0x7ff4b815d010 2024-05-08T23:30:37.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b7fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b80b63c0 con 0x7ff4b806b890 2024-05-08T23:30:37.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b77fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff4b806bfb0 0x7ff4b80addc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54978/0 (socket says 172.21.15.81:54978) 2024-05-08T23:30:37.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b77fe700 1 -- 172.21.15.81:0/30463855 learned_addr learned my addr 172.21.15.81:0/30463855 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:37.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b67fc700 1 --2- 172.21.15.81:0/30463855 >> v2:172.21.15.117:3300/0 conn(0x7ff4b806b890 0x7ff4b809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:37.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b5ffb700 1 -- 172.21.15.81:0/30463855 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164272379 0 0) 0x7ff4b0001490 con 0x7ff4b806bfb0 2024-05-08T23:30:37.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b5ffb700 1 -- 172.21.15.81:0/30463855 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4a00035d0 con 0x7ff4b806bfb0 2024-05-08T23:30:37.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b5ffb700 1 -- 172.21.15.81:0/30463855 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2236237686 0 0) 0x7ff4b80b4c50 con 0x7ff4b815d010 2024-05-08T23:30:37.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b5ffb700 1 -- 172.21.15.81:0/30463855 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4b0001490 con 0x7ff4b815d010 2024-05-08T23:30:37.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b67fc700 1 -- 172.21.15.81:0/30463855 >> v1:172.21.15.117:6789/0 conn(0x7ff4b815d010 legacy=0x7ff4b80b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:37.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b67fc700 1 -- 172.21.15.81:0/30463855 >> v1:172.21.15.81:6789/0 conn(0x7ff4b806bfb0 legacy=0x7ff4b80addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:37.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b5ffb700 1 -- 172.21.15.81:0/30463855 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2672265909 0 0) 0x7ff4a00035d0 con 0x7ff4b806bfb0 2024-05-08T23:30:37.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b67fc700 1 -- 172.21.15.81:0/30463855 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b80b4ef0 con 0x7ff4b806b890 2024-05-08T23:30:37.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b67fc700 1 --2- 172.21.15.81:0/30463855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b806b890 0x7ff4b809a480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009850 tx=0x7ff4ac009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7d3317a9560b63f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:37.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b5ffb700 1 -- 172.21.15.81:0/30463855 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac011830 con 0x7ff4b806b890 2024-05-08T23:30:37.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b5ffb700 1 -- 172.21.15.81:0/30463855 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac011990 con 0x7ff4b806b890 2024-05-08T23:30:37.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b5ffb700 1 -- 172.21.15.81:0/30463855 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac011d00 con 0x7ff4b806b890 2024-05-08T23:30:37.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b7fff700 1 -- 172.21.15.81:0/30463855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b806b890 msgr2=0x7ff4b809a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:37.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b7fff700 1 --2- 172.21.15.81:0/30463855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b806b890 0x7ff4b809a480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009850 tx=0x7ff4ac009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:37.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b7fff700 1 -- 172.21.15.81:0/30463855 shutdown_connections 2024-05-08T23:30:37.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b7fff700 1 --2- 172.21.15.81:0/30463855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b806b890 0x7ff4b809a480 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:37.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b7fff700 1 -- 172.21.15.81:0/30463855 >> 172.21.15.81:0/30463855 conn(0x7ff4b80b2ae0 msgr2=0x7ff4b80b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:37.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b7fff700 1 -- 172.21.15.81:0/30463855 shutdown_connections 2024-05-08T23:30:37.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.429+0000 7ff4b7fff700 1 -- 172.21.15.81:0/30463855 wait complete. 2024-05-08T23:30:37.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.433+0000 7ff4b7fff700 1 Processor -- start 2024-05-08T23:30:37.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.433+0000 7ff4b7fff700 1 -- start start 2024-05-08T23:30:37.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b806b890 0x7ff4b81f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:37.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4b806bfb0 0x7ff4b8154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:37.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4b815d010 0x7ff4b815e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:37.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b7fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b815dd10 con 0x7ff4b806bfb0 2024-05-08T23:30:37.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b7fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b81f3190 con 0x7ff4b806b890 2024-05-08T23:30:37.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b7fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b81f3470 con 0x7ff4b815d010 2024-05-08T23:30:37.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b67fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4b806bfb0 0x7ff4b8154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:37.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b67fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4b806bfb0 0x7ff4b8154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32832/0 (socket says 172.21.15.81:32832) 2024-05-08T23:30:37.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b67fc700 1 -- 172.21.15.81:0/3138262027 learned_addr learned my addr 172.21.15.81:0/3138262027 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:37.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b6ffd700 1 --2- 172.21.15.81:0/3138262027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b806b890 0x7ff4b81f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:37.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b77fe700 1 --2- 172.21.15.81:0/3138262027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4b815d010 0x7ff4b815e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:37.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b67fc700 1 -- 172.21.15.81:0/3138262027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4b815d010 msgr2=0x7ff4b815e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:37.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b67fc700 1 --2- 172.21.15.81:0/3138262027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4b815d010 0x7ff4b815e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:37.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b67fc700 1 -- 172.21.15.81:0/3138262027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b806b890 msgr2=0x7ff4b81f28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:37.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b67fc700 1 --2- 172.21.15.81:0/3138262027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b806b890 0x7ff4b81f28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:37.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b67fc700 1 -- 172.21.15.81:0/3138262027 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ac0094b0 con 0x7ff4b806bfb0 2024-05-08T23:30:37.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b6ffd700 1 --2- 172.21.15.81:0/3138262027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b806b890 0x7ff4b81f28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:37.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b67fc700 1 --2- 172.21.15.81:0/3138262027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4b806bfb0 0x7ff4b8154940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0090b0 tx=0x7ff4ac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:37.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff49f7fe700 1 -- 172.21.15.81:0/3138262027 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac004380 con 0x7ff4b806bfb0 2024-05-08T23:30:37.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff49f7fe700 1 -- 172.21.15.81:0/3138262027 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac01f070 con 0x7ff4b806bfb0 2024-05-08T23:30:37.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b7fff700 1 -- 172.21.15.81:0/3138262027 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b81f3750 con 0x7ff4b806bfb0 2024-05-08T23:30:37.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff49f7fe700 1 -- 172.21.15.81:0/3138262027 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac0036c0 con 0x7ff4b806bfb0 2024-05-08T23:30:37.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b7fff700 1 -- 172.21.15.81:0/3138262027 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b81f2e10 con 0x7ff4b806bfb0 2024-05-08T23:30:37.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.437+0000 7ff4b7fff700 1 -- 172.21.15.81:0/3138262027 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b815de50 con 0x7ff4b806bfb0 2024-05-08T23:30:37.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.441+0000 7ff49f7fe700 1 -- 172.21.15.81:0/3138262027 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff4ac021030 con 0x7ff4b806bfb0 2024-05-08T23:30:37.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.441+0000 7ff49f7fe700 1 --2- 172.21.15.81:0/3138262027 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff4a4066730 0x7ff4a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:37.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.441+0000 7ff4b6ffd700 1 --2- 172.21.15.81:0/3138262027 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff4a4066730 0x7ff4a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:37.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.441+0000 7ff49f7fe700 1 -- 172.21.15.81:0/3138262027 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff4ac085480 con 0x7ff4b806bfb0 2024-05-08T23:30:37.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.441+0000 7ff4b6ffd700 1 --2- 172.21.15.81:0/3138262027 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff4a4066730 0x7ff4a4068bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80011e0 tx=0x7ff4a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:37.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.449+0000 7ff49f7fe700 1 -- 172.21.15.81:0/3138262027 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff4ac052a40 con 0x7ff4b806bfb0 2024-05-08T23:30:37.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:37.617+0000 7ff4b7fff700 1 -- 172.21.15.81:0/3138262027 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff4b815de50 con 0x7ff4b806bfb0 2024-05-08T23:30:38.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.293+0000 7ff49f7fe700 1 -- 172.21.15.81:0/3138262027 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff4b815de50 con 0x7ff4b806bfb0 2024-05-08T23:30:38.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.301+0000 7ff4b7fff700 1 -- 172.21.15.81:0/3138262027 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff4a4066730 msgr2=0x7ff4a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.301+0000 7ff4b7fff700 1 --2- 172.21.15.81:0/3138262027 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff4a4066730 0x7ff4a4068bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80011e0 tx=0x7ff4a8009040 comp rx=0 tx=0).stop 2024-05-08T23:30:38.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.301+0000 7ff4b7fff700 1 -- 172.21.15.81:0/3138262027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4b806bfb0 msgr2=0x7ff4b8154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.301+0000 7ff4b7fff700 1 --2- 172.21.15.81:0/3138262027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4b806bfb0 0x7ff4b8154940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0090b0 tx=0x7ff4ac004770 comp rx=0 tx=0).stop 2024-05-08T23:30:38.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.301+0000 7ff4b7fff700 1 -- 172.21.15.81:0/3138262027 shutdown_connections 2024-05-08T23:30:38.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.301+0000 7ff4b7fff700 1 --2- 172.21.15.81:0/3138262027 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff4a4066730 0x7ff4a4068bf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.301+0000 7ff4b7fff700 1 --2- 172.21.15.81:0/3138262027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4b815d010 0x7ff4b815e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.301+0000 7ff4b7fff700 1 --2- 172.21.15.81:0/3138262027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4b806bfb0 0x7ff4b8154940 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.301+0000 7ff4b7fff700 1 --2- 172.21.15.81:0/3138262027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4b806b890 0x7ff4b81f28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.301+0000 7ff4b7fff700 1 -- 172.21.15.81:0/3138262027 >> 172.21.15.81:0/3138262027 conn(0x7ff4b80b2ae0 msgr2=0x7ff4b8160bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:38.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.301+0000 7ff4b7fff700 1 -- 172.21.15.81:0/3138262027 shutdown_connections 2024-05-08T23:30:38.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.301+0000 7ff4b7fff700 1 -- 172.21.15.81:0/3138262027 wait complete. 2024-05-08T23:30:38.315 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' created 2024-05-08T23:30:38.328 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T23:30:38.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.437+0000 7f1f115d2700 1 Processor -- start 2024-05-08T23:30:38.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.437+0000 7f1f115d2700 1 -- start start 2024-05-08T23:30:38.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.437+0000 7f1f115d2700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1f0c15cfa0 0x7f1f0c0b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.437+0000 7f1f115d2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f0c0aed90 0x7f1f0c06a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.437+0000 7f1f115d2700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c161940 con 0x7f1f0c06a8a0 2024-05-08T23:30:38.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.437+0000 7f1f115d2700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c06ad10 con 0x7f1f0c15cfa0 2024-05-08T23:30:38.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.437+0000 7f1f115d2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0af1f0 con 0x7f1f0c0aed90 2024-05-08T23:30:38.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.437+0000 7f1f0bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1f0c15cfa0 0x7f1f0c0b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.437+0000 7f1f0b7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f0c0aed90 0x7f1f0c06a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f10dd1700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f1f0c06a8a0 0x7f1f0c0711c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:41168/0 (socket says 172.21.15.81:41168) 2024-05-08T23:30:38.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f10dd1700 1 -- 172.21.15.81:0/1955390535 learned_addr learned my addr 172.21.15.81:0/1955390535 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:38.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f0affd700 1 -- 172.21.15.81:0/1955390535 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1873067368 0 0) 0x7f1f0c161940 con 0x7f1f0c06a8a0 2024-05-08T23:30:38.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f0affd700 1 -- 172.21.15.81:0/1955390535 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ef40035d0 con 0x7f1f0c06a8a0 2024-05-08T23:30:38.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f0b7fe700 1 -- 172.21.15.81:0/1955390535 >> v2:172.21.15.117:3300/0 conn(0x7f1f0c15cfa0 msgr2=0x7f1f0c0b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:38.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f0b7fe700 1 --2- 172.21.15.81:0/1955390535 >> v2:172.21.15.117:3300/0 conn(0x7f1f0c15cfa0 0x7f1f0c0b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f0b7fe700 1 -- 172.21.15.81:0/1955390535 >> v1:172.21.15.117:6789/0 conn(0x7f1f0c06a8a0 legacy=0x7f1f0c0711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f0b7fe700 1 -- 172.21.15.81:0/1955390535 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c160d30 con 0x7f1f0c0aed90 2024-05-08T23:30:38.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f0bfff700 1 --2- 172.21.15.81:0/1955390535 >> v2:172.21.15.117:3300/0 conn(0x7f1f0c15cfa0 0x7f1f0c0b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:38.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f0b7fe700 1 --2- 172.21.15.81:0/1955390535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f0c0aed90 0x7f1f0c06a360 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009850 tx=0x7f1f00009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e6e48df1b2c8482 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:38.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f0affd700 1 -- 172.21.15.81:0/1955390535 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00011830 con 0x7f1f0c0aed90 2024-05-08T23:30:38.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f0affd700 1 -- 172.21.15.81:0/1955390535 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f00011990 con 0x7f1f0c0aed90 2024-05-08T23:30:38.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f0affd700 1 -- 172.21.15.81:0/1955390535 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00011d40 con 0x7f1f0c0aed90 2024-05-08T23:30:38.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f115d2700 1 -- 172.21.15.81:0/1955390535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f0c0aed90 msgr2=0x7f1f0c06a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f115d2700 1 --2- 172.21.15.81:0/1955390535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f0c0aed90 0x7f1f0c06a360 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009850 tx=0x7f1f00009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:38.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f115d2700 1 -- 172.21.15.81:0/1955390535 shutdown_connections 2024-05-08T23:30:38.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f115d2700 1 --2- 172.21.15.81:0/1955390535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f0c0aed90 0x7f1f0c06a360 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f115d2700 1 --2- 172.21.15.81:0/1955390535 >> v2:172.21.15.117:3300/0 conn(0x7f1f0c15cfa0 0x7f1f0c0b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f115d2700 1 -- 172.21.15.81:0/1955390535 >> 172.21.15.81:0/1955390535 conn(0x7f1f0c06b470 msgr2=0x7f1f0c06b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:38.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f115d2700 1 -- 172.21.15.81:0/1955390535 shutdown_connections 2024-05-08T23:30:38.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.441+0000 7f1f115d2700 1 -- 172.21.15.81:0/1955390535 wait complete. 2024-05-08T23:30:38.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f115d2700 1 Processor -- start 2024-05-08T23:30:38.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f115d2700 1 -- start start 2024-05-08T23:30:38.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f115d2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f0c06a8a0 0x7f1f0c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f115d2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f0c0aed90 0x7f1f0c14b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f115d2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f0c15cfa0 0x7f1f0c1f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f115d2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c14bcd0 con 0x7f1f0c0aed90 2024-05-08T23:30:38.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f115d2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c1f78d0 con 0x7f1f0c15cfa0 2024-05-08T23:30:38.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f115d2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c1f7bb0 con 0x7f1f0c06a8a0 2024-05-08T23:30:38.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f0b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f0c0aed90 0x7f1f0c14b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f0b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f0c0aed90 0x7f1f0c14b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32844/0 (socket says 172.21.15.81:32844) 2024-05-08T23:30:38.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f0b7fe700 1 -- 172.21.15.81:0/1941614825 learned_addr learned my addr 172.21.15.81:0/1941614825 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:38.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f0bfff700 1 --2- 172.21.15.81:0/1941614825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f0c06a8a0 0x7f1f0c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f10dd1700 1 --2- 172.21.15.81:0/1941614825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f0c15cfa0 0x7f1f0c1f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f0b7fe700 1 -- 172.21.15.81:0/1941614825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f0c06a8a0 msgr2=0x7f1f0c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f0b7fe700 1 --2- 172.21.15.81:0/1941614825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f0c06a8a0 0x7f1f0c0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f0b7fe700 1 -- 172.21.15.81:0/1941614825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f0c15cfa0 msgr2=0x7f1f0c1f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.445+0000 7f1f0b7fe700 1 --2- 172.21.15.81:0/1941614825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f0c15cfa0 0x7f1f0c1f70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.449+0000 7f1f0b7fe700 1 -- 172.21.15.81:0/1941614825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc0094b0 con 0x7f1f0c0aed90 2024-05-08T23:30:38.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.449+0000 7f1f0b7fe700 1 --2- 172.21.15.81:0/1941614825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f0c0aed90 0x7f1f0c14b790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f000090b0 tx=0x7f1f00004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:38.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.449+0000 7f1f08ff9700 1 -- 172.21.15.81:0/1941614825 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00004590 con 0x7f1f0c0aed90 2024-05-08T23:30:38.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.449+0000 7f1f08ff9700 1 -- 172.21.15.81:0/1941614825 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0001f070 con 0x7f1f0c0aed90 2024-05-08T23:30:38.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.449+0000 7f1f115d2700 1 -- 172.21.15.81:0/1941614825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f000094b0 con 0x7f1f0c0aed90 2024-05-08T23:30:38.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.449+0000 7f1f08ff9700 1 -- 172.21.15.81:0/1941614825 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00003900 con 0x7f1f0c0aed90 2024-05-08T23:30:38.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.449+0000 7f1f115d2700 1 -- 172.21.15.81:0/1941614825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c15d6c0 con 0x7f1f0c0aed90 2024-05-08T23:30:38.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.449+0000 7f1f115d2700 1 -- 172.21.15.81:0/1941614825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c0672a0 con 0x7f1f0c0aed90 2024-05-08T23:30:38.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.449+0000 7f1f08ff9700 1 -- 172.21.15.81:0/1941614825 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f00003a60 con 0x7f1f0c0aed90 2024-05-08T23:30:38.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.453+0000 7f1f08ff9700 1 --2- 172.21.15.81:0/1941614825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1ef8066730 0x7f1ef8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.453+0000 7f1f08ff9700 1 -- 172.21.15.81:0/1941614825 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f1f00021070 con 0x7f1f0c0aed90 2024-05-08T23:30:38.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.453+0000 7f1f0bfff700 1 --2- 172.21.15.81:0/1941614825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1ef8066730 0x7f1ef8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.453+0000 7f1f0bfff700 1 --2- 172.21.15.81:0/1941614825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1ef8066730 0x7f1ef8068bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009d80 tx=0x7f1efc009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:38.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.453+0000 7f1f08ff9700 1 -- 172.21.15.81:0/1941614825 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1f00052dc0 con 0x7f1f0c0aed90 2024-05-08T23:30:38.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.657+0000 7f1f115d2700 1 -- 172.21.15.81:0/1941614825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1f0c06d3c0 con 0x7f1f0c0aed90 2024-05-08T23:30:38.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.665+0000 7f1f08ff9700 1 -- 172.21.15.81:0/1941614825 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1f000150c0 con 0x7f1f0c0aed90 2024-05-08T23:30:38.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.673+0000 7f1f115d2700 1 -- 172.21.15.81:0/1941614825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1ef8066730 msgr2=0x7f1ef8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.673+0000 7f1f115d2700 1 --2- 172.21.15.81:0/1941614825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1ef8066730 0x7f1ef8068bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009d80 tx=0x7f1efc009240 comp rx=0 tx=0).stop 2024-05-08T23:30:38.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.673+0000 7f1f115d2700 1 -- 172.21.15.81:0/1941614825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f0c0aed90 msgr2=0x7f1f0c14b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.673+0000 7f1f115d2700 1 --2- 172.21.15.81:0/1941614825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f0c0aed90 0x7f1f0c14b790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f000090b0 tx=0x7f1f00004770 comp rx=0 tx=0).stop 2024-05-08T23:30:38.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.673+0000 7f1f115d2700 1 -- 172.21.15.81:0/1941614825 shutdown_connections 2024-05-08T23:30:38.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.673+0000 7f1f115d2700 1 --2- 172.21.15.81:0/1941614825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1ef8066730 0x7f1ef8068bf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.673+0000 7f1f115d2700 1 --2- 172.21.15.81:0/1941614825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f0c15cfa0 0x7f1f0c1f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.673+0000 7f1f115d2700 1 --2- 172.21.15.81:0/1941614825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f0c0aed90 0x7f1f0c14b790 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.673+0000 7f1f115d2700 1 --2- 172.21.15.81:0/1941614825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f0c06a8a0 0x7f1f0c0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.673+0000 7f1f115d2700 1 -- 172.21.15.81:0/1941614825 >> 172.21.15.81:0/1941614825 conn(0x7f1f0c06b470 msgr2=0x7f1f0c06e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:38.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.673+0000 7f1f115d2700 1 -- 172.21.15.81:0/1941614825 shutdown_connections 2024-05-08T23:30:38.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.673+0000 7f1f115d2700 1 -- 172.21.15.81:0/1941614825 wait complete. 2024-05-08T23:30:38.680 INFO:teuthology.orchestra.run.smithi081.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T23:30:38.697 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:30:38.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f831efb5700 1 Processor -- start 2024-05-08T23:30:38.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f831efb5700 1 -- start start 2024-05-08T23:30:38.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f831efb5700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831815aed0 0x7f83180cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f831efb5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f83180cb970 0x7f83180b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f831efb5700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83180cd470 con 0x7f83180b2e40 2024-05-08T23:30:38.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f831efb5700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83180ced50 con 0x7f83180cb970 2024-05-08T23:30:38.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f831efb5700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83180cceb0 con 0x7f831815aed0 2024-05-08T23:30:38.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f830ffff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f83180cb970 0x7f83180b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f830ffff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f83180cb970 0x7f83180b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32848/0 (socket says 172.21.15.81:32848) 2024-05-08T23:30:38.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f830ffff700 1 -- 172.21.15.81:0/1707489229 learned_addr learned my addr 172.21.15.81:0/1707489229 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:38.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f830ffff700 1 -- 172.21.15.81:0/1707489229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831815aed0 msgr2=0x7f83180cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:38.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f830ffff700 1 --2- 172.21.15.81:0/1707489229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831815aed0 0x7f83180cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f830ffff700 1 -- 172.21.15.81:0/1707489229 >> v1:172.21.15.117:6789/0 conn(0x7f83180b2e40 legacy=0x7f831806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:38.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f830ffff700 1 -- 172.21.15.81:0/1707489229 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83180cd190 con 0x7f83180cb970 2024-05-08T23:30:38.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f830ffff700 1 --2- 172.21.15.81:0/1707489229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83180cb970 0x7f83180b2900 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8308009910 tx=0x7f8308009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1ae2025f6e566f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:38.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f830f7fe700 1 -- 172.21.15.81:0/1707489229 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308004230 con 0x7f83180cb970 2024-05-08T23:30:38.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f830f7fe700 1 -- 172.21.15.81:0/1707489229 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8308004390 con 0x7f83180cb970 2024-05-08T23:30:38.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f830f7fe700 1 -- 172.21.15.81:0/1707489229 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308009e60 con 0x7f83180cb970 2024-05-08T23:30:38.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f831efb5700 1 -- 172.21.15.81:0/1707489229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83180cb970 msgr2=0x7f83180b2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f831efb5700 1 --2- 172.21.15.81:0/1707489229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83180cb970 0x7f83180b2900 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8308009910 tx=0x7f8308009c20 comp rx=0 tx=0).stop 2024-05-08T23:30:38.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f831efb5700 1 -- 172.21.15.81:0/1707489229 shutdown_connections 2024-05-08T23:30:38.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f831efb5700 1 --2- 172.21.15.81:0/1707489229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83180cb970 0x7f83180b2900 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f831efb5700 1 --2- 172.21.15.81:0/1707489229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831815aed0 0x7f83180cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f831efb5700 1 -- 172.21.15.81:0/1707489229 >> 172.21.15.81:0/1707489229 conn(0x7f831806a360 msgr2=0x7f831806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:38.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f831efb5700 1 -- 172.21.15.81:0/1707489229 shutdown_connections 2024-05-08T23:30:38.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.793+0000 7f831efb5700 1 -- 172.21.15.81:0/1707489229 wait complete. 2024-05-08T23:30:38.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f831efb5700 1 Processor -- start 2024-05-08T23:30:38.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f831efb5700 1 -- start start 2024-05-08T23:30:38.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f831efb5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83180b2e40 0x7f831814fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f831efb5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83180cb970 0x7f83181e04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f831efb5700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831815aed0 0x7f8318154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f831efb5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83181e1aa0 con 0x7f83180cb970 2024-05-08T23:30:38.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f831efb5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8318150070 con 0x7f83180b2e40 2024-05-08T23:30:38.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f831efb5700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83181e0f20 con 0x7f831815aed0 2024-05-08T23:30:38.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f830ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83180cb970 0x7f83181e04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f830ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83180cb970 0x7f83181e04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32850/0 (socket says 172.21.15.81:32850) 2024-05-08T23:30:38.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f830ffff700 1 -- 172.21.15.81:0/534210945 learned_addr learned my addr 172.21.15.81:0/534210945 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:38.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f831d552700 1 --2- 172.21.15.81:0/534210945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831815aed0 0x7f8318154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f831cd51700 1 --2- 172.21.15.81:0/534210945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83180b2e40 0x7f831814fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f830ffff700 1 -- 172.21.15.81:0/534210945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831815aed0 msgr2=0x7f8318154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f830ffff700 1 --2- 172.21.15.81:0/534210945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831815aed0 0x7f8318154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f830ffff700 1 -- 172.21.15.81:0/534210945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83180b2e40 msgr2=0x7f831814fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f830ffff700 1 --2- 172.21.15.81:0/534210945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83180b2e40 0x7f831814fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f830ffff700 1 -- 172.21.15.81:0/534210945 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8308009570 con 0x7f83180cb970 2024-05-08T23:30:38.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f831cd51700 1 --2- 172.21.15.81:0/534210945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83180b2e40 0x7f831814fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:38.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.797+0000 7f830ffff700 1 --2- 172.21.15.81:0/534210945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83180cb970 0x7f83181e04f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8308004000 tx=0x7f8308004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:38.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.801+0000 7f830d7fa700 1 -- 172.21.15.81:0/534210945 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830800bc80 con 0x7f83180cb970 2024-05-08T23:30:38.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.801+0000 7f830d7fa700 1 -- 172.21.15.81:0/534210945 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830801f070 con 0x7f83180cb970 2024-05-08T23:30:38.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.801+0000 7f831efb5700 1 -- 172.21.15.81:0/534210945 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83181e1200 con 0x7f83180cb970 2024-05-08T23:30:38.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.801+0000 7f830d7fa700 1 -- 172.21.15.81:0/534210945 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83080036c0 con 0x7f83180cb970 2024-05-08T23:30:38.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.801+0000 7f831efb5700 1 -- 172.21.15.81:0/534210945 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83181e17c0 con 0x7f83180cb970 2024-05-08T23:30:38.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.801+0000 7f831efb5700 1 -- 172.21.15.81:0/534210945 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83180684b0 con 0x7f83180cb970 2024-05-08T23:30:38.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.801+0000 7f830d7fa700 1 -- 172.21.15.81:0/534210945 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8308021030 con 0x7f83180cb970 2024-05-08T23:30:38.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.801+0000 7f830d7fa700 1 --2- 172.21.15.81:0/534210945 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8304066730 0x7f8304068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.801+0000 7f830d7fa700 1 -- 172.21.15.81:0/534210945 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8308085820 con 0x7f83180cb970 2024-05-08T23:30:38.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.801+0000 7f831cd51700 1 --2- 172.21.15.81:0/534210945 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8304066730 0x7f8304068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.805+0000 7f831cd51700 1 --2- 172.21.15.81:0/534210945 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8304066730 0x7f8304068bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8314006fd0 tx=0x7f8314008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:38.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.809+0000 7f830d7fa700 1 -- 172.21.15.81:0/534210945 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8308052ba0 con 0x7f83180cb970 2024-05-08T23:30:38.989 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.985+0000 7f831efb5700 1 -- 172.21.15.81:0/534210945 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83180684b0 con 0x7f83180cb970 2024-05-08T23:30:38.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.989+0000 7f830d7fa700 1 -- 172.21.15.81:0/534210945 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16050 (secure 0 0 0) 0x7f8308085020 con 0x7f83180cb970 2024-05-08T23:30:38.993 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:38.993 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":21,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:38.664445+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:30:38.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.989+0000 7f831efb5700 1 -- 172.21.15.81:0/534210945 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8304066730 msgr2=0x7f8304068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.989+0000 7f831efb5700 1 --2- 172.21.15.81:0/534210945 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8304066730 0x7f8304068bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8314006fd0 tx=0x7f8314008040 comp rx=0 tx=0).stop 2024-05-08T23:30:38.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.989+0000 7f831efb5700 1 -- 172.21.15.81:0/534210945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83180cb970 msgr2=0x7f83181e04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.989+0000 7f831efb5700 1 --2- 172.21.15.81:0/534210945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83180cb970 0x7f83181e04f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8308004000 tx=0x7f8308004770 comp rx=0 tx=0).stop 2024-05-08T23:30:38.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.989+0000 7f831efb5700 1 -- 172.21.15.81:0/534210945 shutdown_connections 2024-05-08T23:30:38.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.989+0000 7f831efb5700 1 --2- 172.21.15.81:0/534210945 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8304066730 0x7f8304068bf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.989+0000 7f831efb5700 1 --2- 172.21.15.81:0/534210945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831815aed0 0x7f8318154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.989+0000 7f831efb5700 1 --2- 172.21.15.81:0/534210945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83180cb970 0x7f83181e04f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.989+0000 7f831efb5700 1 --2- 172.21.15.81:0/534210945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83180b2e40 0x7f831814fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.989+0000 7f831efb5700 1 -- 172.21.15.81:0/534210945 >> 172.21.15.81:0/534210945 conn(0x7f831806a360 msgr2=0x7f83180ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:38.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.989+0000 7f831efb5700 1 -- 172.21.15.81:0/534210945 shutdown_connections 2024-05-08T23:30:38.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:38.989+0000 7f831efb5700 1 -- 172.21.15.81:0/534210945 wait complete. 2024-05-08T23:30:39.008 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:30:39.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff387fff700 1 Processor -- start 2024-05-08T23:30:39.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff387fff700 1 -- start start 2024-05-08T23:30:39.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff387fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff3880b3c20 0x7ff3881624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff387fff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff388158960 con 0x7ff388070110 2024-05-08T23:30:39.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff387fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3880ae990 con 0x7ff388071e70 2024-05-08T23:30:39.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff387fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3880b1f60 con 0x7ff3880b3c20 2024-05-08T23:30:39.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff3877fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff388070110 0x7ff3880705a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54992/0 (socket says 172.21.15.81:54992) 2024-05-08T23:30:39.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff3877fe700 1 -- 172.21.15.81:0/1400132122 learned_addr learned my addr 172.21.15.81:0/1400132122 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:39.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff386ffd700 1 --2- 172.21.15.81:0/1400132122 >> v2:172.21.15.117:3300/0 conn(0x7ff3880b3c20 0x7ff3881624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff385ffb700 1 -- 172.21.15.81:0/1400132122 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38122380 0 0) 0x7ff388158960 con 0x7ff388070110 2024-05-08T23:30:39.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff385ffb700 1 -- 172.21.15.81:0/1400132122 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3740035d0 con 0x7ff388070110 2024-05-08T23:30:39.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff386ffd700 1 -- 172.21.15.81:0/1400132122 >> v1:172.21.15.117:6789/0 conn(0x7ff388071e70 legacy=0x7ff3880b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff386ffd700 1 -- 172.21.15.81:0/1400132122 >> v1:172.21.15.81:6789/0 conn(0x7ff388070110 legacy=0x7ff3880705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff386ffd700 1 -- 172.21.15.81:0/1400132122 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3880b2170 con 0x7ff3880b3c20 2024-05-08T23:30:39.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff386ffd700 1 --2- 172.21.15.81:0/1400132122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3880b3c20 0x7ff3881624b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff370009850 tx=0x7ff370009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18f83e8abfc39072 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff385ffb700 1 -- 172.21.15.81:0/1400132122 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff370011830 con 0x7ff3880b3c20 2024-05-08T23:30:39.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff385ffb700 1 -- 172.21.15.81:0/1400132122 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff370011990 con 0x7ff3880b3c20 2024-05-08T23:30:39.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.113+0000 7ff385ffb700 1 -- 172.21.15.81:0/1400132122 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff370011d00 con 0x7ff3880b3c20 2024-05-08T23:30:39.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff387fff700 1 -- 172.21.15.81:0/1400132122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3880b3c20 msgr2=0x7ff3881624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff387fff700 1 --2- 172.21.15.81:0/1400132122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3880b3c20 0x7ff3881624b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff370009850 tx=0x7ff370009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:39.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff387fff700 1 -- 172.21.15.81:0/1400132122 shutdown_connections 2024-05-08T23:30:39.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff387fff700 1 --2- 172.21.15.81:0/1400132122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3880b3c20 0x7ff3881624b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff387fff700 1 -- 172.21.15.81:0/1400132122 >> 172.21.15.81:0/1400132122 conn(0x7ff38806c330 msgr2=0x7ff38806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff387fff700 1 -- 172.21.15.81:0/1400132122 shutdown_connections 2024-05-08T23:30:39.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff387fff700 1 -- 172.21.15.81:0/1400132122 wait complete. 2024-05-08T23:30:39.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff387fff700 1 Processor -- start 2024-05-08T23:30:39.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff387fff700 1 -- start start 2024-05-08T23:30:39.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff387fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff388070110 0x7ff3881654e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff387fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff388071e70 0x7ff3881404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff387fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3880b3c20 0x7ff388140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff387fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff388151ae0 con 0x7ff3880b3c20 2024-05-08T23:30:39.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff387fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff388165a20 con 0x7ff388070110 2024-05-08T23:30:39.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff387fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff388141110 con 0x7ff388071e70 2024-05-08T23:30:39.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff3877fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3880b3c20 0x7ff388140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff3877fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3880b3c20 0x7ff388140a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32866/0 (socket says 172.21.15.81:32866) 2024-05-08T23:30:39.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff3877fe700 1 -- 172.21.15.81:0/973615093 learned_addr learned my addr 172.21.15.81:0/973615093 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:39.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff3867fc700 1 --2- 172.21.15.81:0/973615093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff388071e70 0x7ff3881404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.117+0000 7ff386ffd700 1 --2- 172.21.15.81:0/973615093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff388070110 0x7ff3881654e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.121+0000 7ff3877fe700 1 -- 172.21.15.81:0/973615093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff388071e70 msgr2=0x7ff3881404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.121+0000 7ff3877fe700 1 --2- 172.21.15.81:0/973615093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff388071e70 0x7ff3881404c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.121+0000 7ff3877fe700 1 -- 172.21.15.81:0/973615093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff388070110 msgr2=0x7ff3881654e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.121+0000 7ff3877fe700 1 --2- 172.21.15.81:0/973615093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff388070110 0x7ff3881654e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.121+0000 7ff3877fe700 1 -- 172.21.15.81:0/973615093 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3700094b0 con 0x7ff3880b3c20 2024-05-08T23:30:39.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.121+0000 7ff3877fe700 1 --2- 172.21.15.81:0/973615093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3880b3c20 0x7ff388140a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff37800c500 tx=0x7ff37800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.121+0000 7ff37ffff700 1 -- 172.21.15.81:0/973615093 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff378005d90 con 0x7ff3880b3c20 2024-05-08T23:30:39.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.121+0000 7ff37ffff700 1 -- 172.21.15.81:0/973615093 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37800ce70 con 0x7ff3880b3c20 2024-05-08T23:30:39.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.121+0000 7ff387fff700 1 -- 172.21.15.81:0/973615093 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff388141c90 con 0x7ff3880b3c20 2024-05-08T23:30:39.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.121+0000 7ff37ffff700 1 -- 172.21.15.81:0/973615093 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3780072d0 con 0x7ff3880b3c20 2024-05-08T23:30:39.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.121+0000 7ff387fff700 1 -- 172.21.15.81:0/973615093 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff388141640 con 0x7ff3880b3c20 2024-05-08T23:30:39.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.121+0000 7ff387fff700 1 -- 172.21.15.81:0/973615093 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff388141e20 con 0x7ff3880b3c20 2024-05-08T23:30:39.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.121+0000 7ff37ffff700 1 -- 172.21.15.81:0/973615093 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff378007430 con 0x7ff3880b3c20 2024-05-08T23:30:39.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.125+0000 7ff37ffff700 1 --2- 172.21.15.81:0/973615093 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff368066730 0x7ff368068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.125+0000 7ff386ffd700 1 --2- 172.21.15.81:0/973615093 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff368066730 0x7ff368068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.125+0000 7ff37ffff700 1 -- 172.21.15.81:0/973615093 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3780839e0 con 0x7ff3880b3c20 2024-05-08T23:30:39.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.125+0000 7ff386ffd700 1 --2- 172.21.15.81:0/973615093 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff368066730 0x7ff368068bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff3700090b0 tx=0x7ff370008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.129+0000 7ff37ffff700 1 -- 172.21.15.81:0/973615093 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff378050ca0 con 0x7ff3880b3c20 2024-05-08T23:30:39.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.301+0000 7ff387fff700 1 -- 172.21.15.81:0/973615093 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3880684b0 con 0x7ff3880b3c20 2024-05-08T23:30:39.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.305+0000 7ff37ffff700 1 -- 172.21.15.81:0/973615093 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16050 (secure 0 0 0) 0x7ff378017020 con 0x7ff3880b3c20 2024-05-08T23:30:39.310 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:39.310 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":21,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:38.664445+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:30:39.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.309+0000 7ff387fff700 1 -- 172.21.15.81:0/973615093 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff368066730 msgr2=0x7ff368068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.309+0000 7ff387fff700 1 --2- 172.21.15.81:0/973615093 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff368066730 0x7ff368068bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff3700090b0 tx=0x7ff370008030 comp rx=0 tx=0).stop 2024-05-08T23:30:39.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.309+0000 7ff387fff700 1 -- 172.21.15.81:0/973615093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3880b3c20 msgr2=0x7ff388140a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.309+0000 7ff387fff700 1 --2- 172.21.15.81:0/973615093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3880b3c20 0x7ff388140a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff37800c500 tx=0x7ff37800c810 comp rx=0 tx=0).stop 2024-05-08T23:30:39.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.309+0000 7ff387fff700 1 -- 172.21.15.81:0/973615093 shutdown_connections 2024-05-08T23:30:39.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.309+0000 7ff387fff700 1 --2- 172.21.15.81:0/973615093 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff368066730 0x7ff368068bf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.309+0000 7ff387fff700 1 --2- 172.21.15.81:0/973615093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3880b3c20 0x7ff388140a00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.309+0000 7ff387fff700 1 --2- 172.21.15.81:0/973615093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff388071e70 0x7ff3881404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.309+0000 7ff387fff700 1 --2- 172.21.15.81:0/973615093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff388070110 0x7ff3881654e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.309+0000 7ff387fff700 1 -- 172.21.15.81:0/973615093 >> 172.21.15.81:0/973615093 conn(0x7ff38806c330 msgr2=0x7ff388157500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.309+0000 7ff387fff700 1 -- 172.21.15.81:0/973615093 shutdown_connections 2024-05-08T23:30:39.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.309+0000 7ff387fff700 1 -- 172.21.15.81:0/973615093 wait complete. 2024-05-08T23:30:39.329 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T23:30:39.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.445+0000 7fa2e56a1700 1 Processor -- start 2024-05-08T23:30:39.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.445+0000 7fa2e56a1700 1 -- start start 2024-05-08T23:30:39.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.445+0000 7fa2e56a1700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2e00d1da0 0x7fa2e00cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.445+0000 7fa2e56a1700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa2e00cfe80 0x7fa2e00b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.445+0000 7fa2e56a1700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e00d3d40 con 0x7fa2e0160920 2024-05-08T23:30:39.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.445+0000 7fa2e56a1700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e00d0310 con 0x7fa2e00cfe80 2024-05-08T23:30:39.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.445+0000 7fa2e56a1700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00d0d00 con 0x7fa2e00d1da0 2024-05-08T23:30:39.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.445+0000 7fa2e4ea0700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa2e00cfe80 0x7fa2e00b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.445+0000 7fa2df7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2e00d1da0 0x7fa2e00cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2e4ea0700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa2e00cfe80 0x7fa2e00b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:44048/0 (socket says 172.21.15.81:44048) 2024-05-08T23:30:39.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2e4ea0700 1 -- 172.21.15.81:0/3465993087 learned_addr learned my addr 172.21.15.81:0/3465993087 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:39.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2df7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2e00d1da0 0x7fa2e00cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:52942/0 (socket says 172.21.15.81:52942) 2024-05-08T23:30:39.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2e4ea0700 1 -- 172.21.15.81:0/3465993087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2e00d1da0 msgr2=0x7fa2e00cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2e4ea0700 1 --2- 172.21.15.81:0/3465993087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2e00d1da0 0x7fa2e00cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2e4ea0700 1 -- 172.21.15.81:0/3465993087 >> v1:172.21.15.117:6789/0 conn(0x7fa2e0160920 legacy=0x7fa2e00ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2e4ea0700 1 -- 172.21.15.81:0/3465993087 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e00d0fe0 con 0x7fa2e00cfe80 2024-05-08T23:30:39.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2df7fe700 1 --2- 172.21.15.81:0/3465993087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2e00d1da0 0x7fa2e00cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:39.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2e4ea0700 1 --2- 172.21.15.81:0/3465993087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e00cfe80 0x7fa2e00b5e50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009e50 tx=0x7fa2d800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=819682ad7a62df1f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2deffd700 1 -- 172.21.15.81:0/3465993087 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800e070 con 0x7fa2e00cfe80 2024-05-08T23:30:39.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2deffd700 1 -- 172.21.15.81:0/3465993087 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d800b910 con 0x7fa2e00cfe80 2024-05-08T23:30:39.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2deffd700 1 -- 172.21.15.81:0/3465993087 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800bcc0 con 0x7fa2e00cfe80 2024-05-08T23:30:39.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2e56a1700 1 -- 172.21.15.81:0/3465993087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e00cfe80 msgr2=0x7fa2e00b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2e56a1700 1 --2- 172.21.15.81:0/3465993087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e00cfe80 0x7fa2e00b5e50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009e50 tx=0x7fa2d800b330 comp rx=0 tx=0).stop 2024-05-08T23:30:39.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2e56a1700 1 -- 172.21.15.81:0/3465993087 shutdown_connections 2024-05-08T23:30:39.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2e56a1700 1 --2- 172.21.15.81:0/3465993087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e00cfe80 0x7fa2e00b5e50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2e56a1700 1 --2- 172.21.15.81:0/3465993087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2e00d1da0 0x7fa2e00cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2e56a1700 1 -- 172.21.15.81:0/3465993087 >> 172.21.15.81:0/3465993087 conn(0x7fa2e009c340 msgr2=0x7fa2e00d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2e56a1700 1 -- 172.21.15.81:0/3465993087 shutdown_connections 2024-05-08T23:30:39.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.449+0000 7fa2e56a1700 1 -- 172.21.15.81:0/3465993087 wait complete. 2024-05-08T23:30:39.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2e56a1700 1 Processor -- start 2024-05-08T23:30:39.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2e56a1700 1 -- start start 2024-05-08T23:30:39.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2e56a1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2e00cfe80 0x7fa2e0157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2e56a1700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2e00d1da0 0x7fa2e00b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2e56a1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e0160920 0x7fa2e00b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2e56a1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00b39d0 con 0x7fa2e00cfe80 2024-05-08T23:30:39.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2e56a1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00b2930 con 0x7fa2e0160920 2024-05-08T23:30:39.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2e56a1700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00b2c10 con 0x7fa2e00d1da0 2024-05-08T23:30:39.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2dffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2e00cfe80 0x7fa2e0157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2dffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2e00cfe80 0x7fa2e0157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32872/0 (socket says 172.21.15.81:32872) 2024-05-08T23:30:39.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2dffff700 1 -- 172.21.15.81:0/2445356116 learned_addr learned my addr 172.21.15.81:0/2445356116 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:39.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2e4ea0700 1 --2- 172.21.15.81:0/2445356116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e0160920 0x7fa2e00b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2df7fe700 1 --2- 172.21.15.81:0/2445356116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2e00d1da0 0x7fa2e00b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2dffff700 1 -- 172.21.15.81:0/2445356116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2e00d1da0 msgr2=0x7fa2e00b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2dffff700 1 --2- 172.21.15.81:0/2445356116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2e00d1da0 0x7fa2e00b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2dffff700 1 -- 172.21.15.81:0/2445356116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e0160920 msgr2=0x7fa2e00b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2dffff700 1 --2- 172.21.15.81:0/2445356116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e0160920 0x7fa2e00b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2dffff700 1 -- 172.21.15.81:0/2445356116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d00094b0 con 0x7fa2e00cfe80 2024-05-08T23:30:39.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2dffff700 1 --2- 172.21.15.81:0/2445356116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2e00cfe80 0x7fa2e0157190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400c410 tx=0x7fa2d400c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2dcff9700 1 -- 172.21.15.81:0/2445356116 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40052d0 con 0x7fa2e00cfe80 2024-05-08T23:30:39.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2e56a1700 1 -- 172.21.15.81:0/2445356116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d8009b30 con 0x7fa2e00cfe80 2024-05-08T23:30:39.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2dcff9700 1 -- 172.21.15.81:0/2445356116 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d4005430 con 0x7fa2e00cfe80 2024-05-08T23:30:39.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2dcff9700 1 -- 172.21.15.81:0/2445356116 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40055a0 con 0x7fa2e00cfe80 2024-05-08T23:30:39.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.453+0000 7fa2e56a1700 1 -- 172.21.15.81:0/2445356116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e00b31d0 con 0x7fa2e00cfe80 2024-05-08T23:30:39.461 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.457+0000 7fa2e56a1700 1 -- 172.21.15.81:0/2445356116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e00684b0 con 0x7fa2e00cfe80 2024-05-08T23:30:39.461 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.457+0000 7fa2dcff9700 1 -- 172.21.15.81:0/2445356116 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa2d40036a0 con 0x7fa2e00cfe80 2024-05-08T23:30:39.461 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.457+0000 7fa2dcff9700 1 --2- 172.21.15.81:0/2445356116 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa2cc066730 0x7fa2cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.457+0000 7fa2dcff9700 1 -- 172.21.15.81:0/2445356116 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2d40832f0 con 0x7fa2e00cfe80 2024-05-08T23:30:39.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.457+0000 7fa2df7fe700 1 --2- 172.21.15.81:0/2445356116 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa2cc066730 0x7fa2cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.457+0000 7fa2df7fe700 1 --2- 172.21.15.81:0/2445356116 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa2cc066730 0x7fa2cc068bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00c9630 tx=0x7fa2d0009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.465+0000 7fa2dcff9700 1 -- 172.21.15.81:0/2445356116 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa2d40505e0 con 0x7fa2e00cfe80 2024-05-08T23:30:39.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:39.677+0000 7fa2e56a1700 1 -- 172.21.15.81:0/2445356116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa2e006cd80 con 0x7fa2e00cfe80 2024-05-08T23:30:40.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.673+0000 7fa2dcff9700 1 -- 172.21.15.81:0/2445356116 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa2d4070e60 con 0x7fa2e00cfe80 2024-05-08T23:30:40.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.677+0000 7fa2e56a1700 1 -- 172.21.15.81:0/2445356116 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa2cc066730 msgr2=0x7fa2cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:40.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.677+0000 7fa2e56a1700 1 --2- 172.21.15.81:0/2445356116 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa2cc066730 0x7fa2cc068bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00c9630 tx=0x7fa2d0009240 comp rx=0 tx=0).stop 2024-05-08T23:30:40.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.677+0000 7fa2e56a1700 1 -- 172.21.15.81:0/2445356116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2e00cfe80 msgr2=0x7fa2e0157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:40.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.677+0000 7fa2e56a1700 1 --2- 172.21.15.81:0/2445356116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2e00cfe80 0x7fa2e0157190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400c410 tx=0x7fa2d400c720 comp rx=0 tx=0).stop 2024-05-08T23:30:40.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.677+0000 7fa2e56a1700 1 -- 172.21.15.81:0/2445356116 shutdown_connections 2024-05-08T23:30:40.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.677+0000 7fa2e56a1700 1 --2- 172.21.15.81:0/2445356116 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa2cc066730 0x7fa2cc068bf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:40.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.677+0000 7fa2e56a1700 1 --2- 172.21.15.81:0/2445356116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2e0160920 0x7fa2e00b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:40.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.677+0000 7fa2e56a1700 1 --2- 172.21.15.81:0/2445356116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2e00d1da0 0x7fa2e00b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:40.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.677+0000 7fa2e56a1700 1 --2- 172.21.15.81:0/2445356116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2e00cfe80 0x7fa2e0157190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:40.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.677+0000 7fa2e56a1700 1 -- 172.21.15.81:0/2445356116 >> 172.21.15.81:0/2445356116 conn(0x7fa2e009c340 msgr2=0x7fa2e015f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:40.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.677+0000 7fa2e56a1700 1 -- 172.21.15.81:0/2445356116 shutdown_connections 2024-05-08T23:30:40.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.677+0000 7fa2e56a1700 1 -- 172.21.15.81:0/2445356116 wait complete. 2024-05-08T23:30:40.697 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T23:30:40.697 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T23:30:40.826 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.821+0000 7f119d5f0700 1 Processor -- start 2024-05-08T23:30:40.826 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.821+0000 7f119d5f0700 1 -- start start 2024-05-08T23:30:40.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.821+0000 7f119d5f0700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f11980d1da0 0x7f11980cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:40.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.821+0000 7f119d5f0700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f11980cfe80 0x7f11980b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:40.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.821+0000 7f119d5f0700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11980d3d40 con 0x7f1198160920 2024-05-08T23:30:40.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.821+0000 7f119d5f0700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11980d02d0 con 0x7f11980d1da0 2024-05-08T23:30:40.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.821+0000 7f119d5f0700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11980d0d00 con 0x7f11980cfe80 2024-05-08T23:30:40.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f11977fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f11980d1da0 0x7f11980cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:40.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f11977fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f11980d1da0 0x7f11980cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32884/0 (socket says 172.21.15.81:32884) 2024-05-08T23:30:40.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f11977fe700 1 -- 172.21.15.81:0/2606177242 learned_addr learned my addr 172.21.15.81:0/2606177242 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:40.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f119cdef700 1 --2- 172.21.15.81:0/2606177242 >> v2:172.21.15.117:3300/0 conn(0x7f11980cfe80 0x7f11980b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:40.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f11977fe700 1 -- 172.21.15.81:0/2606177242 >> v2:172.21.15.117:3300/0 conn(0x7f11980cfe80 msgr2=0x7f11980b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:40.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f11977fe700 1 --2- 172.21.15.81:0/2606177242 >> v2:172.21.15.117:3300/0 conn(0x7f11980cfe80 0x7f11980b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:40.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f11977fe700 1 -- 172.21.15.81:0/2606177242 >> v1:172.21.15.117:6789/0 conn(0x7f1198160920 legacy=0x7f11980ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:40.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f11977fe700 1 -- 172.21.15.81:0/2606177242 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11980d0fe0 con 0x7f11980d1da0 2024-05-08T23:30:40.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f11977fe700 1 --2- 172.21.15.81:0/2606177242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11980d1da0 0x7f11980cf940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f11980cd6e0 tx=0x7f1188009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b3850760f9a26f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:40.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f1196ffd700 1 -- 172.21.15.81:0/2606177242 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188011830 con 0x7f11980d1da0 2024-05-08T23:30:40.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f1196ffd700 1 -- 172.21.15.81:0/2606177242 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1188011990 con 0x7f11980d1da0 2024-05-08T23:30:40.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f1196ffd700 1 -- 172.21.15.81:0/2606177242 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188011d20 con 0x7f11980d1da0 2024-05-08T23:30:40.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f119d5f0700 1 -- 172.21.15.81:0/2606177242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11980d1da0 msgr2=0x7f11980cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:40.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f119d5f0700 1 --2- 172.21.15.81:0/2606177242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11980d1da0 0x7f11980cf940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f11980cd6e0 tx=0x7f1188009bf0 comp rx=0 tx=0).stop 2024-05-08T23:30:40.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f119d5f0700 1 -- 172.21.15.81:0/2606177242 shutdown_connections 2024-05-08T23:30:40.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f119d5f0700 1 --2- 172.21.15.81:0/2606177242 >> v2:172.21.15.117:3300/0 conn(0x7f11980cfe80 0x7f11980b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:40.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f119d5f0700 1 --2- 172.21.15.81:0/2606177242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11980d1da0 0x7f11980cf940 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:40.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f119d5f0700 1 -- 172.21.15.81:0/2606177242 >> 172.21.15.81:0/2606177242 conn(0x7f119809c340 msgr2=0x7f11980d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:40.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f119d5f0700 1 -- 172.21.15.81:0/2606177242 shutdown_connections 2024-05-08T23:30:40.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.825+0000 7f119d5f0700 1 -- 172.21.15.81:0/2606177242 wait complete. 2024-05-08T23:30:40.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f119d5f0700 1 Processor -- start 2024-05-08T23:30:40.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f119d5f0700 1 -- start start 2024-05-08T23:30:40.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f119d5f0700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11980cfe80 0x7f119815bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:40.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f119d5f0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11980d1da0 0x7f11980cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:40.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f119d5f0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1198160920 0x7f11980cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:40.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f119d5f0700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11980b2e10 con 0x7f11980d1da0 2024-05-08T23:30:40.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f119d5f0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119815c180 con 0x7f1198160920 2024-05-08T23:30:40.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f119d5f0700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11980b11f0 con 0x7f11980cfe80 2024-05-08T23:30:40.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f11977fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11980d1da0 0x7f11980cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:40.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f11977fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11980d1da0 0x7f11980cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32888/0 (socket says 172.21.15.81:32888) 2024-05-08T23:30:40.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f11977fe700 1 -- 172.21.15.81:0/798771838 learned_addr learned my addr 172.21.15.81:0/798771838 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:40.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f119cdef700 1 --2- 172.21.15.81:0/798771838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1198160920 0x7f11980cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:40.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f1197fff700 1 --2- 172.21.15.81:0/798771838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11980cfe80 0x7f119815bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:40.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f11977fe700 1 -- 172.21.15.81:0/798771838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11980cfe80 msgr2=0x7f119815bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:40.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f11977fe700 1 --2- 172.21.15.81:0/798771838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11980cfe80 0x7f119815bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:40.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f11977fe700 1 -- 172.21.15.81:0/798771838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1198160920 msgr2=0x7f11980cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:40.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f11977fe700 1 --2- 172.21.15.81:0/798771838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1198160920 0x7f11980cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:40.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f11977fe700 1 -- 172.21.15.81:0/798771838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1188009570 con 0x7f11980d1da0 2024-05-08T23:30:40.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f119cdef700 1 --2- 172.21.15.81:0/798771838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1198160920 0x7f11980cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:40.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f11977fe700 1 --2- 172.21.15.81:0/798771838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11980d1da0 0x7f11980cc0e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1188000c00 tx=0x7f1188004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:40.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f1194ff9700 1 -- 172.21.15.81:0/798771838 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188004610 con 0x7f11980d1da0 2024-05-08T23:30:40.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f1194ff9700 1 -- 172.21.15.81:0/798771838 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118801e070 con 0x7f11980d1da0 2024-05-08T23:30:40.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f1194ff9700 1 -- 172.21.15.81:0/798771838 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11880038e0 con 0x7f11980d1da0 2024-05-08T23:30:40.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f119d5f0700 1 -- 172.21.15.81:0/798771838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11980b14d0 con 0x7f11980d1da0 2024-05-08T23:30:40.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.829+0000 7f119d5f0700 1 -- 172.21.15.81:0/798771838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11980b1a90 con 0x7f11980d1da0 2024-05-08T23:30:40.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.833+0000 7f119d5f0700 1 -- 172.21.15.81:0/798771838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11980684b0 con 0x7f11980d1da0 2024-05-08T23:30:40.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.833+0000 7f1194ff9700 1 -- 172.21.15.81:0/798771838 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1188003a40 con 0x7f11980d1da0 2024-05-08T23:30:40.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.833+0000 7f1194ff9700 1 --2- 172.21.15.81:0/798771838 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1184066730 0x7f1184068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:40.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.837+0000 7f1197fff700 1 --2- 172.21.15.81:0/798771838 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1184066730 0x7f1184068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:40.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.837+0000 7f1194ff9700 1 -- 172.21.15.81:0/798771838 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1188084ca0 con 0x7f11980d1da0 2024-05-08T23:30:40.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.837+0000 7f1197fff700 1 --2- 172.21.15.81:0/798771838 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1184066730 0x7f1184068bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f118c000f60 tx=0x7f118c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:40.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:40.845+0000 7f1194ff9700 1 -- 172.21.15.81:0/798771838 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1188052120 con 0x7f11980d1da0 2024-05-08T23:30:41.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.049+0000 7f119d5f0700 1 -- 172.21.15.81:0/798771838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f11980cf510 con 0x7f11980d1da0 2024-05-08T23:30:41.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.673+0000 7f1194ff9700 1 -- 172.21.15.81:0/798771838 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f11880200c0 con 0x7f11980d1da0 2024-05-08T23:30:41.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.677+0000 7f119d5f0700 1 -- 172.21.15.81:0/798771838 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1184066730 msgr2=0x7f1184068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.677+0000 7f119d5f0700 1 --2- 172.21.15.81:0/798771838 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1184066730 0x7f1184068bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f118c000f60 tx=0x7f118c009040 comp rx=0 tx=0).stop 2024-05-08T23:30:41.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.677+0000 7f119d5f0700 1 -- 172.21.15.81:0/798771838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11980d1da0 msgr2=0x7f11980cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.677+0000 7f119d5f0700 1 --2- 172.21.15.81:0/798771838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11980d1da0 0x7f11980cc0e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1188000c00 tx=0x7f1188004770 comp rx=0 tx=0).stop 2024-05-08T23:30:41.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.677+0000 7f119d5f0700 1 -- 172.21.15.81:0/798771838 shutdown_connections 2024-05-08T23:30:41.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.677+0000 7f119d5f0700 1 --2- 172.21.15.81:0/798771838 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1184066730 0x7f1184068bf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.677+0000 7f119d5f0700 1 --2- 172.21.15.81:0/798771838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1198160920 0x7f11980cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.677+0000 7f119d5f0700 1 --2- 172.21.15.81:0/798771838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11980d1da0 0x7f11980cc0e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.677+0000 7f119d5f0700 1 --2- 172.21.15.81:0/798771838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11980cfe80 0x7f119815bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.677+0000 7f119d5f0700 1 -- 172.21.15.81:0/798771838 >> 172.21.15.81:0/798771838 conn(0x7f119809c340 msgr2=0x7f11980ce580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:41.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.677+0000 7f119d5f0700 1 -- 172.21.15.81:0/798771838 shutdown_connections 2024-05-08T23:30:41.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.677+0000 7f119d5f0700 1 -- 172.21.15.81:0/798771838 wait complete. 2024-05-08T23:30:41.701 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T23:30:41.701 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:30:41.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f9113f700 1 Processor -- start 2024-05-08T23:30:41.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f9113f700 1 -- start start 2024-05-08T23:30:41.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f9113f700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0f8c0d1690 0x7f0f8c0d37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f9113f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f8c0b2ea0 0x7f0f8c0b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f9113f700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f8c0ce870 con 0x7f0f8c0d0f20 2024-05-08T23:30:41.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f9113f700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8c0d3e50 con 0x7f0f8c0d1690 2024-05-08T23:30:41.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f9113f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0ceb50 con 0x7f0f8c0b2ea0 2024-05-08T23:30:41.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f9093e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f8c0b2ea0 0x7f0f8c0b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f8bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0f8c0d1690 0x7f0f8c0d37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f8bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0f8c0d1690 0x7f0f8c0d37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:44106/0 (socket says 172.21.15.81:44106) 2024-05-08T23:30:41.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f8bfff700 1 -- 172.21.15.81:0/70030337 learned_addr learned my addr 172.21.15.81:0/70030337 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:41.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f8bfff700 1 -- 172.21.15.81:0/70030337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f8c0b2ea0 msgr2=0x7f0f8c0b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f8bfff700 1 --2- 172.21.15.81:0/70030337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f8c0b2ea0 0x7f0f8c0b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f8bfff700 1 -- 172.21.15.81:0/70030337 >> v1:172.21.15.117:6789/0 conn(0x7f0f8c0d0f20 legacy=0x7f0f8c0d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f8bfff700 1 -- 172.21.15.81:0/70030337 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c0cee30 con 0x7f0f8c0d1690 2024-05-08T23:30:41.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f8bfff700 1 --2- 172.21.15.81:0/70030337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f8c0d1690 0x7f0f8c0d37c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009910 tx=0x7f0f7c009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c181e64be8688b4d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:41.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f8b7fe700 1 -- 172.21.15.81:0/70030337 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c011830 con 0x7f0f8c0d1690 2024-05-08T23:30:41.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f8b7fe700 1 -- 172.21.15.81:0/70030337 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c011990 con 0x7f0f8c0d1690 2024-05-08T23:30:41.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.809+0000 7f0f8b7fe700 1 -- 172.21.15.81:0/70030337 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c011d40 con 0x7f0f8c0d1690 2024-05-08T23:30:41.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 -- 172.21.15.81:0/70030337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f8c0d1690 msgr2=0x7f0f8c0d37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 --2- 172.21.15.81:0/70030337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f8c0d1690 0x7f0f8c0d37c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009910 tx=0x7f0f7c009c20 comp rx=0 tx=0).stop 2024-05-08T23:30:41.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 -- 172.21.15.81:0/70030337 shutdown_connections 2024-05-08T23:30:41.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 --2- 172.21.15.81:0/70030337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f8c0b2ea0 0x7f0f8c0b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 --2- 172.21.15.81:0/70030337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f8c0d1690 0x7f0f8c0d37c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 -- 172.21.15.81:0/70030337 >> 172.21.15.81:0/70030337 conn(0x7f0f8c09c3c0 msgr2=0x7f0f8c0d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:41.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 -- 172.21.15.81:0/70030337 shutdown_connections 2024-05-08T23:30:41.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 -- 172.21.15.81:0/70030337 wait complete. 2024-05-08T23:30:41.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 Processor -- start 2024-05-08T23:30:41.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 -- start start 2024-05-08T23:30:41.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f8c0b2ea0 0x7f0f8c1f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f8c0d0f20 0x7f0f8c0cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f8c0d1690 0x7f0f8c1603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0cc640 con 0x7f0f8c0d1690 2024-05-08T23:30:41.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c160930 con 0x7f0f8c0d0f20 2024-05-08T23:30:41.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.813+0000 7f0f9113f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c1f33a0 con 0x7f0f8c0b2ea0 2024-05-08T23:30:41.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f9093e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f8c0d1690 0x7f0f8c1603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f8bfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f8c0b2ea0 0x7f0f8c1f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f9093e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f8c0d1690 0x7f0f8c1603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32892/0 (socket says 172.21.15.81:32892) 2024-05-08T23:30:41.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f9093e700 1 -- 172.21.15.81:0/3953734535 learned_addr learned my addr 172.21.15.81:0/3953734535 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:41.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f837fe700 1 --2- 172.21.15.81:0/3953734535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f8c0d0f20 0x7f0f8c0cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f8bfff700 1 -- 172.21.15.81:0/3953734535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f8c0d0f20 msgr2=0x7f0f8c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f8bfff700 1 --2- 172.21.15.81:0/3953734535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f8c0d0f20 0x7f0f8c0cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f8bfff700 1 -- 172.21.15.81:0/3953734535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f8c0d1690 msgr2=0x7f0f8c1603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f8bfff700 1 --2- 172.21.15.81:0/3953734535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f8c0d1690 0x7f0f8c1603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f8bfff700 1 -- 172.21.15.81:0/3953734535 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c009570 con 0x7f0f8c0b2ea0 2024-05-08T23:30:41.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f8bfff700 1 --2- 172.21.15.81:0/3953734535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f8c0b2ea0 0x7f0f8c1f2d90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c016000 tx=0x7f0f7c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:41.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f897fa700 1 -- 172.21.15.81:0/3953734535 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c004510 con 0x7f0f8c0b2ea0 2024-05-08T23:30:41.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f9113f700 1 -- 172.21.15.81:0/3953734535 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8c1f3680 con 0x7f0f8c0b2ea0 2024-05-08T23:30:41.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f897fa700 1 -- 172.21.15.81:0/3953734535 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c01e070 con 0x7f0f8c0b2ea0 2024-05-08T23:30:41.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f897fa700 1 -- 172.21.15.81:0/3953734535 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c003900 con 0x7f0f8c0b2ea0 2024-05-08T23:30:41.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f9113f700 1 -- 172.21.15.81:0/3953734535 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c1f3c40 con 0x7f0f8c0b2ea0 2024-05-08T23:30:41.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.817+0000 7f0f9113f700 1 -- 172.21.15.81:0/3953734535 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8c071f80 con 0x7f0f8c0b2ea0 2024-05-08T23:30:41.824 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.821+0000 7f0f897fa700 1 -- 172.21.15.81:0/3953734535 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0f7c003a60 con 0x7f0f8c0b2ea0 2024-05-08T23:30:41.825 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.821+0000 7f0f897fa700 1 --2- 172.21.15.81:0/3953734535 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0f6c066730 0x7f0f6c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.825 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.821+0000 7f0f897fa700 1 -- 172.21.15.81:0/3953734535 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f7c085520 con 0x7f0f8c0b2ea0 2024-05-08T23:30:41.825 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.821+0000 7f0f837fe700 1 --2- 172.21.15.81:0/3953734535 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0f6c066730 0x7f0f6c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.825 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.821+0000 7f0f837fe700 1 --2- 172.21.15.81:0/3953734535 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0f6c066730 0x7f0f6c068bf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c1f32d0 tx=0x7f0f74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:41.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:41.829+0000 7f0f897fa700 1 -- 172.21.15.81:0/3953734535 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0f7c052880 con 0x7f0f8c0b2ea0 2024-05-08T23:30:42.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.037+0000 7f0f9113f700 1 -- 172.21.15.81:0/3953734535 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f8c0684b0 con 0x7f0f8c0b2ea0 2024-05-08T23:30:42.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.037+0000 7f0f897fa700 1 -- 172.21.15.81:0/3953734535 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7f0f7c055ea0 con 0x7f0f8c0b2ea0 2024-05-08T23:30:42.043 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:42.043 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1475726755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1475726755},{"type":"v1","addr":"172.21.15.81:6835","nonce":1475726755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3535820540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3535820540},{"type":"v1","addr":"172.21.15.81:6837","nonce":3535820540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1531019955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1531019955},{"type":"v1","addr":"172.21.15.117:6833","nonce":1531019955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:30:38.665368+0000","modified":"2024-05-08T23:30:41.672722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/491911303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":491911303},{"type":"v1","addr":"172.21.15.117:6835","nonce":491911303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:30:42.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.041+0000 7f0f9113f700 1 -- 172.21.15.81:0/3953734535 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0f6c066730 msgr2=0x7f0f6c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.041+0000 7f0f9113f700 1 --2- 172.21.15.81:0/3953734535 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0f6c066730 0x7f0f6c068bf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c1f32d0 tx=0x7f0f74009040 comp rx=0 tx=0).stop 2024-05-08T23:30:42.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.041+0000 7f0f9113f700 1 -- 172.21.15.81:0/3953734535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f8c0b2ea0 msgr2=0x7f0f8c1f2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.041+0000 7f0f9113f700 1 --2- 172.21.15.81:0/3953734535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f8c0b2ea0 0x7f0f8c1f2d90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c016000 tx=0x7f0f7c004770 comp rx=0 tx=0).stop 2024-05-08T23:30:42.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.041+0000 7f0f9113f700 1 -- 172.21.15.81:0/3953734535 shutdown_connections 2024-05-08T23:30:42.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.041+0000 7f0f9113f700 1 --2- 172.21.15.81:0/3953734535 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0f6c066730 0x7f0f6c068bf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.041+0000 7f0f9113f700 1 --2- 172.21.15.81:0/3953734535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f8c0d1690 0x7f0f8c1603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.041+0000 7f0f9113f700 1 --2- 172.21.15.81:0/3953734535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f8c0d0f20 0x7f0f8c0cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.041+0000 7f0f9113f700 1 --2- 172.21.15.81:0/3953734535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f8c0b2ea0 0x7f0f8c1f2d90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.041+0000 7f0f9113f700 1 -- 172.21.15.81:0/3953734535 >> 172.21.15.81:0/3953734535 conn(0x7f0f8c09c3c0 msgr2=0x7f0f8c06b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:42.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.041+0000 7f0f9113f700 1 -- 172.21.15.81:0/3953734535 shutdown_connections 2024-05-08T23:30:42.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.041+0000 7f0f9113f700 1 -- 172.21.15.81:0/3953734535 wait complete. 2024-05-08T23:30:42.047 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 7 2024-05-08T23:30:42.058 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:30:42.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.161+0000 7fd8e5e49700 1 Processor -- start 2024-05-08T23:30:42.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.161+0000 7fd8e5e49700 1 -- start start 2024-05-08T23:30:42.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.161+0000 7fd8e5e49700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd8e006dcc0 0x7fd8e00b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.161+0000 7fd8e5e49700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd8e00b4530 0x7fd8e00b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.161+0000 7fd8e5e49700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8e00acdb0 0x7fd8e00ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.161+0000 7fd8e5e49700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e00af8b0 con 0x7fd8e006dcc0 2024-05-08T23:30:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.161+0000 7fd8e5e49700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e00b3490 con 0x7fd8e00b4530 2024-05-08T23:30:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.161+0000 7fd8e5e49700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00b3770 con 0x7fd8e00acdb0 2024-05-08T23:30:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.161+0000 7fd8e4e47700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd8e006dcc0 0x7fd8e00b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.161+0000 7fd8e4e47700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd8e006dcc0 0x7fd8e00b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32908/0 (socket says 172.21.15.81:32908) 2024-05-08T23:30:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.161+0000 7fd8e4e47700 1 -- 172.21.15.81:0/3763628910 learned_addr learned my addr 172.21.15.81:0/3763628910 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.161+0000 7fd8dffff700 1 --2- 172.21.15.81:0/3763628910 >> v2:172.21.15.117:3300/0 conn(0x7fd8e00b4530 0x7fd8e00b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.161+0000 7fd8e5648700 1 --2- 172.21.15.81:0/3763628910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8e00acdb0 0x7fd8e00ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.161+0000 7fd8e4e47700 1 -- 172.21.15.81:0/3763628910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8e00acdb0 msgr2=0x7fd8e00ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e4e47700 1 --2- 172.21.15.81:0/3763628910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8e00acdb0 0x7fd8e00ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e4e47700 1 -- 172.21.15.81:0/3763628910 >> v2:172.21.15.117:3300/0 conn(0x7fd8e00b4530 msgr2=0x7fd8e00b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e4e47700 1 --2- 172.21.15.81:0/3763628910 >> v2:172.21.15.117:3300/0 conn(0x7fd8e00b4530 0x7fd8e00b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e4e47700 1 -- 172.21.15.81:0/3763628910 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e00b3a50 con 0x7fd8e006dcc0 2024-05-08T23:30:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e5648700 1 --2- 172.21.15.81:0/3763628910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8e00acdb0 0x7fd8e00ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e4e47700 1 --2- 172.21.15.81:0/3763628910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8e006dcc0 0x7fd8e00b2dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009910 tx=0x7fd8d4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6777c1f330df1b9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8df7fe700 1 -- 172.21.15.81:0/3763628910 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4011830 con 0x7fd8e006dcc0 2024-05-08T23:30:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8df7fe700 1 -- 172.21.15.81:0/3763628910 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d4011990 con 0x7fd8e006dcc0 2024-05-08T23:30:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8df7fe700 1 -- 172.21.15.81:0/3763628910 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4011d40 con 0x7fd8e006dcc0 2024-05-08T23:30:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e5e49700 1 -- 172.21.15.81:0/3763628910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8e006dcc0 msgr2=0x7fd8e00b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e5e49700 1 --2- 172.21.15.81:0/3763628910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8e006dcc0 0x7fd8e00b2dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009910 tx=0x7fd8d4009c20 comp rx=0 tx=0).stop 2024-05-08T23:30:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e5e49700 1 -- 172.21.15.81:0/3763628910 shutdown_connections 2024-05-08T23:30:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e5e49700 1 --2- 172.21.15.81:0/3763628910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8e00acdb0 0x7fd8e00ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e5e49700 1 --2- 172.21.15.81:0/3763628910 >> v2:172.21.15.117:3300/0 conn(0x7fd8e00b4530 0x7fd8e00b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e5e49700 1 --2- 172.21.15.81:0/3763628910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8e006dcc0 0x7fd8e00b2dd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e5e49700 1 -- 172.21.15.81:0/3763628910 >> 172.21.15.81:0/3763628910 conn(0x7fd8e006b470 msgr2=0x7fd8e006b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e5e49700 1 -- 172.21.15.81:0/3763628910 shutdown_connections 2024-05-08T23:30:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e5e49700 1 -- 172.21.15.81:0/3763628910 wait complete. 2024-05-08T23:30:42.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e5e49700 1 Processor -- start 2024-05-08T23:30:42.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.165+0000 7fd8e5e49700 1 -- start start 2024-05-08T23:30:42.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8e5e49700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8e006dcc0 0x7fd8e0159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8e5e49700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8e00acdb0 0x7fd8e0141800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8e5e49700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8e00b4530 0x7fd8e015e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8e5e49700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e0141d40 con 0x7fd8e00acdb0 2024-05-08T23:30:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8e5e49700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e0152f00 con 0x7fd8e00b4530 2024-05-08T23:30:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8e5e49700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e0157da0 con 0x7fd8e006dcc0 2024-05-08T23:30:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8dffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8e00acdb0 0x7fd8e0141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8dffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8e00acdb0 0x7fd8e0141800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32920/0 (socket says 172.21.15.81:32920) 2024-05-08T23:30:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8dffff700 1 -- 172.21.15.81:0/253938097 learned_addr learned my addr 172.21.15.81:0/253938097 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8e5648700 1 --2- 172.21.15.81:0/253938097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8e00b4530 0x7fd8e015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8e4e47700 1 --2- 172.21.15.81:0/253938097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8e006dcc0 0x7fd8e0159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8dffff700 1 -- 172.21.15.81:0/253938097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8e006dcc0 msgr2=0x7fd8e0159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8dffff700 1 --2- 172.21.15.81:0/253938097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8e006dcc0 0x7fd8e0159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8dffff700 1 -- 172.21.15.81:0/253938097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8e00b4530 msgr2=0x7fd8e015e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8dffff700 1 --2- 172.21.15.81:0/253938097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8e00b4530 0x7fd8e015e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8dffff700 1 -- 172.21.15.81:0/253938097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d4009570 con 0x7fd8e00acdb0 2024-05-08T23:30:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8e5648700 1 --2- 172.21.15.81:0/253938097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8e00b4530 0x7fd8e015e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8dffff700 1 --2- 172.21.15.81:0/253938097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8e00acdb0 0x7fd8e0141800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000ea30 tx=0x7fd8d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:42.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8dd7fa700 1 -- 172.21.15.81:0/253938097 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0009720 con 0x7fd8e00acdb0 2024-05-08T23:30:42.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8dd7fa700 1 -- 172.21.15.81:0/253938097 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d000c910 con 0x7fd8e00acdb0 2024-05-08T23:30:42.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8e5e49700 1 -- 172.21.15.81:0/253938097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e0157ff0 con 0x7fd8e00acdb0 2024-05-08T23:30:42.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8dd7fa700 1 -- 172.21.15.81:0/253938097 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d000cc80 con 0x7fd8e00acdb0 2024-05-08T23:30:42.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8e5e49700 1 -- 172.21.15.81:0/253938097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e01585b0 con 0x7fd8e00acdb0 2024-05-08T23:30:42.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.169+0000 7fd8e5e49700 1 -- 172.21.15.81:0/253938097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e00684b0 con 0x7fd8e00acdb0 2024-05-08T23:30:42.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.173+0000 7fd8dd7fa700 1 -- 172.21.15.81:0/253938097 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd8d0013470 con 0x7fd8e00acdb0 2024-05-08T23:30:42.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.173+0000 7fd8dd7fa700 1 --2- 172.21.15.81:0/253938097 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd8cc066730 0x7fd8cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.173+0000 7fd8e4e47700 1 --2- 172.21.15.81:0/253938097 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd8cc066730 0x7fd8cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.173+0000 7fd8dd7fa700 1 -- 172.21.15.81:0/253938097 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8d0084a70 con 0x7fd8e00acdb0 2024-05-08T23:30:42.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.173+0000 7fd8e4e47700 1 --2- 172.21.15.81:0/253938097 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd8cc066730 0x7fd8cc068bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4014040 tx=0x7fd8d4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:42.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.181+0000 7fd8dd7fa700 1 -- 172.21.15.81:0/253938097 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd8d0051dc0 con 0x7fd8e00acdb0 2024-05-08T23:30:42.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.353+0000 7fd8e5e49700 1 -- 172.21.15.81:0/253938097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8e006c3a0 con 0x7fd8e00acdb0 2024-05-08T23:30:42.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.357+0000 7fd8dd7fa700 1 -- 172.21.15.81:0/253938097 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fd8d0017090 con 0x7fd8e00acdb0 2024-05-08T23:30:42.360 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:42.360 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:30:42.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.357+0000 7fd8e5e49700 1 -- 172.21.15.81:0/253938097 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd8cc066730 msgr2=0x7fd8cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.357+0000 7fd8e5e49700 1 --2- 172.21.15.81:0/253938097 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd8cc066730 0x7fd8cc068bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4014040 tx=0x7fd8d4008030 comp rx=0 tx=0).stop 2024-05-08T23:30:42.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.357+0000 7fd8e5e49700 1 -- 172.21.15.81:0/253938097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8e00acdb0 msgr2=0x7fd8e0141800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.357+0000 7fd8e5e49700 1 --2- 172.21.15.81:0/253938097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8e00acdb0 0x7fd8e0141800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000ea30 tx=0x7fd8d000edf0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.357+0000 7fd8e5e49700 1 -- 172.21.15.81:0/253938097 shutdown_connections 2024-05-08T23:30:42.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.357+0000 7fd8e5e49700 1 --2- 172.21.15.81:0/253938097 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd8cc066730 0x7fd8cc068bf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.357+0000 7fd8e5e49700 1 --2- 172.21.15.81:0/253938097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8e00b4530 0x7fd8e015e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.357+0000 7fd8e5e49700 1 --2- 172.21.15.81:0/253938097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8e00acdb0 0x7fd8e0141800 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.357+0000 7fd8e5e49700 1 --2- 172.21.15.81:0/253938097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8e006dcc0 0x7fd8e0159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.357+0000 7fd8e5e49700 1 -- 172.21.15.81:0/253938097 >> 172.21.15.81:0/253938097 conn(0x7fd8e006b470 msgr2=0x7fd8e00ae6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:42.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.357+0000 7fd8e5e49700 1 -- 172.21.15.81:0/253938097 shutdown_connections 2024-05-08T23:30:42.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.357+0000 7fd8e5e49700 1 -- 172.21.15.81:0/253938097 wait complete. 2024-05-08T23:30:42.376 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T23:30:42.376 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:30:42.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.481+0000 7f3db2a97700 1 Processor -- start 2024-05-08T23:30:42.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.481+0000 7f3db2a97700 1 -- start start 2024-05-08T23:30:42.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2a97700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dac15afd0 0x7f3dac0cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2a97700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3dac15f1b0 0x7f3dac06fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2a97700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dac0cf300 con 0x7f3dac15f6b0 2024-05-08T23:30:42.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2a97700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dac15c3c0 con 0x7f3dac15f1b0 2024-05-08T23:30:42.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2a97700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac0ced80 con 0x7f3dac15afd0 2024-05-08T23:30:42.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2296700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3dac15f1b0 0x7f3dac06fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db1294700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f3dac15f6b0 0x7f3dac15eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54998/0 (socket says 172.21.15.81:54998) 2024-05-08T23:30:42.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db1294700 1 -- 172.21.15.81:0/2766314632 learned_addr learned my addr 172.21.15.81:0/2766314632 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:42.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db1a95700 1 --2- 172.21.15.81:0/2766314632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dac15afd0 0x7f3dac0cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2296700 1 -- 172.21.15.81:0/2766314632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dac15afd0 msgr2=0x7f3dac0cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:42.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2296700 1 --2- 172.21.15.81:0/2766314632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dac15afd0 0x7f3dac0cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2296700 1 -- 172.21.15.81:0/2766314632 >> v1:172.21.15.81:6789/0 conn(0x7f3dac15f6b0 legacy=0x7f3dac15eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2296700 1 -- 172.21.15.81:0/2766314632 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac09a620 con 0x7f3dac15f1b0 2024-05-08T23:30:42.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db1a95700 1 --2- 172.21.15.81:0/2766314632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dac15afd0 0x7f3dac0cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:42.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2296700 1 --2- 172.21.15.81:0/2766314632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dac15f1b0 0x7f3dac06fdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009d90 tx=0x7f3da400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d541074feebc70 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:42.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db0a93700 1 -- 172.21.15.81:0/2766314632 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da400e070 con 0x7f3dac15f1b0 2024-05-08T23:30:42.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db0a93700 1 -- 172.21.15.81:0/2766314632 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da400bbd0 con 0x7f3dac15f1b0 2024-05-08T23:30:42.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db0a93700 1 -- 172.21.15.81:0/2766314632 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4012430 con 0x7f3dac15f1b0 2024-05-08T23:30:42.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2a97700 1 -- 172.21.15.81:0/2766314632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dac15f1b0 msgr2=0x7f3dac06fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2a97700 1 --2- 172.21.15.81:0/2766314632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dac15f1b0 0x7f3dac06fdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009d90 tx=0x7f3da400b330 comp rx=0 tx=0).stop 2024-05-08T23:30:42.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2a97700 1 -- 172.21.15.81:0/2766314632 shutdown_connections 2024-05-08T23:30:42.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2a97700 1 --2- 172.21.15.81:0/2766314632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dac15f1b0 0x7f3dac06fdf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2a97700 1 --2- 172.21.15.81:0/2766314632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dac15afd0 0x7f3dac0cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2a97700 1 -- 172.21.15.81:0/2766314632 >> 172.21.15.81:0/2766314632 conn(0x7f3dac0d0e40 msgr2=0x7f3dac0d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:42.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2a97700 1 -- 172.21.15.81:0/2766314632 shutdown_connections 2024-05-08T23:30:42.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.485+0000 7f3db2a97700 1 -- 172.21.15.81:0/2766314632 wait complete. 2024-05-08T23:30:42.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db2a97700 1 Processor -- start 2024-05-08T23:30:42.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db2a97700 1 -- start start 2024-05-08T23:30:42.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db2a97700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dac15afd0 0x7f3dac0aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db2a97700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dac15f1b0 0x7f3dac14bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db2a97700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dac15f6b0 0x7f3dac1f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db2a97700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac14c000 con 0x7f3dac15afd0 2024-05-08T23:30:42.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db2a97700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac14b3d0 con 0x7f3dac15f1b0 2024-05-08T23:30:42.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db2a97700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac0b0cc0 con 0x7f3dac15f6b0 2024-05-08T23:30:42.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db1a95700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dac15afd0 0x7f3dac0aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db1a95700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dac15afd0 0x7f3dac0aea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32926/0 (socket says 172.21.15.81:32926) 2024-05-08T23:30:42.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db1a95700 1 -- 172.21.15.81:0/1536545219 learned_addr learned my addr 172.21.15.81:0/1536545219 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:42.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db1294700 1 --2- 172.21.15.81:0/1536545219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dac15f1b0 0x7f3dac14bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db2296700 1 --2- 172.21.15.81:0/1536545219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dac15f6b0 0x7f3dac1f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db1a95700 1 -- 172.21.15.81:0/1536545219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dac15f6b0 msgr2=0x7f3dac1f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db1a95700 1 --2- 172.21.15.81:0/1536545219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dac15f6b0 0x7f3dac1f2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db1a95700 1 -- 172.21.15.81:0/1536545219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dac15f1b0 msgr2=0x7f3dac14bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db1a95700 1 --2- 172.21.15.81:0/1536545219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dac15f1b0 0x7f3dac14bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db1a95700 1 -- 172.21.15.81:0/1536545219 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da4009a70 con 0x7f3dac15afd0 2024-05-08T23:30:42.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.489+0000 7f3db1a95700 1 --2- 172.21.15.81:0/1536545219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dac15afd0 0x7f3dac0aea20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3da800e8b0 tx=0x7f3da800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:42.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.493+0000 7f3da27fc700 1 -- 172.21.15.81:0/1536545219 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8013610 con 0x7f3dac15afd0 2024-05-08T23:30:42.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.493+0000 7f3da27fc700 1 -- 172.21.15.81:0/1536545219 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da8013770 con 0x7f3dac15afd0 2024-05-08T23:30:42.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.493+0000 7f3db2a97700 1 -- 172.21.15.81:0/1536545219 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dac0af120 con 0x7f3dac15afd0 2024-05-08T23:30:42.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.493+0000 7f3da27fc700 1 -- 172.21.15.81:0/1536545219 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8013b00 con 0x7f3dac15afd0 2024-05-08T23:30:42.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.493+0000 7f3db2a97700 1 -- 172.21.15.81:0/1536545219 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dac0af6e0 con 0x7f3dac15afd0 2024-05-08T23:30:42.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.493+0000 7f3db2a97700 1 -- 172.21.15.81:0/1536545219 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dac0672a0 con 0x7f3dac15afd0 2024-05-08T23:30:42.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.493+0000 7f3da27fc700 1 -- 172.21.15.81:0/1536545219 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3da8013c60 con 0x7f3dac15afd0 2024-05-08T23:30:42.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.493+0000 7f3da27fc700 1 --2- 172.21.15.81:0/1536545219 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3d90066730 0x7f3d90068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.493+0000 7f3da27fc700 1 -- 172.21.15.81:0/1536545219 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3da8083c80 con 0x7f3dac15afd0 2024-05-08T23:30:42.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.493+0000 7f3db1294700 1 --2- 172.21.15.81:0/1536545219 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3d90066730 0x7f3d90068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.497+0000 7f3db1294700 1 --2- 172.21.15.81:0/1536545219 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3d90066730 0x7f3d90068bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3dac06ec20 tx=0x7f3d9c007f30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:42.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.505+0000 7f3da27fc700 1 -- 172.21.15.81:0/1536545219 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3da8050ff0 con 0x7f3dac15afd0 2024-05-08T23:30:42.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.673+0000 7f3db2a97700 1 -- 172.21.15.81:0/1536545219 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3dac0cea10 con 0x7f3d90066730 2024-05-08T23:30:42.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.681+0000 7f3da27fc700 1 -- 172.21.15.81:0/1536545219 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264502 (secure 0 0 0) 0x7f3d9c004b90 con 0x7f3d90066730 2024-05-08T23:30:42.691 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:42.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.689+0000 7f3db2a97700 1 -- 172.21.15.81:0/1536545219 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3d90066730 msgr2=0x7f3d90068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.689+0000 7f3db2a97700 1 --2- 172.21.15.81:0/1536545219 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3d90066730 0x7f3d90068bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3dac06ec20 tx=0x7f3d9c007f30 comp rx=0 tx=0).stop 2024-05-08T23:30:42.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.689+0000 7f3db2a97700 1 -- 172.21.15.81:0/1536545219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dac15afd0 msgr2=0x7f3dac0aea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.689+0000 7f3db2a97700 1 --2- 172.21.15.81:0/1536545219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dac15afd0 0x7f3dac0aea20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3da800e8b0 tx=0x7f3da800ebc0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.689+0000 7f3db2a97700 1 -- 172.21.15.81:0/1536545219 shutdown_connections 2024-05-08T23:30:42.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.689+0000 7f3db2a97700 1 --2- 172.21.15.81:0/1536545219 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3d90066730 0x7f3d90068bf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.689+0000 7f3db2a97700 1 --2- 172.21.15.81:0/1536545219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dac15f6b0 0x7f3dac1f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.689+0000 7f3db2a97700 1 --2- 172.21.15.81:0/1536545219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dac15f1b0 0x7f3dac14bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.689+0000 7f3db2a97700 1 --2- 172.21.15.81:0/1536545219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dac15afd0 0x7f3dac0aea20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.689+0000 7f3db2a97700 1 -- 172.21.15.81:0/1536545219 >> 172.21.15.81:0/1536545219 conn(0x7f3dac0d0e40 msgr2=0x7f3dac0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:42.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.689+0000 7f3db2a97700 1 -- 172.21.15.81:0/1536545219 shutdown_connections 2024-05-08T23:30:42.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.689+0000 7f3db2a97700 1 -- 172.21.15.81:0/1536545219 wait complete. 2024-05-08T23:30:42.694 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:30:42.705 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-08T23:30:41.325117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219028,"kb_used_data":2852,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755692,"statfs":{"total":773094113280,"available":772869828608,"internally_reserved":0,"allocated":2920448,"data_stored":1748515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.044014"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711846+0000","last_change":"2024-05-08T23:30:38.707899+0000","last_active":"2024-05-08T23:30:38.711846+0000","last_peered":"2024-05-08T23:30:38.711846+0000","last_clean":"2024-05-08T23:30:38.711846+0000","last_became_active":"2024-05-08T23:30:38.706532+0000","last_became_peered":"2024-05-08T23:30:38.706532+0000","last_unstale":"2024-05-08T23:30:38.711846+0000","last_undegraded":"2024-05-08T23:30:38.711846+0000","last_fullsized":"2024-05-08T23:30:38.711846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:21.109032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.689635+0000","last_change":"2024-05-08T23:30:38.684226+0000","last_active":"2024-05-08T23:30:38.689635+0000","last_peered":"2024-05-08T23:30:38.689635+0000","last_clean":"2024-05-08T23:30:38.689635+0000","last_became_active":"2024-05-08T23:30:38.683958+0000","last_became_peered":"2024-05-08T23:30:38.683958+0000","last_unstale":"2024-05-08T23:30:38.689635+0000","last_undegraded":"2024-05-08T23:30:38.689635+0000","last_fullsized":"2024-05-08T23:30:38.689635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:07.222797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.688209+0000","last_change":"2024-05-08T23:30:38.686289+0000","last_active":"2024-05-08T23:30:38.688209+0000","last_peered":"2024-05-08T23:30:38.688209+0000","last_clean":"2024-05-08T23:30:38.688209+0000","last_became_active":"2024-05-08T23:30:38.685661+0000","last_became_peered":"2024-05-08T23:30:38.685661+0000","last_unstale":"2024-05-08T23:30:38.688209+0000","last_undegraded":"2024-05-08T23:30:38.688209+0000","last_fullsized":"2024-05-08T23:30:38.688209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:34.727934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243711+0000","last_change":"2024-05-08T23:30:38.694133+0000","last_active":"2024-05-08T23:30:40.243711+0000","last_peered":"2024-05-08T23:30:40.243711+0000","last_clean":"2024-05-08T23:30:40.243711+0000","last_became_active":"2024-05-08T23:30:38.693846+0000","last_became_peered":"2024-05-08T23:30:38.693846+0000","last_unstale":"2024-05-08T23:30:40.243711+0000","last_undegraded":"2024-05-08T23:30:40.243711+0000","last_fullsized":"2024-05-08T23:30:40.243711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:44.481684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711895+0000","last_change":"2024-05-08T23:30:38.705630+0000","last_active":"2024-05-08T23:30:38.711895+0000","last_peered":"2024-05-08T23:30:38.711895+0000","last_clean":"2024-05-08T23:30:38.711895+0000","last_became_active":"2024-05-08T23:30:38.705310+0000","last_became_peered":"2024-05-08T23:30:38.705310+0000","last_unstale":"2024-05-08T23:30:38.711895+0000","last_undegraded":"2024-05-08T23:30:38.711895+0000","last_fullsized":"2024-05-08T23:30:38.711895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:41.949561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243496+0000","last_change":"2024-05-08T23:30:38.691423+0000","last_active":"2024-05-08T23:30:40.243496+0000","last_peered":"2024-05-08T23:30:40.243496+0000","last_clean":"2024-05-08T23:30:40.243496+0000","last_became_active":"2024-05-08T23:30:38.690973+0000","last_became_peered":"2024-05-08T23:30:38.690973+0000","last_unstale":"2024-05-08T23:30:40.243496+0000","last_undegraded":"2024-05-08T23:30:40.243496+0000","last_fullsized":"2024-05-08T23:30:40.243496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:42.503600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711302+0000","last_change":"2024-05-08T23:30:38.702702+0000","last_active":"2024-05-08T23:30:38.711302+0000","last_peered":"2024-05-08T23:30:38.711302+0000","last_clean":"2024-05-08T23:30:38.711302+0000","last_became_active":"2024-05-08T23:30:38.702341+0000","last_became_peered":"2024-05-08T23:30:38.702341+0000","last_unstale":"2024-05-08T23:30:38.711302+0000","last_undegraded":"2024-05-08T23:30:38.711302+0000","last_fullsized":"2024-05-08T23:30:38.711302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:05.546774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675055+0000","last_change":"2024-05-08T23:30:38.692708+0000","last_active":"2024-05-08T23:30:39.675055+0000","last_peered":"2024-05-08T23:30:39.675055+0000","last_clean":"2024-05-08T23:30:39.675055+0000","last_became_active":"2024-05-08T23:30:38.692424+0000","last_became_peered":"2024-05-08T23:30:38.692424+0000","last_unstale":"2024-05-08T23:30:39.675055+0000","last_undegraded":"2024-05-08T23:30:39.675055+0000","last_fullsized":"2024-05-08T23:30:39.675055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:58.747769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711863+0000","last_change":"2024-05-08T23:30:38.708610+0000","last_active":"2024-05-08T23:30:38.711863+0000","last_peered":"2024-05-08T23:30:38.711863+0000","last_clean":"2024-05-08T23:30:38.711863+0000","last_became_active":"2024-05-08T23:30:38.708389+0000","last_became_peered":"2024-05-08T23:30:38.708389+0000","last_unstale":"2024-05-08T23:30:38.711863+0000","last_undegraded":"2024-05-08T23:30:38.711863+0000","last_fullsized":"2024-05-08T23:30:38.711863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:41.257114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.693809+0000","last_change":"2024-05-08T23:30:38.690662+0000","last_active":"2024-05-08T23:30:38.693809+0000","last_peered":"2024-05-08T23:30:38.693809+0000","last_clean":"2024-05-08T23:30:38.693809+0000","last_became_active":"2024-05-08T23:30:38.690378+0000","last_became_peered":"2024-05-08T23:30:38.690378+0000","last_unstale":"2024-05-08T23:30:38.693809+0000","last_undegraded":"2024-05-08T23:30:38.693809+0000","last_fullsized":"2024-05-08T23:30:38.693809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:33.598895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.694520+0000","last_change":"2024-05-08T23:30:38.692171+0000","last_active":"2024-05-08T23:30:38.694520+0000","last_peered":"2024-05-08T23:30:38.694520+0000","last_clean":"2024-05-08T23:30:38.694520+0000","last_became_active":"2024-05-08T23:30:38.691611+0000","last_became_peered":"2024-05-08T23:30:38.691611+0000","last_unstale":"2024-05-08T23:30:38.694520+0000","last_undegraded":"2024-05-08T23:30:38.694520+0000","last_fullsized":"2024-05-08T23:30:38.694520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959822+0000","last_change":"2024-05-08T23:30:38.694336+0000","last_active":"2024-05-08T23:30:39.959822+0000","last_peered":"2024-05-08T23:30:39.959822+0000","last_clean":"2024-05-08T23:30:39.959822+0000","last_became_active":"2024-05-08T23:30:38.694038+0000","last_became_peered":"2024-05-08T23:30:38.694038+0000","last_unstale":"2024-05-08T23:30:39.959822+0000","last_undegraded":"2024-05-08T23:30:39.959822+0000","last_fullsized":"2024-05-08T23:30:39.959822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:07.443142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676123+0000","last_change":"2024-05-08T23:30:38.691325+0000","last_active":"2024-05-08T23:30:39.676123+0000","last_peered":"2024-05-08T23:30:39.676123+0000","last_clean":"2024-05-08T23:30:39.676123+0000","last_became_active":"2024-05-08T23:30:38.691074+0000","last_became_peered":"2024-05-08T23:30:38.691074+0000","last_unstale":"2024-05-08T23:30:39.676123+0000","last_undegraded":"2024-05-08T23:30:39.676123+0000","last_fullsized":"2024-05-08T23:30:39.676123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:37.994049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.681493+0000","last_change":"2024-05-08T23:30:38.679460+0000","last_active":"2024-05-08T23:30:38.681493+0000","last_peered":"2024-05-08T23:30:38.681493+0000","last_clean":"2024-05-08T23:30:38.681493+0000","last_became_active":"2024-05-08T23:30:38.678871+0000","last_became_peered":"2024-05-08T23:30:38.678871+0000","last_unstale":"2024-05-08T23:30:38.681493+0000","last_undegraded":"2024-05-08T23:30:38.681493+0000","last_fullsized":"2024-05-08T23:30:38.681493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:37.465394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.694972+0000","last_change":"2024-05-08T23:30:38.692896+0000","last_active":"2024-05-08T23:30:38.694972+0000","last_peered":"2024-05-08T23:30:38.694972+0000","last_clean":"2024-05-08T23:30:38.694972+0000","last_became_active":"2024-05-08T23:30:38.692347+0000","last_became_peered":"2024-05-08T23:30:38.692347+0000","last_unstale":"2024-05-08T23:30:38.694972+0000","last_undegraded":"2024-05-08T23:30:38.694972+0000","last_fullsized":"2024-05-08T23:30:38.694972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:01.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243296+0000","last_change":"2024-05-08T23:30:38.689233+0000","last_active":"2024-05-08T23:30:40.243296+0000","last_peered":"2024-05-08T23:30:40.243296+0000","last_clean":"2024-05-08T23:30:40.243296+0000","last_became_active":"2024-05-08T23:30:38.688984+0000","last_became_peered":"2024-05-08T23:30:38.688984+0000","last_unstale":"2024-05-08T23:30:40.243296+0000","last_undegraded":"2024-05-08T23:30:40.243296+0000","last_fullsized":"2024-05-08T23:30:40.243296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:07.682170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244541+0000","last_change":"2024-05-08T23:30:38.706139+0000","last_active":"2024-05-08T23:30:40.244541+0000","last_peered":"2024-05-08T23:30:40.244541+0000","last_clean":"2024-05-08T23:30:40.244541+0000","last_became_active":"2024-05-08T23:30:38.705878+0000","last_became_peered":"2024-05-08T23:30:38.705878+0000","last_unstale":"2024-05-08T23:30:40.244541+0000","last_undegraded":"2024-05-08T23:30:40.244541+0000","last_fullsized":"2024-05-08T23:30:40.244541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:39.788098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.693318+0000","last_change":"2024-05-08T23:30:38.685704+0000","last_active":"2024-05-08T23:30:38.693318+0000","last_peered":"2024-05-08T23:30:38.693318+0000","last_clean":"2024-05-08T23:30:38.693318+0000","last_became_active":"2024-05-08T23:30:38.685435+0000","last_became_peered":"2024-05-08T23:30:38.685435+0000","last_unstale":"2024-05-08T23:30:38.693318+0000","last_undegraded":"2024-05-08T23:30:38.693318+0000","last_fullsized":"2024-05-08T23:30:38.693318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:07.443142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.712025+0000","last_change":"2024-05-08T23:30:38.710968+0000","last_active":"2024-05-08T23:30:38.712025+0000","last_peered":"2024-05-08T23:30:38.712025+0000","last_clean":"2024-05-08T23:30:38.712025+0000","last_became_active":"2024-05-08T23:30:38.710706+0000","last_became_peered":"2024-05-08T23:30:38.710706+0000","last_unstale":"2024-05-08T23:30:38.712025+0000","last_undegraded":"2024-05-08T23:30:38.712025+0000","last_fullsized":"2024-05-08T23:30:38.712025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:32.727647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.705107+0000","last_change":"2024-05-08T23:30:38.701983+0000","last_active":"2024-05-08T23:30:38.705107+0000","last_peered":"2024-05-08T23:30:38.705107+0000","last_clean":"2024-05-08T23:30:38.705107+0000","last_became_active":"2024-05-08T23:30:38.701386+0000","last_became_peered":"2024-05-08T23:30:38.701386+0000","last_unstale":"2024-05-08T23:30:38.705107+0000","last_undegraded":"2024-05-08T23:30:38.705107+0000","last_fullsized":"2024-05-08T23:30:38.705107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:12.789913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.707115+0000","last_change":"2024-05-08T23:30:38.705814+0000","last_active":"2024-05-08T23:30:38.707115+0000","last_peered":"2024-05-08T23:30:38.707115+0000","last_clean":"2024-05-08T23:30:38.707115+0000","last_became_active":"2024-05-08T23:30:38.705393+0000","last_became_peered":"2024-05-08T23:30:38.705393+0000","last_unstale":"2024-05-08T23:30:38.707115+0000","last_undegraded":"2024-05-08T23:30:38.707115+0000","last_fullsized":"2024-05-08T23:30:38.707115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:35.129372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243643+0000","last_change":"2024-05-08T23:30:38.703498+0000","last_active":"2024-05-08T23:30:40.243643+0000","last_peered":"2024-05-08T23:30:40.243643+0000","last_clean":"2024-05-08T23:30:40.243643+0000","last_became_active":"2024-05-08T23:30:38.703206+0000","last_became_peered":"2024-05-08T23:30:38.703206+0000","last_unstale":"2024-05-08T23:30:40.243643+0000","last_undegraded":"2024-05-08T23:30:40.243643+0000","last_fullsized":"2024-05-08T23:30:40.243643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:58.747769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675186+0000","last_change":"2024-05-08T23:30:38.697689+0000","last_active":"2024-05-08T23:30:39.675186+0000","last_peered":"2024-05-08T23:30:39.675186+0000","last_clean":"2024-05-08T23:30:39.675186+0000","last_became_active":"2024-05-08T23:30:38.697373+0000","last_became_peered":"2024-05-08T23:30:38.697373+0000","last_unstale":"2024-05-08T23:30:39.675186+0000","last_undegraded":"2024-05-08T23:30:39.675186+0000","last_fullsized":"2024-05-08T23:30:39.675186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:42.503600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243343+0000","last_change":"2024-05-08T23:30:38.693170+0000","last_active":"2024-05-08T23:30:40.243343+0000","last_peered":"2024-05-08T23:30:40.243343+0000","last_clean":"2024-05-08T23:30:40.243343+0000","last_became_active":"2024-05-08T23:30:38.692809+0000","last_became_peered":"2024-05-08T23:30:38.692809+0000","last_unstale":"2024-05-08T23:30:40.243343+0000","last_undegraded":"2024-05-08T23:30:40.243343+0000","last_fullsized":"2024-05-08T23:30:40.243343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:51.717280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675706+0000","last_change":"2024-05-08T23:30:38.696080+0000","last_active":"2024-05-08T23:30:39.675706+0000","last_peered":"2024-05-08T23:30:39.675706+0000","last_clean":"2024-05-08T23:30:39.675706+0000","last_became_active":"2024-05-08T23:30:38.695821+0000","last_became_peered":"2024-05-08T23:30:38.695821+0000","last_unstale":"2024-05-08T23:30:39.675706+0000","last_undegraded":"2024-05-08T23:30:39.675706+0000","last_fullsized":"2024-05-08T23:30:39.675706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:33.598895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.710727+0000","last_change":"2024-05-08T23:30:38.704917+0000","last_active":"2024-05-08T23:30:38.710727+0000","last_peered":"2024-05-08T23:30:38.710727+0000","last_clean":"2024-05-08T23:30:38.710727+0000","last_became_active":"2024-05-08T23:30:38.704663+0000","last_became_peered":"2024-05-08T23:30:38.704663+0000","last_unstale":"2024-05-08T23:30:38.710727+0000","last_undegraded":"2024-05-08T23:30:38.710727+0000","last_fullsized":"2024-05-08T23:30:38.710727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:41.133153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.695486+0000","last_change":"2024-05-08T23:30:38.693867+0000","last_active":"2024-05-08T23:30:38.695486+0000","last_peered":"2024-05-08T23:30:38.695486+0000","last_clean":"2024-05-08T23:30:38.695486+0000","last_became_active":"2024-05-08T23:30:38.693637+0000","last_became_peered":"2024-05-08T23:30:38.693637+0000","last_unstale":"2024-05-08T23:30:38.695486+0000","last_undegraded":"2024-05-08T23:30:38.695486+0000","last_fullsized":"2024-05-08T23:30:38.695486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:33.598895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676136+0000","last_change":"2024-05-08T23:30:38.697753+0000","last_active":"2024-05-08T23:30:39.676136+0000","last_peered":"2024-05-08T23:30:39.676136+0000","last_clean":"2024-05-08T23:30:39.676136+0000","last_became_active":"2024-05-08T23:30:38.697337+0000","last_became_peered":"2024-05-08T23:30:38.697337+0000","last_unstale":"2024-05-08T23:30:39.676136+0000","last_undegraded":"2024-05-08T23:30:39.676136+0000","last_fullsized":"2024-05-08T23:30:39.676136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.690367+0000","last_change":"2024-05-08T23:30:38.688908+0000","last_active":"2024-05-08T23:30:38.690367+0000","last_peered":"2024-05-08T23:30:38.690367+0000","last_clean":"2024-05-08T23:30:38.690367+0000","last_became_active":"2024-05-08T23:30:38.688636+0000","last_became_peered":"2024-05-08T23:30:38.688636+0000","last_unstale":"2024-05-08T23:30:38.690367+0000","last_undegraded":"2024-05-08T23:30:38.690367+0000","last_fullsized":"2024-05-08T23:30:38.690367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:41.059680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243784+0000","last_change":"2024-05-08T23:30:38.688336+0000","last_active":"2024-05-08T23:30:40.243784+0000","last_peered":"2024-05-08T23:30:40.243784+0000","last_clean":"2024-05-08T23:30:40.243784+0000","last_became_active":"2024-05-08T23:30:38.688041+0000","last_became_peered":"2024-05-08T23:30:38.688041+0000","last_unstale":"2024-05-08T23:30:40.243784+0000","last_undegraded":"2024-05-08T23:30:40.243784+0000","last_fullsized":"2024-05-08T23:30:40.243784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:17.135855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.694658+0000","last_change":"2024-05-08T23:30:38.692840+0000","last_active":"2024-05-08T23:30:38.694658+0000","last_peered":"2024-05-08T23:30:38.694658+0000","last_clean":"2024-05-08T23:30:38.694658+0000","last_became_active":"2024-05-08T23:30:38.690283+0000","last_became_peered":"2024-05-08T23:30:38.690283+0000","last_unstale":"2024-05-08T23:30:38.694658+0000","last_undegraded":"2024-05-08T23:30:38.694658+0000","last_fullsized":"2024-05-08T23:30:38.694658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:11.511515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.695415+0000","last_change":"2024-05-08T23:30:38.693989+0000","last_active":"2024-05-08T23:30:38.695415+0000","last_peered":"2024-05-08T23:30:38.695415+0000","last_clean":"2024-05-08T23:30:38.695415+0000","last_became_active":"2024-05-08T23:30:38.693491+0000","last_became_peered":"2024-05-08T23:30:38.693491+0000","last_unstale":"2024-05-08T23:30:38.695415+0000","last_undegraded":"2024-05-08T23:30:38.695415+0000","last_fullsized":"2024-05-08T23:30:38.695415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:07.222797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959371+0000","last_change":"2024-05-08T23:30:38.681154+0000","last_active":"2024-05-08T23:30:39.959371+0000","last_peered":"2024-05-08T23:30:39.959371+0000","last_clean":"2024-05-08T23:30:39.959371+0000","last_became_active":"2024-05-08T23:30:38.680826+0000","last_became_peered":"2024-05-08T23:30:38.680826+0000","last_unstale":"2024-05-08T23:30:39.959371+0000","last_undegraded":"2024-05-08T23:30:39.959371+0000","last_fullsized":"2024-05-08T23:30:39.959371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:11.511515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675917+0000","last_change":"2024-05-08T23:30:38.317473+0000","last_active":"2024-05-08T23:30:39.675917+0000","last_peered":"2024-05-08T23:30:39.675917+0000","last_clean":"2024-05-08T23:30:39.675917+0000","last_became_active":"2024-05-08T23:30:38.317235+0000","last_became_peered":"2024-05-08T23:30:38.317235+0000","last_unstale":"2024-05-08T23:30:39.675917+0000","last_undegraded":"2024-05-08T23:30:39.675917+0000","last_fullsized":"2024-05-08T23:30:39.675917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:31.924157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.687211+0000","last_change":"2024-05-08T23:30:38.680780+0000","last_active":"2024-05-08T23:30:38.687211+0000","last_peered":"2024-05-08T23:30:38.687211+0000","last_clean":"2024-05-08T23:30:38.687211+0000","last_became_active":"2024-05-08T23:30:38.680508+0000","last_became_peered":"2024-05-08T23:30:38.680508+0000","last_unstale":"2024-05-08T23:30:38.687211+0000","last_undegraded":"2024-05-08T23:30:38.687211+0000","last_fullsized":"2024-05-08T23:30:38.687211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:24.058714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.699238+0000","last_change":"2024-05-08T23:30:38.309456+0000","last_active":"2024-05-08T23:30:38.699238+0000","last_peered":"2024-05-08T23:30:38.699238+0000","last_clean":"2024-05-08T23:30:38.699238+0000","last_became_active":"2024-05-08T23:30:38.308975+0000","last_became_peered":"2024-05-08T23:30:38.308975+0000","last_unstale":"2024-05-08T23:30:38.699238+0000","last_undegraded":"2024-05-08T23:30:38.699238+0000","last_fullsized":"2024-05-08T23:30:38.699238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:00.309450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675599+0000","last_change":"2024-05-08T23:30:38.317811+0000","last_active":"2024-05-08T23:30:39.675599+0000","last_peered":"2024-05-08T23:30:39.675599+0000","last_clean":"2024-05-08T23:30:39.675599+0000","last_became_active":"2024-05-08T23:30:38.317505+0000","last_became_peered":"2024-05-08T23:30:38.317505+0000","last_unstale":"2024-05-08T23:30:39.675599+0000","last_undegraded":"2024-05-08T23:30:39.675599+0000","last_fullsized":"2024-05-08T23:30:39.675599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:14.380730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675152+0000","last_change":"2024-05-08T23:30:38.695850+0000","last_active":"2024-05-08T23:30:39.675152+0000","last_peered":"2024-05-08T23:30:39.675152+0000","last_clean":"2024-05-08T23:30:39.675152+0000","last_became_active":"2024-05-08T23:30:38.693296+0000","last_became_peered":"2024-05-08T23:30:38.693296+0000","last_unstale":"2024-05-08T23:30:39.675152+0000","last_undegraded":"2024-05-08T23:30:39.675152+0000","last_fullsized":"2024-05-08T23:30:39.675152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:33.805704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243572+0000","last_change":"2024-05-08T23:30:38.314634+0000","last_active":"2024-05-08T23:30:40.243572+0000","last_peered":"2024-05-08T23:30:40.243572+0000","last_clean":"2024-05-08T23:30:40.243572+0000","last_became_active":"2024-05-08T23:30:38.314368+0000","last_became_peered":"2024-05-08T23:30:38.314368+0000","last_unstale":"2024-05-08T23:30:40.243572+0000","last_undegraded":"2024-05-08T23:30:40.243572+0000","last_fullsized":"2024-05-08T23:30:40.243572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:12:55.004696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.692984+0000","last_change":"2024-05-08T23:30:38.688337+0000","last_active":"2024-05-08T23:30:38.692984+0000","last_peered":"2024-05-08T23:30:38.692984+0000","last_clean":"2024-05-08T23:30:38.692984+0000","last_became_active":"2024-05-08T23:30:38.687594+0000","last_became_peered":"2024-05-08T23:30:38.687594+0000","last_unstale":"2024-05-08T23:30:38.692984+0000","last_undegraded":"2024-05-08T23:30:38.692984+0000","last_fullsized":"2024-05-08T23:30:38.692984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.683265+0000","last_change":"2024-05-08T23:30:38.315204+0000","last_active":"2024-05-08T23:30:38.683265+0000","last_peered":"2024-05-08T23:30:38.683265+0000","last_clean":"2024-05-08T23:30:38.683265+0000","last_became_active":"2024-05-08T23:30:38.314885+0000","last_became_peered":"2024-05-08T23:30:38.314885+0000","last_unstale":"2024-05-08T23:30:38.683265+0000","last_undegraded":"2024-05-08T23:30:38.683265+0000","last_fullsized":"2024-05-08T23:30:38.683265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:03.938472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244273+0000","last_change":"2024-05-08T23:30:38.703534+0000","last_active":"2024-05-08T23:30:40.244273+0000","last_peered":"2024-05-08T23:30:40.244273+0000","last_clean":"2024-05-08T23:30:40.244273+0000","last_became_active":"2024-05-08T23:30:38.703221+0000","last_became_peered":"2024-05-08T23:30:38.703221+0000","last_unstale":"2024-05-08T23:30:40.244273+0000","last_undegraded":"2024-05-08T23:30:40.244273+0000","last_fullsized":"2024-05-08T23:30:40.244273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:58.807960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244382+0000","last_change":"2024-05-08T23:30:38.318632+0000","last_active":"2024-05-08T23:30:40.244382+0000","last_peered":"2024-05-08T23:30:40.244382+0000","last_clean":"2024-05-08T23:30:40.244382+0000","last_became_active":"2024-05-08T23:30:38.318283+0000","last_became_peered":"2024-05-08T23:30:38.318283+0000","last_unstale":"2024-05-08T23:30:40.244382+0000","last_undegraded":"2024-05-08T23:30:40.244382+0000","last_fullsized":"2024-05-08T23:30:40.244382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:00.309450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.695327+0000","last_change":"2024-05-08T23:30:38.691103+0000","last_active":"2024-05-08T23:30:38.695327+0000","last_peered":"2024-05-08T23:30:38.695327+0000","last_clean":"2024-05-08T23:30:38.695327+0000","last_became_active":"2024-05-08T23:30:38.690525+0000","last_became_peered":"2024-05-08T23:30:38.690525+0000","last_unstale":"2024-05-08T23:30:38.695327+0000","last_undegraded":"2024-05-08T23:30:38.695327+0000","last_fullsized":"2024-05-08T23:30:38.695327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:21.833335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.696624+0000","last_change":"2024-05-08T23:30:38.317224+0000","last_active":"2024-05-08T23:30:38.696624+0000","last_peered":"2024-05-08T23:30:38.696624+0000","last_clean":"2024-05-08T23:30:38.696624+0000","last_became_active":"2024-05-08T23:30:38.316980+0000","last_became_peered":"2024-05-08T23:30:38.316980+0000","last_unstale":"2024-05-08T23:30:38.696624+0000","last_undegraded":"2024-05-08T23:30:38.696624+0000","last_fullsized":"2024-05-08T23:30:38.696624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:10.501763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959964+0000","last_change":"2024-05-08T23:30:38.703383+0000","last_active":"2024-05-08T23:30:39.959964+0000","last_peered":"2024-05-08T23:30:39.959964+0000","last_clean":"2024-05-08T23:30:39.959964+0000","last_became_active":"2024-05-08T23:30:38.703119+0000","last_became_peered":"2024-05-08T23:30:38.703119+0000","last_unstale":"2024-05-08T23:30:39.959964+0000","last_undegraded":"2024-05-08T23:30:39.959964+0000","last_fullsized":"2024-05-08T23:30:39.959964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:01.319202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.710658+0000","last_change":"2024-05-08T23:30:38.315586+0000","last_active":"2024-05-08T23:30:38.710658+0000","last_peered":"2024-05-08T23:30:38.710658+0000","last_clean":"2024-05-08T23:30:38.710658+0000","last_became_active":"2024-05-08T23:30:38.315274+0000","last_became_peered":"2024-05-08T23:30:38.315274+0000","last_unstale":"2024-05-08T23:30:38.710658+0000","last_undegraded":"2024-05-08T23:30:38.710658+0000","last_fullsized":"2024-05-08T23:30:38.710658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:31.924157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711690+0000","last_change":"2024-05-08T23:30:38.705407+0000","last_active":"2024-05-08T23:30:38.711690+0000","last_peered":"2024-05-08T23:30:38.711690+0000","last_clean":"2024-05-08T23:30:38.711690+0000","last_became_active":"2024-05-08T23:30:38.705035+0000","last_became_peered":"2024-05-08T23:30:38.705035+0000","last_unstale":"2024-05-08T23:30:38.711690+0000","last_undegraded":"2024-05-08T23:30:38.711690+0000","last_fullsized":"2024-05-08T23:30:38.711690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:49.850123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.696642+0000","last_change":"2024-05-08T23:30:38.308541+0000","last_active":"2024-05-08T23:30:38.696642+0000","last_peered":"2024-05-08T23:30:38.696642+0000","last_clean":"2024-05-08T23:30:38.696642+0000","last_became_active":"2024-05-08T23:30:38.308285+0000","last_became_peered":"2024-05-08T23:30:38.308285+0000","last_unstale":"2024-05-08T23:30:38.696642+0000","last_undegraded":"2024-05-08T23:30:38.696642+0000","last_fullsized":"2024-05-08T23:30:38.696642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:12.981854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.703316+0000","last_change":"2024-05-08T23:30:38.702752+0000","last_active":"2024-05-08T23:30:38.703316+0000","last_peered":"2024-05-08T23:30:38.703316+0000","last_clean":"2024-05-08T23:30:38.703316+0000","last_became_active":"2024-05-08T23:30:38.702534+0000","last_became_peered":"2024-05-08T23:30:38.702534+0000","last_unstale":"2024-05-08T23:30:38.703316+0000","last_undegraded":"2024-05-08T23:30:38.703316+0000","last_fullsized":"2024-05-08T23:30:38.703316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:40.230149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243826+0000","last_change":"2024-05-08T23:30:38.318065+0000","last_active":"2024-05-08T23:30:40.243826+0000","last_peered":"2024-05-08T23:30:40.243826+0000","last_clean":"2024-05-08T23:30:40.243826+0000","last_became_active":"2024-05-08T23:30:38.317810+0000","last_became_peered":"2024-05-08T23:30:38.317810+0000","last_unstale":"2024-05-08T23:30:40.243826+0000","last_undegraded":"2024-05-08T23:30:40.243826+0000","last_fullsized":"2024-05-08T23:30:40.243826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:39.460999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.712773+0000","last_change":"2024-05-08T23:30:38.711628+0000","last_active":"2024-05-08T23:30:38.712773+0000","last_peered":"2024-05-08T23:30:38.712773+0000","last_clean":"2024-05-08T23:30:38.712773+0000","last_became_active":"2024-05-08T23:30:38.711306+0000","last_became_peered":"2024-05-08T23:30:38.711306+0000","last_unstale":"2024-05-08T23:30:38.712773+0000","last_undegraded":"2024-05-08T23:30:38.712773+0000","last_fullsized":"2024-05-08T23:30:38.712773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:34.727934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.683234+0000","last_change":"2024-05-08T23:30:38.314564+0000","last_active":"2024-05-08T23:30:38.683234+0000","last_peered":"2024-05-08T23:30:38.683234+0000","last_clean":"2024-05-08T23:30:38.683234+0000","last_became_active":"2024-05-08T23:30:38.314293+0000","last_became_peered":"2024-05-08T23:30:38.314293+0000","last_unstale":"2024-05-08T23:30:38.683234+0000","last_undegraded":"2024-05-08T23:30:38.683234+0000","last_fullsized":"2024-05-08T23:30:38.683234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.696957+0000","last_change":"2024-05-08T23:30:38.694428+0000","last_active":"2024-05-08T23:30:38.696957+0000","last_peered":"2024-05-08T23:30:38.696957+0000","last_clean":"2024-05-08T23:30:38.696957+0000","last_became_active":"2024-05-08T23:30:38.693827+0000","last_became_peered":"2024-05-08T23:30:38.693827+0000","last_unstale":"2024-05-08T23:30:38.696957+0000","last_undegraded":"2024-05-08T23:30:38.696957+0000","last_fullsized":"2024-05-08T23:30:38.696957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:01.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.697722+0000","last_change":"2024-05-08T23:30:38.309066+0000","last_active":"2024-05-08T23:30:38.697722+0000","last_peered":"2024-05-08T23:30:38.697722+0000","last_clean":"2024-05-08T23:30:38.697722+0000","last_became_active":"2024-05-08T23:30:38.308581+0000","last_became_peered":"2024-05-08T23:30:38.308581+0000","last_unstale":"2024-05-08T23:30:38.697722+0000","last_undegraded":"2024-05-08T23:30:38.697722+0000","last_fullsized":"2024-05-08T23:30:38.697722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711447+0000","last_change":"2024-05-08T23:30:38.704429+0000","last_active":"2024-05-08T23:30:38.711447+0000","last_peered":"2024-05-08T23:30:38.711447+0000","last_clean":"2024-05-08T23:30:38.711447+0000","last_became_active":"2024-05-08T23:30:38.703989+0000","last_became_peered":"2024-05-08T23:30:38.703989+0000","last_unstale":"2024-05-08T23:30:38.711447+0000","last_undegraded":"2024-05-08T23:30:38.711447+0000","last_fullsized":"2024-05-08T23:30:38.711447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:56.584934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.960069+0000","last_change":"2024-05-08T23:30:38.309970+0000","last_active":"2024-05-08T23:30:39.960069+0000","last_peered":"2024-05-08T23:30:39.960069+0000","last_clean":"2024-05-08T23:30:39.960069+0000","last_became_active":"2024-05-08T23:30:38.308960+0000","last_became_peered":"2024-05-08T23:30:38.308960+0000","last_unstale":"2024-05-08T23:30:39.960069+0000","last_undegraded":"2024-05-08T23:30:39.960069+0000","last_fullsized":"2024-05-08T23:30:39.960069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:41.142396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711654+0000","last_change":"2024-05-08T23:30:38.705438+0000","last_active":"2024-05-08T23:30:38.711654+0000","last_peered":"2024-05-08T23:30:38.711654+0000","last_clean":"2024-05-08T23:30:38.711654+0000","last_became_active":"2024-05-08T23:30:38.705141+0000","last_became_peered":"2024-05-08T23:30:38.705141+0000","last_unstale":"2024-05-08T23:30:38.711654+0000","last_undegraded":"2024-05-08T23:30:38.711654+0000","last_fullsized":"2024-05-08T23:30:38.711654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:37.465394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.698478+0000","last_change":"2024-05-08T23:30:38.312114+0000","last_active":"2024-05-08T23:30:38.698478+0000","last_peered":"2024-05-08T23:30:38.698478+0000","last_clean":"2024-05-08T23:30:38.698478+0000","last_became_active":"2024-05-08T23:30:38.311826+0000","last_became_peered":"2024-05-08T23:30:38.311826+0000","last_unstale":"2024-05-08T23:30:38.698478+0000","last_undegraded":"2024-05-08T23:30:38.698478+0000","last_fullsized":"2024-05-08T23:30:38.698478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:12:55.004696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711876+0000","last_change":"2024-05-08T23:30:38.708759+0000","last_active":"2024-05-08T23:30:38.711876+0000","last_peered":"2024-05-08T23:30:38.711876+0000","last_clean":"2024-05-08T23:30:38.711876+0000","last_became_active":"2024-05-08T23:30:38.708506+0000","last_became_peered":"2024-05-08T23:30:38.708506+0000","last_unstale":"2024-05-08T23:30:38.711876+0000","last_undegraded":"2024-05-08T23:30:38.711876+0000","last_fullsized":"2024-05-08T23:30:38.711876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:15.958328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.698588+0000","last_change":"2024-05-08T23:30:38.314282+0000","last_active":"2024-05-08T23:30:38.698588+0000","last_peered":"2024-05-08T23:30:38.698588+0000","last_clean":"2024-05-08T23:30:38.698588+0000","last_became_active":"2024-05-08T23:30:38.314040+0000","last_became_peered":"2024-05-08T23:30:38.314040+0000","last_unstale":"2024-05-08T23:30:38.698588+0000","last_undegraded":"2024-05-08T23:30:38.698588+0000","last_fullsized":"2024-05-08T23:30:38.698588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:39.460999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243979+0000","last_change":"2024-05-08T23:30:38.688690+0000","last_active":"2024-05-08T23:30:40.243979+0000","last_peered":"2024-05-08T23:30:40.243979+0000","last_clean":"2024-05-08T23:30:40.243979+0000","last_became_active":"2024-05-08T23:30:38.688342+0000","last_became_peered":"2024-05-08T23:30:38.688342+0000","last_unstale":"2024-05-08T23:30:40.243979+0000","last_undegraded":"2024-05-08T23:30:40.243979+0000","last_fullsized":"2024-05-08T23:30:40.243979+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:13.779218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.681408+0000","last_change":"2024-05-08T23:30:38.314058+0000","last_active":"2024-05-08T23:30:38.681408+0000","last_peered":"2024-05-08T23:30:38.681408+0000","last_clean":"2024-05-08T23:30:38.681408+0000","last_became_active":"2024-05-08T23:30:38.313813+0000","last_became_peered":"2024-05-08T23:30:38.313813+0000","last_unstale":"2024-05-08T23:30:38.681408+0000","last_undegraded":"2024-05-08T23:30:38.681408+0000","last_fullsized":"2024-05-08T23:30:38.681408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:14.380730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711783+0000","last_change":"2024-05-08T23:30:38.707800+0000","last_active":"2024-05-08T23:30:38.711783+0000","last_peered":"2024-05-08T23:30:38.711783+0000","last_clean":"2024-05-08T23:30:38.711783+0000","last_became_active":"2024-05-08T23:30:38.706575+0000","last_became_peered":"2024-05-08T23:30:38.706575+0000","last_unstale":"2024-05-08T23:30:38.711783+0000","last_undegraded":"2024-05-08T23:30:38.711783+0000","last_fullsized":"2024-05-08T23:30:38.711783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:16.601918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.310713+0000","last_change":"2024-05-08T23:30:27.226484+0000","last_active":"2024-05-08T23:30:38.310713+0000","last_peered":"2024-05-08T23:30:38.310713+0000","last_clean":"2024-05-08T23:30:38.310713+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:30:38.310713+0000","last_undegraded":"2024-05-08T23:30:38.310713+0000","last_fullsized":"2024-05-08T23:30:38.310713+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_clean_scrub_stamp":"2024-05-08T23:30:26.204118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:14.267287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243573+0000","last_change":"2024-05-08T23:30:38.317146+0000","last_active":"2024-05-08T23:30:40.243573+0000","last_peered":"2024-05-08T23:30:40.243573+0000","last_clean":"2024-05-08T23:30:40.243573+0000","last_became_active":"2024-05-08T23:30:38.316813+0000","last_became_peered":"2024-05-08T23:30:38.316813+0000","last_unstale":"2024-05-08T23:30:40.243573+0000","last_undegraded":"2024-05-08T23:30:40.243573+0000","last_fullsized":"2024-05-08T23:30:40.243573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:03.938472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675574+0000","last_change":"2024-05-08T23:30:38.690153+0000","last_active":"2024-05-08T23:30:39.675574+0000","last_peered":"2024-05-08T23:30:39.675574+0000","last_clean":"2024-05-08T23:30:39.675574+0000","last_became_active":"2024-05-08T23:30:38.689929+0000","last_became_peered":"2024-05-08T23:30:38.689929+0000","last_unstale":"2024-05-08T23:30:39.675574+0000","last_undegraded":"2024-05-08T23:30:39.675574+0000","last_fullsized":"2024-05-08T23:30:39.675574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:07.682170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.695745+0000","last_change":"2024-05-08T23:30:38.319918+0000","last_active":"2024-05-08T23:30:38.695745+0000","last_peered":"2024-05-08T23:30:38.695745+0000","last_clean":"2024-05-08T23:30:38.695745+0000","last_became_active":"2024-05-08T23:30:38.319606+0000","last_became_peered":"2024-05-08T23:30:38.319606+0000","last_unstale":"2024-05-08T23:30:38.695745+0000","last_undegraded":"2024-05-08T23:30:38.695745+0000","last_fullsized":"2024-05-08T23:30:38.695745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:24.905425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.685916+0000","last_change":"2024-05-08T23:30:38.681448+0000","last_active":"2024-05-08T23:30:38.685916+0000","last_peered":"2024-05-08T23:30:38.685916+0000","last_clean":"2024-05-08T23:30:38.685916+0000","last_became_active":"2024-05-08T23:30:38.681002+0000","last_became_peered":"2024-05-08T23:30:38.681002+0000","last_unstale":"2024-05-08T23:30:38.685916+0000","last_undegraded":"2024-05-08T23:30:38.685916+0000","last_fullsized":"2024-05-08T23:30:38.685916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:40.104243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.681396+0000","last_change":"2024-05-08T23:30:38.312104+0000","last_active":"2024-05-08T23:30:38.681396+0000","last_peered":"2024-05-08T23:30:38.681396+0000","last_clean":"2024-05-08T23:30:38.681396+0000","last_became_active":"2024-05-08T23:30:38.311833+0000","last_became_peered":"2024-05-08T23:30:38.311833+0000","last_unstale":"2024-05-08T23:30:38.681396+0000","last_undegraded":"2024-05-08T23:30:38.681396+0000","last_fullsized":"2024-05-08T23:30:38.681396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:31.924157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.689693+0000","last_change":"2024-05-08T23:30:38.686436+0000","last_active":"2024-05-08T23:30:38.689693+0000","last_peered":"2024-05-08T23:30:38.689693+0000","last_clean":"2024-05-08T23:30:38.689693+0000","last_became_active":"2024-05-08T23:30:38.686176+0000","last_became_peered":"2024-05-08T23:30:38.686176+0000","last_unstale":"2024-05-08T23:30:38.689693+0000","last_undegraded":"2024-05-08T23:30:38.689693+0000","last_fullsized":"2024-05-08T23:30:38.689693+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:41.133153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244021+0000","last_change":"2024-05-08T23:30:38.317558+0000","last_active":"2024-05-08T23:30:40.244021+0000","last_peered":"2024-05-08T23:30:40.244021+0000","last_clean":"2024-05-08T23:30:40.244021+0000","last_became_active":"2024-05-08T23:30:38.317296+0000","last_became_peered":"2024-05-08T23:30:38.317296+0000","last_unstale":"2024-05-08T23:30:40.244021+0000","last_undegraded":"2024-05-08T23:30:40.244021+0000","last_fullsized":"2024-05-08T23:30:40.244021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959765+0000","last_change":"2024-05-08T23:30:38.682924+0000","last_active":"2024-05-08T23:30:39.959765+0000","last_peered":"2024-05-08T23:30:39.959765+0000","last_clean":"2024-05-08T23:30:39.959765+0000","last_became_active":"2024-05-08T23:30:38.682112+0000","last_became_peered":"2024-05-08T23:30:38.682112+0000","last_unstale":"2024-05-08T23:30:39.959765+0000","last_undegraded":"2024-05-08T23:30:39.959765+0000","last_fullsized":"2024-05-08T23:30:39.959765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:04.948224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.676243+0000","last_change":"2024-05-08T23:30:38.317398+0000","last_active":"2024-05-08T23:30:38.676243+0000","last_peered":"2024-05-08T23:30:38.676243+0000","last_clean":"2024-05-08T23:30:38.676243+0000","last_became_active":"2024-05-08T23:30:38.317167+0000","last_became_peered":"2024-05-08T23:30:38.317167+0000","last_unstale":"2024-05-08T23:30:38.676243+0000","last_undegraded":"2024-05-08T23:30:38.676243+0000","last_fullsized":"2024-05-08T23:30:38.676243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244386+0000","last_change":"2024-05-08T23:30:38.693098+0000","last_active":"2024-05-08T23:30:40.244386+0000","last_peered":"2024-05-08T23:30:40.244386+0000","last_clean":"2024-05-08T23:30:40.244386+0000","last_became_active":"2024-05-08T23:30:38.692852+0000","last_became_peered":"2024-05-08T23:30:38.692852+0000","last_unstale":"2024-05-08T23:30:40.244386+0000","last_undegraded":"2024-05-08T23:30:40.244386+0000","last_fullsized":"2024-05-08T23:30:40.244386+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:02.678862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.701929+0000","last_change":"2024-05-08T23:30:38.320883+0000","last_active":"2024-05-08T23:30:38.701929+0000","last_peered":"2024-05-08T23:30:38.701929+0000","last_clean":"2024-05-08T23:30:38.701929+0000","last_became_active":"2024-05-08T23:30:38.320610+0000","last_became_peered":"2024-05-08T23:30:38.320610+0000","last_unstale":"2024-05-08T23:30:38.701929+0000","last_undegraded":"2024-05-08T23:30:38.701929+0000","last_fullsized":"2024-05-08T23:30:38.701929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:02.623735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.689580+0000","last_change":"2024-05-08T23:30:38.687478+0000","last_active":"2024-05-08T23:30:38.689580+0000","last_peered":"2024-05-08T23:30:38.689580+0000","last_clean":"2024-05-08T23:30:38.689580+0000","last_became_active":"2024-05-08T23:30:38.686229+0000","last_became_peered":"2024-05-08T23:30:38.686229+0000","last_unstale":"2024-05-08T23:30:38.689580+0000","last_undegraded":"2024-05-08T23:30:38.689580+0000","last_fullsized":"2024-05-08T23:30:38.689580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:37.994049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.677846+0000","last_change":"2024-05-08T23:30:38.318046+0000","last_active":"2024-05-08T23:30:38.677846+0000","last_peered":"2024-05-08T23:30:38.677846+0000","last_clean":"2024-05-08T23:30:38.677846+0000","last_became_active":"2024-05-08T23:30:38.317757+0000","last_became_peered":"2024-05-08T23:30:38.317757+0000","last_unstale":"2024-05-08T23:30:38.677846+0000","last_undegraded":"2024-05-08T23:30:38.677846+0000","last_fullsized":"2024-05-08T23:30:38.677846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:00.309450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676059+0000","last_change":"2024-05-08T23:30:38.706013+0000","last_active":"2024-05-08T23:30:39.676059+0000","last_peered":"2024-05-08T23:30:39.676059+0000","last_clean":"2024-05-08T23:30:39.676059+0000","last_became_active":"2024-05-08T23:30:38.705709+0000","last_became_peered":"2024-05-08T23:30:38.705709+0000","last_unstale":"2024-05-08T23:30:39.676059+0000","last_undegraded":"2024-05-08T23:30:39.676059+0000","last_fullsized":"2024-05-08T23:30:39.676059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:01.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.701098+0000","last_change":"2024-05-08T23:30:38.317850+0000","last_active":"2024-05-08T23:30:38.701098+0000","last_peered":"2024-05-08T23:30:38.701098+0000","last_clean":"2024-05-08T23:30:38.701098+0000","last_became_active":"2024-05-08T23:30:38.317598+0000","last_became_peered":"2024-05-08T23:30:38.317598+0000","last_unstale":"2024-05-08T23:30:38.701098+0000","last_undegraded":"2024-05-08T23:30:38.701098+0000","last_fullsized":"2024-05-08T23:30:38.701098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:39.460999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.681899+0000","last_change":"2024-05-08T23:30:38.679213+0000","last_active":"2024-05-08T23:30:38.681899+0000","last_peered":"2024-05-08T23:30:38.681899+0000","last_clean":"2024-05-08T23:30:38.681899+0000","last_became_active":"2024-05-08T23:30:38.678935+0000","last_became_peered":"2024-05-08T23:30:38.678935+0000","last_unstale":"2024-05-08T23:30:38.681899+0000","last_undegraded":"2024-05-08T23:30:38.681899+0000","last_fullsized":"2024-05-08T23:30:38.681899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:39.006348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.695339+0000","last_change":"2024-05-08T23:30:38.320666+0000","last_active":"2024-05-08T23:30:38.695339+0000","last_peered":"2024-05-08T23:30:38.695339+0000","last_clean":"2024-05-08T23:30:38.695339+0000","last_became_active":"2024-05-08T23:30:38.320387+0000","last_became_peered":"2024-05-08T23:30:38.320387+0000","last_unstale":"2024-05-08T23:30:38.695339+0000","last_undegraded":"2024-05-08T23:30:38.695339+0000","last_fullsized":"2024-05-08T23:30:38.695339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:35.811032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959476+0000","last_change":"2024-05-08T23:30:38.692172+0000","last_active":"2024-05-08T23:30:39.959476+0000","last_peered":"2024-05-08T23:30:39.959476+0000","last_clean":"2024-05-08T23:30:39.959476+0000","last_became_active":"2024-05-08T23:30:38.688675+0000","last_became_peered":"2024-05-08T23:30:38.688675+0000","last_unstale":"2024-05-08T23:30:39.959476+0000","last_undegraded":"2024-05-08T23:30:39.959476+0000","last_fullsized":"2024-05-08T23:30:39.959476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:40.470751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.676354+0000","last_change":"2024-05-08T23:30:38.319809+0000","last_active":"2024-05-08T23:30:38.676354+0000","last_peered":"2024-05-08T23:30:38.676354+0000","last_clean":"2024-05-08T23:30:38.676354+0000","last_became_active":"2024-05-08T23:30:38.319500+0000","last_became_peered":"2024-05-08T23:30:38.319500+0000","last_unstale":"2024-05-08T23:30:38.676354+0000","last_undegraded":"2024-05-08T23:30:38.676354+0000","last_fullsized":"2024-05-08T23:30:38.676354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:06.433390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675251+0000","last_change":"2024-05-08T23:30:38.694080+0000","last_active":"2024-05-08T23:30:39.675251+0000","last_peered":"2024-05-08T23:30:39.675251+0000","last_clean":"2024-05-08T23:30:39.675251+0000","last_became_active":"2024-05-08T23:30:38.693469+0000","last_became_peered":"2024-05-08T23:30:38.693469+0000","last_unstale":"2024-05-08T23:30:39.675251+0000","last_undegraded":"2024-05-08T23:30:39.675251+0000","last_fullsized":"2024-05-08T23:30:39.675251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:07.222797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.676422+0000","last_change":"2024-05-08T23:30:38.316969+0000","last_active":"2024-05-08T23:30:38.676422+0000","last_peered":"2024-05-08T23:30:38.676422+0000","last_clean":"2024-05-08T23:30:38.676422+0000","last_became_active":"2024-05-08T23:30:38.316648+0000","last_became_peered":"2024-05-08T23:30:38.316648+0000","last_unstale":"2024-05-08T23:30:38.676422+0000","last_undegraded":"2024-05-08T23:30:38.676422+0000","last_fullsized":"2024-05-08T23:30:38.676422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:03.938472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675646+0000","last_change":"2024-05-08T23:30:38.696097+0000","last_active":"2024-05-08T23:30:39.675646+0000","last_peered":"2024-05-08T23:30:39.675646+0000","last_clean":"2024-05-08T23:30:39.675646+0000","last_became_active":"2024-05-08T23:30:38.695832+0000","last_became_peered":"2024-05-08T23:30:38.695832+0000","last_unstale":"2024-05-08T23:30:39.675646+0000","last_undegraded":"2024-05-08T23:30:39.675646+0000","last_fullsized":"2024-05-08T23:30:39.675646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:12.789913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711847+0000","last_change":"2024-05-08T23:30:38.320025+0000","last_active":"2024-05-08T23:30:38.711847+0000","last_peered":"2024-05-08T23:30:38.711847+0000","last_clean":"2024-05-08T23:30:38.711847+0000","last_became_active":"2024-05-08T23:30:38.319609+0000","last_became_peered":"2024-05-08T23:30:38.319609+0000","last_unstale":"2024-05-08T23:30:38.711847+0000","last_undegraded":"2024-05-08T23:30:38.711847+0000","last_fullsized":"2024-05-08T23:30:38.711847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:42.475179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.700934+0000","last_change":"2024-05-08T23:30:38.699456+0000","last_active":"2024-05-08T23:30:38.700934+0000","last_peered":"2024-05-08T23:30:38.700934+0000","last_clean":"2024-05-08T23:30:38.700934+0000","last_became_active":"2024-05-08T23:30:38.699013+0000","last_became_peered":"2024-05-08T23:30:38.699013+0000","last_unstale":"2024-05-08T23:30:38.700934+0000","last_undegraded":"2024-05-08T23:30:38.700934+0000","last_fullsized":"2024-05-08T23:30:38.700934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:37.994049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244260+0000","last_change":"2024-05-08T23:30:38.695946+0000","last_active":"2024-05-08T23:30:40.244260+0000","last_peered":"2024-05-08T23:30:40.244260+0000","last_clean":"2024-05-08T23:30:40.244260+0000","last_became_active":"2024-05-08T23:30:38.695704+0000","last_became_peered":"2024-05-08T23:30:38.695704+0000","last_unstale":"2024-05-08T23:30:40.244260+0000","last_undegraded":"2024-05-08T23:30:40.244260+0000","last_fullsized":"2024-05-08T23:30:40.244260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.710659+0000","last_change":"2024-05-08T23:30:38.319876+0000","last_active":"2024-05-08T23:30:38.710659+0000","last_peered":"2024-05-08T23:30:38.710659+0000","last_clean":"2024-05-08T23:30:38.710659+0000","last_became_active":"2024-05-08T23:30:38.319589+0000","last_became_peered":"2024-05-08T23:30:38.319589+0000","last_unstale":"2024-05-08T23:30:38.710659+0000","last_undegraded":"2024-05-08T23:30:38.710659+0000","last_fullsized":"2024-05-08T23:30:38.710659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:12.479867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243895+0000","last_change":"2024-05-08T23:30:38.697783+0000","last_active":"2024-05-08T23:30:40.243895+0000","last_peered":"2024-05-08T23:30:40.243895+0000","last_clean":"2024-05-08T23:30:40.243895+0000","last_became_active":"2024-05-08T23:30:38.697295+0000","last_became_peered":"2024-05-08T23:30:38.697295+0000","last_unstale":"2024-05-08T23:30:40.243895+0000","last_undegraded":"2024-05-08T23:30:40.243895+0000","last_fullsized":"2024-05-08T23:30:40.243895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:12.789913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711551+0000","last_change":"2024-05-08T23:30:38.320543+0000","last_active":"2024-05-08T23:30:38.711551+0000","last_peered":"2024-05-08T23:30:38.711551+0000","last_clean":"2024-05-08T23:30:38.711551+0000","last_became_active":"2024-05-08T23:30:38.320231+0000","last_became_peered":"2024-05-08T23:30:38.320231+0000","last_unstale":"2024-05-08T23:30:38.711551+0000","last_undegraded":"2024-05-08T23:30:38.711551+0000","last_fullsized":"2024-05-08T23:30:38.711551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:08:09.987948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.695345+0000","last_change":"2024-05-08T23:30:38.691260+0000","last_active":"2024-05-08T23:30:38.695345+0000","last_peered":"2024-05-08T23:30:38.695345+0000","last_clean":"2024-05-08T23:30:38.695345+0000","last_became_active":"2024-05-08T23:30:38.690856+0000","last_became_peered":"2024-05-08T23:30:38.690856+0000","last_unstale":"2024-05-08T23:30:38.695345+0000","last_undegraded":"2024-05-08T23:30:38.695345+0000","last_fullsized":"2024-05-08T23:30:38.695345+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:58.926788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676224+0000","last_change":"2024-05-08T23:30:38.311855+0000","last_active":"2024-05-08T23:30:39.676224+0000","last_peered":"2024-05-08T23:30:39.676224+0000","last_clean":"2024-05-08T23:30:39.676224+0000","last_became_active":"2024-05-08T23:30:38.311542+0000","last_became_peered":"2024-05-08T23:30:38.311542+0000","last_unstale":"2024-05-08T23:30:39.676224+0000","last_undegraded":"2024-05-08T23:30:39.676224+0000","last_fullsized":"2024-05-08T23:30:39.676224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:30.035897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676005+0000","last_change":"2024-05-08T23:30:38.688998+0000","last_active":"2024-05-08T23:30:39.676005+0000","last_peered":"2024-05-08T23:30:39.676005+0000","last_clean":"2024-05-08T23:30:39.676005+0000","last_became_active":"2024-05-08T23:30:38.688723+0000","last_became_peered":"2024-05-08T23:30:38.688723+0000","last_unstale":"2024-05-08T23:30:39.676005+0000","last_undegraded":"2024-05-08T23:30:39.676005+0000","last_fullsized":"2024-05-08T23:30:39.676005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:41.133153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.700036+0000","last_change":"2024-05-08T23:30:38.315584+0000","last_active":"2024-05-08T23:30:38.700036+0000","last_peered":"2024-05-08T23:30:38.700036+0000","last_clean":"2024-05-08T23:30:38.700036+0000","last_became_active":"2024-05-08T23:30:38.315234+0000","last_became_peered":"2024-05-08T23:30:38.315234+0000","last_unstale":"2024-05-08T23:30:38.700036+0000","last_undegraded":"2024-05-08T23:30:38.700036+0000","last_fullsized":"2024-05-08T23:30:38.700036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:02.623735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":72078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":70604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4830000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":70640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":78378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]}]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]}]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":654107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":661520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:30:42.710 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:30:42.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.817+0000 7fe4af3f2700 1 Processor -- start 2024-05-08T23:30:42.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.817+0000 7fe4af3f2700 1 -- start start 2024-05-08T23:30:42.823 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.817+0000 7fe4af3f2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a8159a80 0x7fe4a8159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.823 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.817+0000 7fe4af3f2700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a80b3590 con 0x7fe4a80709a0 2024-05-08T23:30:42.823 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.817+0000 7fe4af3f2700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a80b5020 con 0x7fe4a81596a0 2024-05-08T23:30:42.823 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.817+0000 7fe4af3f2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a80b24f0 con 0x7fe4a8159a80 2024-05-08T23:30:42.823 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.817+0000 7fe4aebf1700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe4a80709a0 0x7fe4a8070e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55000/0 (socket says 172.21.15.81:55000) 2024-05-08T23:30:42.823 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.817+0000 7fe4aebf1700 1 -- 172.21.15.81:0/4104629988 learned_addr learned my addr 172.21.15.81:0/4104629988 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:42.823 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.817+0000 7fe4adbef700 1 --2- 172.21.15.81:0/4104629988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a8159a80 0x7fe4a8159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.824 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4ad3ee700 1 -- 172.21.15.81:0/4104629988 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772039282 0 0) 0x7fe4a80b3590 con 0x7fe4a80709a0 2024-05-08T23:30:42.825 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4ad3ee700 1 -- 172.21.15.81:0/4104629988 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4940034f0 con 0x7fe4a80709a0 2024-05-08T23:30:42.825 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4ad3ee700 1 -- 172.21.15.81:0/4104629988 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 560082814 0 0) 0x7fe4a80b5020 con 0x7fe4a81596a0 2024-05-08T23:30:42.825 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4adbef700 1 -- 172.21.15.81:0/4104629988 >> v1:172.21.15.117:6789/0 conn(0x7fe4a81596a0 legacy=0x7fe4a80b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.825 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4adbef700 1 -- 172.21.15.81:0/4104629988 >> v1:172.21.15.81:6789/0 conn(0x7fe4a80709a0 legacy=0x7fe4a8070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.825 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4adbef700 1 -- 172.21.15.81:0/4104629988 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a80b27d0 con 0x7fe4a8159a80 2024-05-08T23:30:42.825 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4adbef700 1 --2- 172.21.15.81:0/4104629988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a8159a80 0x7fe4a8159ec0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe498009880 tx=0x7fe498009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6374223dad190c89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:42.825 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4ad3ee700 1 -- 172.21.15.81:0/4104629988 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498011830 con 0x7fe4a8159a80 2024-05-08T23:30:42.825 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4ad3ee700 1 -- 172.21.15.81:0/4104629988 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe498011990 con 0x7fe4a8159a80 2024-05-08T23:30:42.825 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4ad3ee700 1 -- 172.21.15.81:0/4104629988 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498011d20 con 0x7fe4a8159a80 2024-05-08T23:30:42.826 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4af3f2700 1 -- 172.21.15.81:0/4104629988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a8159a80 msgr2=0x7fe4a8159ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.826 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4af3f2700 1 --2- 172.21.15.81:0/4104629988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a8159a80 0x7fe4a8159ec0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe498009880 tx=0x7fe498009b90 comp rx=0 tx=0).stop 2024-05-08T23:30:42.826 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4af3f2700 1 -- 172.21.15.81:0/4104629988 shutdown_connections 2024-05-08T23:30:42.826 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4af3f2700 1 --2- 172.21.15.81:0/4104629988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a8159a80 0x7fe4a8159ec0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.826 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4af3f2700 1 -- 172.21.15.81:0/4104629988 >> 172.21.15.81:0/4104629988 conn(0x7fe4a806c330 msgr2=0x7fe4a806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:42.826 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4af3f2700 1 -- 172.21.15.81:0/4104629988 shutdown_connections 2024-05-08T23:30:42.826 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4af3f2700 1 -- 172.21.15.81:0/4104629988 wait complete. 2024-05-08T23:30:42.827 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.821+0000 7fe4af3f2700 1 Processor -- start 2024-05-08T23:30:42.827 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4af3f2700 1 -- start start 2024-05-08T23:30:42.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4af3f2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a80709a0 0x7fe4a81493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4af3f2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a81596a0 0x7fe4a8150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4af3f2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4a8159a80 0x7fe4a80ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4af3f2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a81522e0 con 0x7fe4a8159a80 2024-05-08T23:30:42.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4af3f2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a81516d0 con 0x7fe4a80709a0 2024-05-08T23:30:42.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4af3f2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a81519b0 con 0x7fe4a81596a0 2024-05-08T23:30:42.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4aebf1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4a8159a80 0x7fe4a80ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4aebf1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4a8159a80 0x7fe4a80ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32940/0 (socket says 172.21.15.81:32940) 2024-05-08T23:30:42.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4aebf1700 1 -- 172.21.15.81:0/1088498265 learned_addr learned my addr 172.21.15.81:0/1088498265 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:42.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4ae3f0700 1 --2- 172.21.15.81:0/1088498265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a80709a0 0x7fe4a81493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4adbef700 1 --2- 172.21.15.81:0/1088498265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a81596a0 0x7fe4a8150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4aebf1700 1 -- 172.21.15.81:0/1088498265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a81596a0 msgr2=0x7fe4a8150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4aebf1700 1 --2- 172.21.15.81:0/1088498265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a81596a0 0x7fe4a8150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4aebf1700 1 -- 172.21.15.81:0/1088498265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a80709a0 msgr2=0x7fe4a81493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:42.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4aebf1700 1 --2- 172.21.15.81:0/1088498265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a80709a0 0x7fe4a81493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:42.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4aebf1700 1 -- 172.21.15.81:0/1088498265 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4980094b0 con 0x7fe4a8159a80 2024-05-08T23:30:42.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4aebf1700 1 --2- 172.21.15.81:0/1088498265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4a8159a80 0x7fe4a80ccdc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000caa0 tx=0x7fe4a000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:42.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe49effd700 1 -- 172.21.15.81:0/1088498265 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0007480 con 0x7fe4a8159a80 2024-05-08T23:30:42.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe49effd700 1 -- 172.21.15.81:0/1088498265 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a00075e0 con 0x7fe4a8159a80 2024-05-08T23:30:42.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4af3f2700 1 -- 172.21.15.81:0/1088498265 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a8151c90 con 0x7fe4a8159a80 2024-05-08T23:30:42.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe49effd700 1 -- 172.21.15.81:0/1088498265 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0007970 con 0x7fe4a8159a80 2024-05-08T23:30:42.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.825+0000 7fe4af3f2700 1 -- 172.21.15.81:0/1088498265 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a8149930 con 0x7fe4a8159a80 2024-05-08T23:30:42.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.829+0000 7fe4af3f2700 1 -- 172.21.15.81:0/1088498265 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a8152490 con 0x7fe4a8159a80 2024-05-08T23:30:42.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.829+0000 7fe49effd700 1 -- 172.21.15.81:0/1088498265 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe4a001b3b0 con 0x7fe4a8159a80 2024-05-08T23:30:42.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.829+0000 7fe49effd700 1 --2- 172.21.15.81:0/1088498265 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe48c066760 0x7fe48c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:42.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.829+0000 7fe49effd700 1 -- 172.21.15.81:0/1088498265 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4a0083940 con 0x7fe4a8159a80 2024-05-08T23:30:42.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.829+0000 7fe4ae3f0700 1 --2- 172.21.15.81:0/1088498265 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe48c066760 0x7fe48c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:42.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.829+0000 7fe4ae3f0700 1 --2- 172.21.15.81:0/1088498265 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe48c066760 0x7fe48c068c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4000f60 tx=0x7fe4a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:42.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:42.837+0000 7fe49effd700 1 -- 172.21.15.81:0/1088498265 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe4a0050d40 con 0x7fe4a8159a80 2024-05-08T23:30:42.927 INFO:tasks.ceph.mds.b.smithi117.stderr:2024-05-08T23:30:42.923+0000 7f4bb16db700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:30:43.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.009+0000 7fe4af3f2700 1 -- 172.21.15.81:0/1088498265 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4a80b3070 con 0x7fe48c066760 2024-05-08T23:30:43.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.021+0000 7fe49effd700 1 -- 172.21.15.81:0/1088498265 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264502 (secure 0 0 0) 0x7fe4a4001d30 con 0x7fe48c066760 2024-05-08T23:30:43.029 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:43.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.025+0000 7fe4af3f2700 1 -- 172.21.15.81:0/1088498265 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe48c066760 msgr2=0x7fe48c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.025+0000 7fe4af3f2700 1 --2- 172.21.15.81:0/1088498265 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe48c066760 0x7fe48c068c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4000f60 tx=0x7fe4a4009040 comp rx=0 tx=0).stop 2024-05-08T23:30:43.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.025+0000 7fe4af3f2700 1 -- 172.21.15.81:0/1088498265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4a8159a80 msgr2=0x7fe4a80ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.025+0000 7fe4af3f2700 1 --2- 172.21.15.81:0/1088498265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4a8159a80 0x7fe4a80ccdc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000caa0 tx=0x7fe4a000cad0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.025+0000 7fe4af3f2700 1 -- 172.21.15.81:0/1088498265 shutdown_connections 2024-05-08T23:30:43.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.025+0000 7fe4af3f2700 1 --2- 172.21.15.81:0/1088498265 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe48c066760 0x7fe48c068c20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.025+0000 7fe4af3f2700 1 --2- 172.21.15.81:0/1088498265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4a8159a80 0x7fe4a80ccdc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.025+0000 7fe4af3f2700 1 --2- 172.21.15.81:0/1088498265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a81596a0 0x7fe4a8150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.025+0000 7fe4af3f2700 1 --2- 172.21.15.81:0/1088498265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a80709a0 0x7fe4a81493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.025+0000 7fe4af3f2700 1 -- 172.21.15.81:0/1088498265 >> 172.21.15.81:0/1088498265 conn(0x7fe4a806c330 msgr2=0x7fe4a806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:43.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.029+0000 7fe4af3f2700 1 -- 172.21.15.81:0/1088498265 shutdown_connections 2024-05-08T23:30:43.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.029+0000 7fe4af3f2700 1 -- 172.21.15.81:0/1088498265 wait complete. 2024-05-08T23:30:43.033 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:30:43.045 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-08T23:30:41.325117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219028,"kb_used_data":2852,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755692,"statfs":{"total":773094113280,"available":772869828608,"internally_reserved":0,"allocated":2920448,"data_stored":1748515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.044014"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711846+0000","last_change":"2024-05-08T23:30:38.707899+0000","last_active":"2024-05-08T23:30:38.711846+0000","last_peered":"2024-05-08T23:30:38.711846+0000","last_clean":"2024-05-08T23:30:38.711846+0000","last_became_active":"2024-05-08T23:30:38.706532+0000","last_became_peered":"2024-05-08T23:30:38.706532+0000","last_unstale":"2024-05-08T23:30:38.711846+0000","last_undegraded":"2024-05-08T23:30:38.711846+0000","last_fullsized":"2024-05-08T23:30:38.711846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:21.109032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.689635+0000","last_change":"2024-05-08T23:30:38.684226+0000","last_active":"2024-05-08T23:30:38.689635+0000","last_peered":"2024-05-08T23:30:38.689635+0000","last_clean":"2024-05-08T23:30:38.689635+0000","last_became_active":"2024-05-08T23:30:38.683958+0000","last_became_peered":"2024-05-08T23:30:38.683958+0000","last_unstale":"2024-05-08T23:30:38.689635+0000","last_undegraded":"2024-05-08T23:30:38.689635+0000","last_fullsized":"2024-05-08T23:30:38.689635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:07.222797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.688209+0000","last_change":"2024-05-08T23:30:38.686289+0000","last_active":"2024-05-08T23:30:38.688209+0000","last_peered":"2024-05-08T23:30:38.688209+0000","last_clean":"2024-05-08T23:30:38.688209+0000","last_became_active":"2024-05-08T23:30:38.685661+0000","last_became_peered":"2024-05-08T23:30:38.685661+0000","last_unstale":"2024-05-08T23:30:38.688209+0000","last_undegraded":"2024-05-08T23:30:38.688209+0000","last_fullsized":"2024-05-08T23:30:38.688209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:34.727934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243711+0000","last_change":"2024-05-08T23:30:38.694133+0000","last_active":"2024-05-08T23:30:40.243711+0000","last_peered":"2024-05-08T23:30:40.243711+0000","last_clean":"2024-05-08T23:30:40.243711+0000","last_became_active":"2024-05-08T23:30:38.693846+0000","last_became_peered":"2024-05-08T23:30:38.693846+0000","last_unstale":"2024-05-08T23:30:40.243711+0000","last_undegraded":"2024-05-08T23:30:40.243711+0000","last_fullsized":"2024-05-08T23:30:40.243711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:44.481684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711895+0000","last_change":"2024-05-08T23:30:38.705630+0000","last_active":"2024-05-08T23:30:38.711895+0000","last_peered":"2024-05-08T23:30:38.711895+0000","last_clean":"2024-05-08T23:30:38.711895+0000","last_became_active":"2024-05-08T23:30:38.705310+0000","last_became_peered":"2024-05-08T23:30:38.705310+0000","last_unstale":"2024-05-08T23:30:38.711895+0000","last_undegraded":"2024-05-08T23:30:38.711895+0000","last_fullsized":"2024-05-08T23:30:38.711895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:41.949561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243496+0000","last_change":"2024-05-08T23:30:38.691423+0000","last_active":"2024-05-08T23:30:40.243496+0000","last_peered":"2024-05-08T23:30:40.243496+0000","last_clean":"2024-05-08T23:30:40.243496+0000","last_became_active":"2024-05-08T23:30:38.690973+0000","last_became_peered":"2024-05-08T23:30:38.690973+0000","last_unstale":"2024-05-08T23:30:40.243496+0000","last_undegraded":"2024-05-08T23:30:40.243496+0000","last_fullsized":"2024-05-08T23:30:40.243496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:42.503600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711302+0000","last_change":"2024-05-08T23:30:38.702702+0000","last_active":"2024-05-08T23:30:38.711302+0000","last_peered":"2024-05-08T23:30:38.711302+0000","last_clean":"2024-05-08T23:30:38.711302+0000","last_became_active":"2024-05-08T23:30:38.702341+0000","last_became_peered":"2024-05-08T23:30:38.702341+0000","last_unstale":"2024-05-08T23:30:38.711302+0000","last_undegraded":"2024-05-08T23:30:38.711302+0000","last_fullsized":"2024-05-08T23:30:38.711302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:05.546774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675055+0000","last_change":"2024-05-08T23:30:38.692708+0000","last_active":"2024-05-08T23:30:39.675055+0000","last_peered":"2024-05-08T23:30:39.675055+0000","last_clean":"2024-05-08T23:30:39.675055+0000","last_became_active":"2024-05-08T23:30:38.692424+0000","last_became_peered":"2024-05-08T23:30:38.692424+0000","last_unstale":"2024-05-08T23:30:39.675055+0000","last_undegraded":"2024-05-08T23:30:39.675055+0000","last_fullsized":"2024-05-08T23:30:39.675055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:58.747769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711863+0000","last_change":"2024-05-08T23:30:38.708610+0000","last_active":"2024-05-08T23:30:38.711863+0000","last_peered":"2024-05-08T23:30:38.711863+0000","last_clean":"2024-05-08T23:30:38.711863+0000","last_became_active":"2024-05-08T23:30:38.708389+0000","last_became_peered":"2024-05-08T23:30:38.708389+0000","last_unstale":"2024-05-08T23:30:38.711863+0000","last_undegraded":"2024-05-08T23:30:38.711863+0000","last_fullsized":"2024-05-08T23:30:38.711863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:41.257114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.693809+0000","last_change":"2024-05-08T23:30:38.690662+0000","last_active":"2024-05-08T23:30:38.693809+0000","last_peered":"2024-05-08T23:30:38.693809+0000","last_clean":"2024-05-08T23:30:38.693809+0000","last_became_active":"2024-05-08T23:30:38.690378+0000","last_became_peered":"2024-05-08T23:30:38.690378+0000","last_unstale":"2024-05-08T23:30:38.693809+0000","last_undegraded":"2024-05-08T23:30:38.693809+0000","last_fullsized":"2024-05-08T23:30:38.693809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:33.598895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.694520+0000","last_change":"2024-05-08T23:30:38.692171+0000","last_active":"2024-05-08T23:30:38.694520+0000","last_peered":"2024-05-08T23:30:38.694520+0000","last_clean":"2024-05-08T23:30:38.694520+0000","last_became_active":"2024-05-08T23:30:38.691611+0000","last_became_peered":"2024-05-08T23:30:38.691611+0000","last_unstale":"2024-05-08T23:30:38.694520+0000","last_undegraded":"2024-05-08T23:30:38.694520+0000","last_fullsized":"2024-05-08T23:30:38.694520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959822+0000","last_change":"2024-05-08T23:30:38.694336+0000","last_active":"2024-05-08T23:30:39.959822+0000","last_peered":"2024-05-08T23:30:39.959822+0000","last_clean":"2024-05-08T23:30:39.959822+0000","last_became_active":"2024-05-08T23:30:38.694038+0000","last_became_peered":"2024-05-08T23:30:38.694038+0000","last_unstale":"2024-05-08T23:30:39.959822+0000","last_undegraded":"2024-05-08T23:30:39.959822+0000","last_fullsized":"2024-05-08T23:30:39.959822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:07.443142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676123+0000","last_change":"2024-05-08T23:30:38.691325+0000","last_active":"2024-05-08T23:30:39.676123+0000","last_peered":"2024-05-08T23:30:39.676123+0000","last_clean":"2024-05-08T23:30:39.676123+0000","last_became_active":"2024-05-08T23:30:38.691074+0000","last_became_peered":"2024-05-08T23:30:38.691074+0000","last_unstale":"2024-05-08T23:30:39.676123+0000","last_undegraded":"2024-05-08T23:30:39.676123+0000","last_fullsized":"2024-05-08T23:30:39.676123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:37.994049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.681493+0000","last_change":"2024-05-08T23:30:38.679460+0000","last_active":"2024-05-08T23:30:38.681493+0000","last_peered":"2024-05-08T23:30:38.681493+0000","last_clean":"2024-05-08T23:30:38.681493+0000","last_became_active":"2024-05-08T23:30:38.678871+0000","last_became_peered":"2024-05-08T23:30:38.678871+0000","last_unstale":"2024-05-08T23:30:38.681493+0000","last_undegraded":"2024-05-08T23:30:38.681493+0000","last_fullsized":"2024-05-08T23:30:38.681493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:37.465394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.694972+0000","last_change":"2024-05-08T23:30:38.692896+0000","last_active":"2024-05-08T23:30:38.694972+0000","last_peered":"2024-05-08T23:30:38.694972+0000","last_clean":"2024-05-08T23:30:38.694972+0000","last_became_active":"2024-05-08T23:30:38.692347+0000","last_became_peered":"2024-05-08T23:30:38.692347+0000","last_unstale":"2024-05-08T23:30:38.694972+0000","last_undegraded":"2024-05-08T23:30:38.694972+0000","last_fullsized":"2024-05-08T23:30:38.694972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:01.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243296+0000","last_change":"2024-05-08T23:30:38.689233+0000","last_active":"2024-05-08T23:30:40.243296+0000","last_peered":"2024-05-08T23:30:40.243296+0000","last_clean":"2024-05-08T23:30:40.243296+0000","last_became_active":"2024-05-08T23:30:38.688984+0000","last_became_peered":"2024-05-08T23:30:38.688984+0000","last_unstale":"2024-05-08T23:30:40.243296+0000","last_undegraded":"2024-05-08T23:30:40.243296+0000","last_fullsized":"2024-05-08T23:30:40.243296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:07.682170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244541+0000","last_change":"2024-05-08T23:30:38.706139+0000","last_active":"2024-05-08T23:30:40.244541+0000","last_peered":"2024-05-08T23:30:40.244541+0000","last_clean":"2024-05-08T23:30:40.244541+0000","last_became_active":"2024-05-08T23:30:38.705878+0000","last_became_peered":"2024-05-08T23:30:38.705878+0000","last_unstale":"2024-05-08T23:30:40.244541+0000","last_undegraded":"2024-05-08T23:30:40.244541+0000","last_fullsized":"2024-05-08T23:30:40.244541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:39.788098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.693318+0000","last_change":"2024-05-08T23:30:38.685704+0000","last_active":"2024-05-08T23:30:38.693318+0000","last_peered":"2024-05-08T23:30:38.693318+0000","last_clean":"2024-05-08T23:30:38.693318+0000","last_became_active":"2024-05-08T23:30:38.685435+0000","last_became_peered":"2024-05-08T23:30:38.685435+0000","last_unstale":"2024-05-08T23:30:38.693318+0000","last_undegraded":"2024-05-08T23:30:38.693318+0000","last_fullsized":"2024-05-08T23:30:38.693318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:07.443142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.712025+0000","last_change":"2024-05-08T23:30:38.710968+0000","last_active":"2024-05-08T23:30:38.712025+0000","last_peered":"2024-05-08T23:30:38.712025+0000","last_clean":"2024-05-08T23:30:38.712025+0000","last_became_active":"2024-05-08T23:30:38.710706+0000","last_became_peered":"2024-05-08T23:30:38.710706+0000","last_unstale":"2024-05-08T23:30:38.712025+0000","last_undegraded":"2024-05-08T23:30:38.712025+0000","last_fullsized":"2024-05-08T23:30:38.712025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:32.727647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.705107+0000","last_change":"2024-05-08T23:30:38.701983+0000","last_active":"2024-05-08T23:30:38.705107+0000","last_peered":"2024-05-08T23:30:38.705107+0000","last_clean":"2024-05-08T23:30:38.705107+0000","last_became_active":"2024-05-08T23:30:38.701386+0000","last_became_peered":"2024-05-08T23:30:38.701386+0000","last_unstale":"2024-05-08T23:30:38.705107+0000","last_undegraded":"2024-05-08T23:30:38.705107+0000","last_fullsized":"2024-05-08T23:30:38.705107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:12.789913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.707115+0000","last_change":"2024-05-08T23:30:38.705814+0000","last_active":"2024-05-08T23:30:38.707115+0000","last_peered":"2024-05-08T23:30:38.707115+0000","last_clean":"2024-05-08T23:30:38.707115+0000","last_became_active":"2024-05-08T23:30:38.705393+0000","last_became_peered":"2024-05-08T23:30:38.705393+0000","last_unstale":"2024-05-08T23:30:38.707115+0000","last_undegraded":"2024-05-08T23:30:38.707115+0000","last_fullsized":"2024-05-08T23:30:38.707115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:35.129372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243643+0000","last_change":"2024-05-08T23:30:38.703498+0000","last_active":"2024-05-08T23:30:40.243643+0000","last_peered":"2024-05-08T23:30:40.243643+0000","last_clean":"2024-05-08T23:30:40.243643+0000","last_became_active":"2024-05-08T23:30:38.703206+0000","last_became_peered":"2024-05-08T23:30:38.703206+0000","last_unstale":"2024-05-08T23:30:40.243643+0000","last_undegraded":"2024-05-08T23:30:40.243643+0000","last_fullsized":"2024-05-08T23:30:40.243643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:58.747769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675186+0000","last_change":"2024-05-08T23:30:38.697689+0000","last_active":"2024-05-08T23:30:39.675186+0000","last_peered":"2024-05-08T23:30:39.675186+0000","last_clean":"2024-05-08T23:30:39.675186+0000","last_became_active":"2024-05-08T23:30:38.697373+0000","last_became_peered":"2024-05-08T23:30:38.697373+0000","last_unstale":"2024-05-08T23:30:39.675186+0000","last_undegraded":"2024-05-08T23:30:39.675186+0000","last_fullsized":"2024-05-08T23:30:39.675186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:42.503600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243343+0000","last_change":"2024-05-08T23:30:38.693170+0000","last_active":"2024-05-08T23:30:40.243343+0000","last_peered":"2024-05-08T23:30:40.243343+0000","last_clean":"2024-05-08T23:30:40.243343+0000","last_became_active":"2024-05-08T23:30:38.692809+0000","last_became_peered":"2024-05-08T23:30:38.692809+0000","last_unstale":"2024-05-08T23:30:40.243343+0000","last_undegraded":"2024-05-08T23:30:40.243343+0000","last_fullsized":"2024-05-08T23:30:40.243343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:51.717280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675706+0000","last_change":"2024-05-08T23:30:38.696080+0000","last_active":"2024-05-08T23:30:39.675706+0000","last_peered":"2024-05-08T23:30:39.675706+0000","last_clean":"2024-05-08T23:30:39.675706+0000","last_became_active":"2024-05-08T23:30:38.695821+0000","last_became_peered":"2024-05-08T23:30:38.695821+0000","last_unstale":"2024-05-08T23:30:39.675706+0000","last_undegraded":"2024-05-08T23:30:39.675706+0000","last_fullsized":"2024-05-08T23:30:39.675706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:33.598895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.710727+0000","last_change":"2024-05-08T23:30:38.704917+0000","last_active":"2024-05-08T23:30:38.710727+0000","last_peered":"2024-05-08T23:30:38.710727+0000","last_clean":"2024-05-08T23:30:38.710727+0000","last_became_active":"2024-05-08T23:30:38.704663+0000","last_became_peered":"2024-05-08T23:30:38.704663+0000","last_unstale":"2024-05-08T23:30:38.710727+0000","last_undegraded":"2024-05-08T23:30:38.710727+0000","last_fullsized":"2024-05-08T23:30:38.710727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:41.133153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.695486+0000","last_change":"2024-05-08T23:30:38.693867+0000","last_active":"2024-05-08T23:30:38.695486+0000","last_peered":"2024-05-08T23:30:38.695486+0000","last_clean":"2024-05-08T23:30:38.695486+0000","last_became_active":"2024-05-08T23:30:38.693637+0000","last_became_peered":"2024-05-08T23:30:38.693637+0000","last_unstale":"2024-05-08T23:30:38.695486+0000","last_undegraded":"2024-05-08T23:30:38.695486+0000","last_fullsized":"2024-05-08T23:30:38.695486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:33.598895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676136+0000","last_change":"2024-05-08T23:30:38.697753+0000","last_active":"2024-05-08T23:30:39.676136+0000","last_peered":"2024-05-08T23:30:39.676136+0000","last_clean":"2024-05-08T23:30:39.676136+0000","last_became_active":"2024-05-08T23:30:38.697337+0000","last_became_peered":"2024-05-08T23:30:38.697337+0000","last_unstale":"2024-05-08T23:30:39.676136+0000","last_undegraded":"2024-05-08T23:30:39.676136+0000","last_fullsized":"2024-05-08T23:30:39.676136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.690367+0000","last_change":"2024-05-08T23:30:38.688908+0000","last_active":"2024-05-08T23:30:38.690367+0000","last_peered":"2024-05-08T23:30:38.690367+0000","last_clean":"2024-05-08T23:30:38.690367+0000","last_became_active":"2024-05-08T23:30:38.688636+0000","last_became_peered":"2024-05-08T23:30:38.688636+0000","last_unstale":"2024-05-08T23:30:38.690367+0000","last_undegraded":"2024-05-08T23:30:38.690367+0000","last_fullsized":"2024-05-08T23:30:38.690367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:41.059680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243784+0000","last_change":"2024-05-08T23:30:38.688336+0000","last_active":"2024-05-08T23:30:40.243784+0000","last_peered":"2024-05-08T23:30:40.243784+0000","last_clean":"2024-05-08T23:30:40.243784+0000","last_became_active":"2024-05-08T23:30:38.688041+0000","last_became_peered":"2024-05-08T23:30:38.688041+0000","last_unstale":"2024-05-08T23:30:40.243784+0000","last_undegraded":"2024-05-08T23:30:40.243784+0000","last_fullsized":"2024-05-08T23:30:40.243784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:17.135855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.694658+0000","last_change":"2024-05-08T23:30:38.692840+0000","last_active":"2024-05-08T23:30:38.694658+0000","last_peered":"2024-05-08T23:30:38.694658+0000","last_clean":"2024-05-08T23:30:38.694658+0000","last_became_active":"2024-05-08T23:30:38.690283+0000","last_became_peered":"2024-05-08T23:30:38.690283+0000","last_unstale":"2024-05-08T23:30:38.694658+0000","last_undegraded":"2024-05-08T23:30:38.694658+0000","last_fullsized":"2024-05-08T23:30:38.694658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:11.511515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.695415+0000","last_change":"2024-05-08T23:30:38.693989+0000","last_active":"2024-05-08T23:30:38.695415+0000","last_peered":"2024-05-08T23:30:38.695415+0000","last_clean":"2024-05-08T23:30:38.695415+0000","last_became_active":"2024-05-08T23:30:38.693491+0000","last_became_peered":"2024-05-08T23:30:38.693491+0000","last_unstale":"2024-05-08T23:30:38.695415+0000","last_undegraded":"2024-05-08T23:30:38.695415+0000","last_fullsized":"2024-05-08T23:30:38.695415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:07.222797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959371+0000","last_change":"2024-05-08T23:30:38.681154+0000","last_active":"2024-05-08T23:30:39.959371+0000","last_peered":"2024-05-08T23:30:39.959371+0000","last_clean":"2024-05-08T23:30:39.959371+0000","last_became_active":"2024-05-08T23:30:38.680826+0000","last_became_peered":"2024-05-08T23:30:38.680826+0000","last_unstale":"2024-05-08T23:30:39.959371+0000","last_undegraded":"2024-05-08T23:30:39.959371+0000","last_fullsized":"2024-05-08T23:30:39.959371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:11.511515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675917+0000","last_change":"2024-05-08T23:30:38.317473+0000","last_active":"2024-05-08T23:30:39.675917+0000","last_peered":"2024-05-08T23:30:39.675917+0000","last_clean":"2024-05-08T23:30:39.675917+0000","last_became_active":"2024-05-08T23:30:38.317235+0000","last_became_peered":"2024-05-08T23:30:38.317235+0000","last_unstale":"2024-05-08T23:30:39.675917+0000","last_undegraded":"2024-05-08T23:30:39.675917+0000","last_fullsized":"2024-05-08T23:30:39.675917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:31.924157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.687211+0000","last_change":"2024-05-08T23:30:38.680780+0000","last_active":"2024-05-08T23:30:38.687211+0000","last_peered":"2024-05-08T23:30:38.687211+0000","last_clean":"2024-05-08T23:30:38.687211+0000","last_became_active":"2024-05-08T23:30:38.680508+0000","last_became_peered":"2024-05-08T23:30:38.680508+0000","last_unstale":"2024-05-08T23:30:38.687211+0000","last_undegraded":"2024-05-08T23:30:38.687211+0000","last_fullsized":"2024-05-08T23:30:38.687211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:24.058714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.699238+0000","last_change":"2024-05-08T23:30:38.309456+0000","last_active":"2024-05-08T23:30:38.699238+0000","last_peered":"2024-05-08T23:30:38.699238+0000","last_clean":"2024-05-08T23:30:38.699238+0000","last_became_active":"2024-05-08T23:30:38.308975+0000","last_became_peered":"2024-05-08T23:30:38.308975+0000","last_unstale":"2024-05-08T23:30:38.699238+0000","last_undegraded":"2024-05-08T23:30:38.699238+0000","last_fullsized":"2024-05-08T23:30:38.699238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:00.309450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675599+0000","last_change":"2024-05-08T23:30:38.317811+0000","last_active":"2024-05-08T23:30:39.675599+0000","last_peered":"2024-05-08T23:30:39.675599+0000","last_clean":"2024-05-08T23:30:39.675599+0000","last_became_active":"2024-05-08T23:30:38.317505+0000","last_became_peered":"2024-05-08T23:30:38.317505+0000","last_unstale":"2024-05-08T23:30:39.675599+0000","last_undegraded":"2024-05-08T23:30:39.675599+0000","last_fullsized":"2024-05-08T23:30:39.675599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:14.380730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675152+0000","last_change":"2024-05-08T23:30:38.695850+0000","last_active":"2024-05-08T23:30:39.675152+0000","last_peered":"2024-05-08T23:30:39.675152+0000","last_clean":"2024-05-08T23:30:39.675152+0000","last_became_active":"2024-05-08T23:30:38.693296+0000","last_became_peered":"2024-05-08T23:30:38.693296+0000","last_unstale":"2024-05-08T23:30:39.675152+0000","last_undegraded":"2024-05-08T23:30:39.675152+0000","last_fullsized":"2024-05-08T23:30:39.675152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:33.805704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243572+0000","last_change":"2024-05-08T23:30:38.314634+0000","last_active":"2024-05-08T23:30:40.243572+0000","last_peered":"2024-05-08T23:30:40.243572+0000","last_clean":"2024-05-08T23:30:40.243572+0000","last_became_active":"2024-05-08T23:30:38.314368+0000","last_became_peered":"2024-05-08T23:30:38.314368+0000","last_unstale":"2024-05-08T23:30:40.243572+0000","last_undegraded":"2024-05-08T23:30:40.243572+0000","last_fullsized":"2024-05-08T23:30:40.243572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:12:55.004696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.692984+0000","last_change":"2024-05-08T23:30:38.688337+0000","last_active":"2024-05-08T23:30:38.692984+0000","last_peered":"2024-05-08T23:30:38.692984+0000","last_clean":"2024-05-08T23:30:38.692984+0000","last_became_active":"2024-05-08T23:30:38.687594+0000","last_became_peered":"2024-05-08T23:30:38.687594+0000","last_unstale":"2024-05-08T23:30:38.692984+0000","last_undegraded":"2024-05-08T23:30:38.692984+0000","last_fullsized":"2024-05-08T23:30:38.692984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.683265+0000","last_change":"2024-05-08T23:30:38.315204+0000","last_active":"2024-05-08T23:30:38.683265+0000","last_peered":"2024-05-08T23:30:38.683265+0000","last_clean":"2024-05-08T23:30:38.683265+0000","last_became_active":"2024-05-08T23:30:38.314885+0000","last_became_peered":"2024-05-08T23:30:38.314885+0000","last_unstale":"2024-05-08T23:30:38.683265+0000","last_undegraded":"2024-05-08T23:30:38.683265+0000","last_fullsized":"2024-05-08T23:30:38.683265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:03.938472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244273+0000","last_change":"2024-05-08T23:30:38.703534+0000","last_active":"2024-05-08T23:30:40.244273+0000","last_peered":"2024-05-08T23:30:40.244273+0000","last_clean":"2024-05-08T23:30:40.244273+0000","last_became_active":"2024-05-08T23:30:38.703221+0000","last_became_peered":"2024-05-08T23:30:38.703221+0000","last_unstale":"2024-05-08T23:30:40.244273+0000","last_undegraded":"2024-05-08T23:30:40.244273+0000","last_fullsized":"2024-05-08T23:30:40.244273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:58.807960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244382+0000","last_change":"2024-05-08T23:30:38.318632+0000","last_active":"2024-05-08T23:30:40.244382+0000","last_peered":"2024-05-08T23:30:40.244382+0000","last_clean":"2024-05-08T23:30:40.244382+0000","last_became_active":"2024-05-08T23:30:38.318283+0000","last_became_peered":"2024-05-08T23:30:38.318283+0000","last_unstale":"2024-05-08T23:30:40.244382+0000","last_undegraded":"2024-05-08T23:30:40.244382+0000","last_fullsized":"2024-05-08T23:30:40.244382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:00.309450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.695327+0000","last_change":"2024-05-08T23:30:38.691103+0000","last_active":"2024-05-08T23:30:38.695327+0000","last_peered":"2024-05-08T23:30:38.695327+0000","last_clean":"2024-05-08T23:30:38.695327+0000","last_became_active":"2024-05-08T23:30:38.690525+0000","last_became_peered":"2024-05-08T23:30:38.690525+0000","last_unstale":"2024-05-08T23:30:38.695327+0000","last_undegraded":"2024-05-08T23:30:38.695327+0000","last_fullsized":"2024-05-08T23:30:38.695327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:21.833335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.696624+0000","last_change":"2024-05-08T23:30:38.317224+0000","last_active":"2024-05-08T23:30:38.696624+0000","last_peered":"2024-05-08T23:30:38.696624+0000","last_clean":"2024-05-08T23:30:38.696624+0000","last_became_active":"2024-05-08T23:30:38.316980+0000","last_became_peered":"2024-05-08T23:30:38.316980+0000","last_unstale":"2024-05-08T23:30:38.696624+0000","last_undegraded":"2024-05-08T23:30:38.696624+0000","last_fullsized":"2024-05-08T23:30:38.696624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:10.501763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959964+0000","last_change":"2024-05-08T23:30:38.703383+0000","last_active":"2024-05-08T23:30:39.959964+0000","last_peered":"2024-05-08T23:30:39.959964+0000","last_clean":"2024-05-08T23:30:39.959964+0000","last_became_active":"2024-05-08T23:30:38.703119+0000","last_became_peered":"2024-05-08T23:30:38.703119+0000","last_unstale":"2024-05-08T23:30:39.959964+0000","last_undegraded":"2024-05-08T23:30:39.959964+0000","last_fullsized":"2024-05-08T23:30:39.959964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:01.319202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.710658+0000","last_change":"2024-05-08T23:30:38.315586+0000","last_active":"2024-05-08T23:30:38.710658+0000","last_peered":"2024-05-08T23:30:38.710658+0000","last_clean":"2024-05-08T23:30:38.710658+0000","last_became_active":"2024-05-08T23:30:38.315274+0000","last_became_peered":"2024-05-08T23:30:38.315274+0000","last_unstale":"2024-05-08T23:30:38.710658+0000","last_undegraded":"2024-05-08T23:30:38.710658+0000","last_fullsized":"2024-05-08T23:30:38.710658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:31.924157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711690+0000","last_change":"2024-05-08T23:30:38.705407+0000","last_active":"2024-05-08T23:30:38.711690+0000","last_peered":"2024-05-08T23:30:38.711690+0000","last_clean":"2024-05-08T23:30:38.711690+0000","last_became_active":"2024-05-08T23:30:38.705035+0000","last_became_peered":"2024-05-08T23:30:38.705035+0000","last_unstale":"2024-05-08T23:30:38.711690+0000","last_undegraded":"2024-05-08T23:30:38.711690+0000","last_fullsized":"2024-05-08T23:30:38.711690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:49.850123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.696642+0000","last_change":"2024-05-08T23:30:38.308541+0000","last_active":"2024-05-08T23:30:38.696642+0000","last_peered":"2024-05-08T23:30:38.696642+0000","last_clean":"2024-05-08T23:30:38.696642+0000","last_became_active":"2024-05-08T23:30:38.308285+0000","last_became_peered":"2024-05-08T23:30:38.308285+0000","last_unstale":"2024-05-08T23:30:38.696642+0000","last_undegraded":"2024-05-08T23:30:38.696642+0000","last_fullsized":"2024-05-08T23:30:38.696642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:12.981854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.703316+0000","last_change":"2024-05-08T23:30:38.702752+0000","last_active":"2024-05-08T23:30:38.703316+0000","last_peered":"2024-05-08T23:30:38.703316+0000","last_clean":"2024-05-08T23:30:38.703316+0000","last_became_active":"2024-05-08T23:30:38.702534+0000","last_became_peered":"2024-05-08T23:30:38.702534+0000","last_unstale":"2024-05-08T23:30:38.703316+0000","last_undegraded":"2024-05-08T23:30:38.703316+0000","last_fullsized":"2024-05-08T23:30:38.703316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:40.230149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243826+0000","last_change":"2024-05-08T23:30:38.318065+0000","last_active":"2024-05-08T23:30:40.243826+0000","last_peered":"2024-05-08T23:30:40.243826+0000","last_clean":"2024-05-08T23:30:40.243826+0000","last_became_active":"2024-05-08T23:30:38.317810+0000","last_became_peered":"2024-05-08T23:30:38.317810+0000","last_unstale":"2024-05-08T23:30:40.243826+0000","last_undegraded":"2024-05-08T23:30:40.243826+0000","last_fullsized":"2024-05-08T23:30:40.243826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:39.460999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.712773+0000","last_change":"2024-05-08T23:30:38.711628+0000","last_active":"2024-05-08T23:30:38.712773+0000","last_peered":"2024-05-08T23:30:38.712773+0000","last_clean":"2024-05-08T23:30:38.712773+0000","last_became_active":"2024-05-08T23:30:38.711306+0000","last_became_peered":"2024-05-08T23:30:38.711306+0000","last_unstale":"2024-05-08T23:30:38.712773+0000","last_undegraded":"2024-05-08T23:30:38.712773+0000","last_fullsized":"2024-05-08T23:30:38.712773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:34.727934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.683234+0000","last_change":"2024-05-08T23:30:38.314564+0000","last_active":"2024-05-08T23:30:38.683234+0000","last_peered":"2024-05-08T23:30:38.683234+0000","last_clean":"2024-05-08T23:30:38.683234+0000","last_became_active":"2024-05-08T23:30:38.314293+0000","last_became_peered":"2024-05-08T23:30:38.314293+0000","last_unstale":"2024-05-08T23:30:38.683234+0000","last_undegraded":"2024-05-08T23:30:38.683234+0000","last_fullsized":"2024-05-08T23:30:38.683234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.696957+0000","last_change":"2024-05-08T23:30:38.694428+0000","last_active":"2024-05-08T23:30:38.696957+0000","last_peered":"2024-05-08T23:30:38.696957+0000","last_clean":"2024-05-08T23:30:38.696957+0000","last_became_active":"2024-05-08T23:30:38.693827+0000","last_became_peered":"2024-05-08T23:30:38.693827+0000","last_unstale":"2024-05-08T23:30:38.696957+0000","last_undegraded":"2024-05-08T23:30:38.696957+0000","last_fullsized":"2024-05-08T23:30:38.696957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:01.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.697722+0000","last_change":"2024-05-08T23:30:38.309066+0000","last_active":"2024-05-08T23:30:38.697722+0000","last_peered":"2024-05-08T23:30:38.697722+0000","last_clean":"2024-05-08T23:30:38.697722+0000","last_became_active":"2024-05-08T23:30:38.308581+0000","last_became_peered":"2024-05-08T23:30:38.308581+0000","last_unstale":"2024-05-08T23:30:38.697722+0000","last_undegraded":"2024-05-08T23:30:38.697722+0000","last_fullsized":"2024-05-08T23:30:38.697722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711447+0000","last_change":"2024-05-08T23:30:38.704429+0000","last_active":"2024-05-08T23:30:38.711447+0000","last_peered":"2024-05-08T23:30:38.711447+0000","last_clean":"2024-05-08T23:30:38.711447+0000","last_became_active":"2024-05-08T23:30:38.703989+0000","last_became_peered":"2024-05-08T23:30:38.703989+0000","last_unstale":"2024-05-08T23:30:38.711447+0000","last_undegraded":"2024-05-08T23:30:38.711447+0000","last_fullsized":"2024-05-08T23:30:38.711447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:56.584934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.960069+0000","last_change":"2024-05-08T23:30:38.309970+0000","last_active":"2024-05-08T23:30:39.960069+0000","last_peered":"2024-05-08T23:30:39.960069+0000","last_clean":"2024-05-08T23:30:39.960069+0000","last_became_active":"2024-05-08T23:30:38.308960+0000","last_became_peered":"2024-05-08T23:30:38.308960+0000","last_unstale":"2024-05-08T23:30:39.960069+0000","last_undegraded":"2024-05-08T23:30:39.960069+0000","last_fullsized":"2024-05-08T23:30:39.960069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:41.142396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711654+0000","last_change":"2024-05-08T23:30:38.705438+0000","last_active":"2024-05-08T23:30:38.711654+0000","last_peered":"2024-05-08T23:30:38.711654+0000","last_clean":"2024-05-08T23:30:38.711654+0000","last_became_active":"2024-05-08T23:30:38.705141+0000","last_became_peered":"2024-05-08T23:30:38.705141+0000","last_unstale":"2024-05-08T23:30:38.711654+0000","last_undegraded":"2024-05-08T23:30:38.711654+0000","last_fullsized":"2024-05-08T23:30:38.711654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:37.465394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.698478+0000","last_change":"2024-05-08T23:30:38.312114+0000","last_active":"2024-05-08T23:30:38.698478+0000","last_peered":"2024-05-08T23:30:38.698478+0000","last_clean":"2024-05-08T23:30:38.698478+0000","last_became_active":"2024-05-08T23:30:38.311826+0000","last_became_peered":"2024-05-08T23:30:38.311826+0000","last_unstale":"2024-05-08T23:30:38.698478+0000","last_undegraded":"2024-05-08T23:30:38.698478+0000","last_fullsized":"2024-05-08T23:30:38.698478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:12:55.004696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711876+0000","last_change":"2024-05-08T23:30:38.708759+0000","last_active":"2024-05-08T23:30:38.711876+0000","last_peered":"2024-05-08T23:30:38.711876+0000","last_clean":"2024-05-08T23:30:38.711876+0000","last_became_active":"2024-05-08T23:30:38.708506+0000","last_became_peered":"2024-05-08T23:30:38.708506+0000","last_unstale":"2024-05-08T23:30:38.711876+0000","last_undegraded":"2024-05-08T23:30:38.711876+0000","last_fullsized":"2024-05-08T23:30:38.711876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:15.958328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.698588+0000","last_change":"2024-05-08T23:30:38.314282+0000","last_active":"2024-05-08T23:30:38.698588+0000","last_peered":"2024-05-08T23:30:38.698588+0000","last_clean":"2024-05-08T23:30:38.698588+0000","last_became_active":"2024-05-08T23:30:38.314040+0000","last_became_peered":"2024-05-08T23:30:38.314040+0000","last_unstale":"2024-05-08T23:30:38.698588+0000","last_undegraded":"2024-05-08T23:30:38.698588+0000","last_fullsized":"2024-05-08T23:30:38.698588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:39.460999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243979+0000","last_change":"2024-05-08T23:30:38.688690+0000","last_active":"2024-05-08T23:30:40.243979+0000","last_peered":"2024-05-08T23:30:40.243979+0000","last_clean":"2024-05-08T23:30:40.243979+0000","last_became_active":"2024-05-08T23:30:38.688342+0000","last_became_peered":"2024-05-08T23:30:38.688342+0000","last_unstale":"2024-05-08T23:30:40.243979+0000","last_undegraded":"2024-05-08T23:30:40.243979+0000","last_fullsized":"2024-05-08T23:30:40.243979+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:13.779218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.681408+0000","last_change":"2024-05-08T23:30:38.314058+0000","last_active":"2024-05-08T23:30:38.681408+0000","last_peered":"2024-05-08T23:30:38.681408+0000","last_clean":"2024-05-08T23:30:38.681408+0000","last_became_active":"2024-05-08T23:30:38.313813+0000","last_became_peered":"2024-05-08T23:30:38.313813+0000","last_unstale":"2024-05-08T23:30:38.681408+0000","last_undegraded":"2024-05-08T23:30:38.681408+0000","last_fullsized":"2024-05-08T23:30:38.681408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:14.380730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711783+0000","last_change":"2024-05-08T23:30:38.707800+0000","last_active":"2024-05-08T23:30:38.711783+0000","last_peered":"2024-05-08T23:30:38.711783+0000","last_clean":"2024-05-08T23:30:38.711783+0000","last_became_active":"2024-05-08T23:30:38.706575+0000","last_became_peered":"2024-05-08T23:30:38.706575+0000","last_unstale":"2024-05-08T23:30:38.711783+0000","last_undegraded":"2024-05-08T23:30:38.711783+0000","last_fullsized":"2024-05-08T23:30:38.711783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:16.601918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.310713+0000","last_change":"2024-05-08T23:30:27.226484+0000","last_active":"2024-05-08T23:30:38.310713+0000","last_peered":"2024-05-08T23:30:38.310713+0000","last_clean":"2024-05-08T23:30:38.310713+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:30:38.310713+0000","last_undegraded":"2024-05-08T23:30:38.310713+0000","last_fullsized":"2024-05-08T23:30:38.310713+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_clean_scrub_stamp":"2024-05-08T23:30:26.204118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:14.267287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243573+0000","last_change":"2024-05-08T23:30:38.317146+0000","last_active":"2024-05-08T23:30:40.243573+0000","last_peered":"2024-05-08T23:30:40.243573+0000","last_clean":"2024-05-08T23:30:40.243573+0000","last_became_active":"2024-05-08T23:30:38.316813+0000","last_became_peered":"2024-05-08T23:30:38.316813+0000","last_unstale":"2024-05-08T23:30:40.243573+0000","last_undegraded":"2024-05-08T23:30:40.243573+0000","last_fullsized":"2024-05-08T23:30:40.243573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:03.938472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675574+0000","last_change":"2024-05-08T23:30:38.690153+0000","last_active":"2024-05-08T23:30:39.675574+0000","last_peered":"2024-05-08T23:30:39.675574+0000","last_clean":"2024-05-08T23:30:39.675574+0000","last_became_active":"2024-05-08T23:30:38.689929+0000","last_became_peered":"2024-05-08T23:30:38.689929+0000","last_unstale":"2024-05-08T23:30:39.675574+0000","last_undegraded":"2024-05-08T23:30:39.675574+0000","last_fullsized":"2024-05-08T23:30:39.675574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:07.682170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.695745+0000","last_change":"2024-05-08T23:30:38.319918+0000","last_active":"2024-05-08T23:30:38.695745+0000","last_peered":"2024-05-08T23:30:38.695745+0000","last_clean":"2024-05-08T23:30:38.695745+0000","last_became_active":"2024-05-08T23:30:38.319606+0000","last_became_peered":"2024-05-08T23:30:38.319606+0000","last_unstale":"2024-05-08T23:30:38.695745+0000","last_undegraded":"2024-05-08T23:30:38.695745+0000","last_fullsized":"2024-05-08T23:30:38.695745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:24.905425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.685916+0000","last_change":"2024-05-08T23:30:38.681448+0000","last_active":"2024-05-08T23:30:38.685916+0000","last_peered":"2024-05-08T23:30:38.685916+0000","last_clean":"2024-05-08T23:30:38.685916+0000","last_became_active":"2024-05-08T23:30:38.681002+0000","last_became_peered":"2024-05-08T23:30:38.681002+0000","last_unstale":"2024-05-08T23:30:38.685916+0000","last_undegraded":"2024-05-08T23:30:38.685916+0000","last_fullsized":"2024-05-08T23:30:38.685916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:40.104243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.681396+0000","last_change":"2024-05-08T23:30:38.312104+0000","last_active":"2024-05-08T23:30:38.681396+0000","last_peered":"2024-05-08T23:30:38.681396+0000","last_clean":"2024-05-08T23:30:38.681396+0000","last_became_active":"2024-05-08T23:30:38.311833+0000","last_became_peered":"2024-05-08T23:30:38.311833+0000","last_unstale":"2024-05-08T23:30:38.681396+0000","last_undegraded":"2024-05-08T23:30:38.681396+0000","last_fullsized":"2024-05-08T23:30:38.681396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:31.924157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.689693+0000","last_change":"2024-05-08T23:30:38.686436+0000","last_active":"2024-05-08T23:30:38.689693+0000","last_peered":"2024-05-08T23:30:38.689693+0000","last_clean":"2024-05-08T23:30:38.689693+0000","last_became_active":"2024-05-08T23:30:38.686176+0000","last_became_peered":"2024-05-08T23:30:38.686176+0000","last_unstale":"2024-05-08T23:30:38.689693+0000","last_undegraded":"2024-05-08T23:30:38.689693+0000","last_fullsized":"2024-05-08T23:30:38.689693+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:41.133153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244021+0000","last_change":"2024-05-08T23:30:38.317558+0000","last_active":"2024-05-08T23:30:40.244021+0000","last_peered":"2024-05-08T23:30:40.244021+0000","last_clean":"2024-05-08T23:30:40.244021+0000","last_became_active":"2024-05-08T23:30:38.317296+0000","last_became_peered":"2024-05-08T23:30:38.317296+0000","last_unstale":"2024-05-08T23:30:40.244021+0000","last_undegraded":"2024-05-08T23:30:40.244021+0000","last_fullsized":"2024-05-08T23:30:40.244021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959765+0000","last_change":"2024-05-08T23:30:38.682924+0000","last_active":"2024-05-08T23:30:39.959765+0000","last_peered":"2024-05-08T23:30:39.959765+0000","last_clean":"2024-05-08T23:30:39.959765+0000","last_became_active":"2024-05-08T23:30:38.682112+0000","last_became_peered":"2024-05-08T23:30:38.682112+0000","last_unstale":"2024-05-08T23:30:39.959765+0000","last_undegraded":"2024-05-08T23:30:39.959765+0000","last_fullsized":"2024-05-08T23:30:39.959765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:04.948224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.676243+0000","last_change":"2024-05-08T23:30:38.317398+0000","last_active":"2024-05-08T23:30:38.676243+0000","last_peered":"2024-05-08T23:30:38.676243+0000","last_clean":"2024-05-08T23:30:38.676243+0000","last_became_active":"2024-05-08T23:30:38.317167+0000","last_became_peered":"2024-05-08T23:30:38.317167+0000","last_unstale":"2024-05-08T23:30:38.676243+0000","last_undegraded":"2024-05-08T23:30:38.676243+0000","last_fullsized":"2024-05-08T23:30:38.676243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244386+0000","last_change":"2024-05-08T23:30:38.693098+0000","last_active":"2024-05-08T23:30:40.244386+0000","last_peered":"2024-05-08T23:30:40.244386+0000","last_clean":"2024-05-08T23:30:40.244386+0000","last_became_active":"2024-05-08T23:30:38.692852+0000","last_became_peered":"2024-05-08T23:30:38.692852+0000","last_unstale":"2024-05-08T23:30:40.244386+0000","last_undegraded":"2024-05-08T23:30:40.244386+0000","last_fullsized":"2024-05-08T23:30:40.244386+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:02.678862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.701929+0000","last_change":"2024-05-08T23:30:38.320883+0000","last_active":"2024-05-08T23:30:38.701929+0000","last_peered":"2024-05-08T23:30:38.701929+0000","last_clean":"2024-05-08T23:30:38.701929+0000","last_became_active":"2024-05-08T23:30:38.320610+0000","last_became_peered":"2024-05-08T23:30:38.320610+0000","last_unstale":"2024-05-08T23:30:38.701929+0000","last_undegraded":"2024-05-08T23:30:38.701929+0000","last_fullsized":"2024-05-08T23:30:38.701929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:02.623735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.689580+0000","last_change":"2024-05-08T23:30:38.687478+0000","last_active":"2024-05-08T23:30:38.689580+0000","last_peered":"2024-05-08T23:30:38.689580+0000","last_clean":"2024-05-08T23:30:38.689580+0000","last_became_active":"2024-05-08T23:30:38.686229+0000","last_became_peered":"2024-05-08T23:30:38.686229+0000","last_unstale":"2024-05-08T23:30:38.689580+0000","last_undegraded":"2024-05-08T23:30:38.689580+0000","last_fullsized":"2024-05-08T23:30:38.689580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:37.994049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.677846+0000","last_change":"2024-05-08T23:30:38.318046+0000","last_active":"2024-05-08T23:30:38.677846+0000","last_peered":"2024-05-08T23:30:38.677846+0000","last_clean":"2024-05-08T23:30:38.677846+0000","last_became_active":"2024-05-08T23:30:38.317757+0000","last_became_peered":"2024-05-08T23:30:38.317757+0000","last_unstale":"2024-05-08T23:30:38.677846+0000","last_undegraded":"2024-05-08T23:30:38.677846+0000","last_fullsized":"2024-05-08T23:30:38.677846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:00.309450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676059+0000","last_change":"2024-05-08T23:30:38.706013+0000","last_active":"2024-05-08T23:30:39.676059+0000","last_peered":"2024-05-08T23:30:39.676059+0000","last_clean":"2024-05-08T23:30:39.676059+0000","last_became_active":"2024-05-08T23:30:38.705709+0000","last_became_peered":"2024-05-08T23:30:38.705709+0000","last_unstale":"2024-05-08T23:30:39.676059+0000","last_undegraded":"2024-05-08T23:30:39.676059+0000","last_fullsized":"2024-05-08T23:30:39.676059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:01.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.701098+0000","last_change":"2024-05-08T23:30:38.317850+0000","last_active":"2024-05-08T23:30:38.701098+0000","last_peered":"2024-05-08T23:30:38.701098+0000","last_clean":"2024-05-08T23:30:38.701098+0000","last_became_active":"2024-05-08T23:30:38.317598+0000","last_became_peered":"2024-05-08T23:30:38.317598+0000","last_unstale":"2024-05-08T23:30:38.701098+0000","last_undegraded":"2024-05-08T23:30:38.701098+0000","last_fullsized":"2024-05-08T23:30:38.701098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:39.460999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.681899+0000","last_change":"2024-05-08T23:30:38.679213+0000","last_active":"2024-05-08T23:30:38.681899+0000","last_peered":"2024-05-08T23:30:38.681899+0000","last_clean":"2024-05-08T23:30:38.681899+0000","last_became_active":"2024-05-08T23:30:38.678935+0000","last_became_peered":"2024-05-08T23:30:38.678935+0000","last_unstale":"2024-05-08T23:30:38.681899+0000","last_undegraded":"2024-05-08T23:30:38.681899+0000","last_fullsized":"2024-05-08T23:30:38.681899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:39.006348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.695339+0000","last_change":"2024-05-08T23:30:38.320666+0000","last_active":"2024-05-08T23:30:38.695339+0000","last_peered":"2024-05-08T23:30:38.695339+0000","last_clean":"2024-05-08T23:30:38.695339+0000","last_became_active":"2024-05-08T23:30:38.320387+0000","last_became_peered":"2024-05-08T23:30:38.320387+0000","last_unstale":"2024-05-08T23:30:38.695339+0000","last_undegraded":"2024-05-08T23:30:38.695339+0000","last_fullsized":"2024-05-08T23:30:38.695339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:35.811032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959476+0000","last_change":"2024-05-08T23:30:38.692172+0000","last_active":"2024-05-08T23:30:39.959476+0000","last_peered":"2024-05-08T23:30:39.959476+0000","last_clean":"2024-05-08T23:30:39.959476+0000","last_became_active":"2024-05-08T23:30:38.688675+0000","last_became_peered":"2024-05-08T23:30:38.688675+0000","last_unstale":"2024-05-08T23:30:39.959476+0000","last_undegraded":"2024-05-08T23:30:39.959476+0000","last_fullsized":"2024-05-08T23:30:39.959476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:40.470751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.676354+0000","last_change":"2024-05-08T23:30:38.319809+0000","last_active":"2024-05-08T23:30:38.676354+0000","last_peered":"2024-05-08T23:30:38.676354+0000","last_clean":"2024-05-08T23:30:38.676354+0000","last_became_active":"2024-05-08T23:30:38.319500+0000","last_became_peered":"2024-05-08T23:30:38.319500+0000","last_unstale":"2024-05-08T23:30:38.676354+0000","last_undegraded":"2024-05-08T23:30:38.676354+0000","last_fullsized":"2024-05-08T23:30:38.676354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:06.433390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675251+0000","last_change":"2024-05-08T23:30:38.694080+0000","last_active":"2024-05-08T23:30:39.675251+0000","last_peered":"2024-05-08T23:30:39.675251+0000","last_clean":"2024-05-08T23:30:39.675251+0000","last_became_active":"2024-05-08T23:30:38.693469+0000","last_became_peered":"2024-05-08T23:30:38.693469+0000","last_unstale":"2024-05-08T23:30:39.675251+0000","last_undegraded":"2024-05-08T23:30:39.675251+0000","last_fullsized":"2024-05-08T23:30:39.675251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:07.222797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.676422+0000","last_change":"2024-05-08T23:30:38.316969+0000","last_active":"2024-05-08T23:30:38.676422+0000","last_peered":"2024-05-08T23:30:38.676422+0000","last_clean":"2024-05-08T23:30:38.676422+0000","last_became_active":"2024-05-08T23:30:38.316648+0000","last_became_peered":"2024-05-08T23:30:38.316648+0000","last_unstale":"2024-05-08T23:30:38.676422+0000","last_undegraded":"2024-05-08T23:30:38.676422+0000","last_fullsized":"2024-05-08T23:30:38.676422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:03.938472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675646+0000","last_change":"2024-05-08T23:30:38.696097+0000","last_active":"2024-05-08T23:30:39.675646+0000","last_peered":"2024-05-08T23:30:39.675646+0000","last_clean":"2024-05-08T23:30:39.675646+0000","last_became_active":"2024-05-08T23:30:38.695832+0000","last_became_peered":"2024-05-08T23:30:38.695832+0000","last_unstale":"2024-05-08T23:30:39.675646+0000","last_undegraded":"2024-05-08T23:30:39.675646+0000","last_fullsized":"2024-05-08T23:30:39.675646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:12.789913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711847+0000","last_change":"2024-05-08T23:30:38.320025+0000","last_active":"2024-05-08T23:30:38.711847+0000","last_peered":"2024-05-08T23:30:38.711847+0000","last_clean":"2024-05-08T23:30:38.711847+0000","last_became_active":"2024-05-08T23:30:38.319609+0000","last_became_peered":"2024-05-08T23:30:38.319609+0000","last_unstale":"2024-05-08T23:30:38.711847+0000","last_undegraded":"2024-05-08T23:30:38.711847+0000","last_fullsized":"2024-05-08T23:30:38.711847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:42.475179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.700934+0000","last_change":"2024-05-08T23:30:38.699456+0000","last_active":"2024-05-08T23:30:38.700934+0000","last_peered":"2024-05-08T23:30:38.700934+0000","last_clean":"2024-05-08T23:30:38.700934+0000","last_became_active":"2024-05-08T23:30:38.699013+0000","last_became_peered":"2024-05-08T23:30:38.699013+0000","last_unstale":"2024-05-08T23:30:38.700934+0000","last_undegraded":"2024-05-08T23:30:38.700934+0000","last_fullsized":"2024-05-08T23:30:38.700934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:37.994049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244260+0000","last_change":"2024-05-08T23:30:38.695946+0000","last_active":"2024-05-08T23:30:40.244260+0000","last_peered":"2024-05-08T23:30:40.244260+0000","last_clean":"2024-05-08T23:30:40.244260+0000","last_became_active":"2024-05-08T23:30:38.695704+0000","last_became_peered":"2024-05-08T23:30:38.695704+0000","last_unstale":"2024-05-08T23:30:40.244260+0000","last_undegraded":"2024-05-08T23:30:40.244260+0000","last_fullsized":"2024-05-08T23:30:40.244260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.710659+0000","last_change":"2024-05-08T23:30:38.319876+0000","last_active":"2024-05-08T23:30:38.710659+0000","last_peered":"2024-05-08T23:30:38.710659+0000","last_clean":"2024-05-08T23:30:38.710659+0000","last_became_active":"2024-05-08T23:30:38.319589+0000","last_became_peered":"2024-05-08T23:30:38.319589+0000","last_unstale":"2024-05-08T23:30:38.710659+0000","last_undegraded":"2024-05-08T23:30:38.710659+0000","last_fullsized":"2024-05-08T23:30:38.710659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:12.479867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243895+0000","last_change":"2024-05-08T23:30:38.697783+0000","last_active":"2024-05-08T23:30:40.243895+0000","last_peered":"2024-05-08T23:30:40.243895+0000","last_clean":"2024-05-08T23:30:40.243895+0000","last_became_active":"2024-05-08T23:30:38.697295+0000","last_became_peered":"2024-05-08T23:30:38.697295+0000","last_unstale":"2024-05-08T23:30:40.243895+0000","last_undegraded":"2024-05-08T23:30:40.243895+0000","last_fullsized":"2024-05-08T23:30:40.243895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:12.789913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.711551+0000","last_change":"2024-05-08T23:30:38.320543+0000","last_active":"2024-05-08T23:30:38.711551+0000","last_peered":"2024-05-08T23:30:38.711551+0000","last_clean":"2024-05-08T23:30:38.711551+0000","last_became_active":"2024-05-08T23:30:38.320231+0000","last_became_peered":"2024-05-08T23:30:38.320231+0000","last_unstale":"2024-05-08T23:30:38.711551+0000","last_undegraded":"2024-05-08T23:30:38.711551+0000","last_fullsized":"2024-05-08T23:30:38.711551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:08:09.987948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.695345+0000","last_change":"2024-05-08T23:30:38.691260+0000","last_active":"2024-05-08T23:30:38.695345+0000","last_peered":"2024-05-08T23:30:38.695345+0000","last_clean":"2024-05-08T23:30:38.695345+0000","last_became_active":"2024-05-08T23:30:38.690856+0000","last_became_peered":"2024-05-08T23:30:38.690856+0000","last_unstale":"2024-05-08T23:30:38.695345+0000","last_undegraded":"2024-05-08T23:30:38.695345+0000","last_fullsized":"2024-05-08T23:30:38.695345+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:58.926788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676224+0000","last_change":"2024-05-08T23:30:38.311855+0000","last_active":"2024-05-08T23:30:39.676224+0000","last_peered":"2024-05-08T23:30:39.676224+0000","last_clean":"2024-05-08T23:30:39.676224+0000","last_became_active":"2024-05-08T23:30:38.311542+0000","last_became_peered":"2024-05-08T23:30:38.311542+0000","last_unstale":"2024-05-08T23:30:39.676224+0000","last_undegraded":"2024-05-08T23:30:39.676224+0000","last_fullsized":"2024-05-08T23:30:39.676224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:30.035897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676005+0000","last_change":"2024-05-08T23:30:38.688998+0000","last_active":"2024-05-08T23:30:39.676005+0000","last_peered":"2024-05-08T23:30:39.676005+0000","last_clean":"2024-05-08T23:30:39.676005+0000","last_became_active":"2024-05-08T23:30:38.688723+0000","last_became_peered":"2024-05-08T23:30:38.688723+0000","last_unstale":"2024-05-08T23:30:39.676005+0000","last_undegraded":"2024-05-08T23:30:39.676005+0000","last_fullsized":"2024-05-08T23:30:39.676005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:41.133153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.700036+0000","last_change":"2024-05-08T23:30:38.315584+0000","last_active":"2024-05-08T23:30:38.700036+0000","last_peered":"2024-05-08T23:30:38.700036+0000","last_clean":"2024-05-08T23:30:38.700036+0000","last_became_active":"2024-05-08T23:30:38.315234+0000","last_became_peered":"2024-05-08T23:30:38.315234+0000","last_unstale":"2024-05-08T23:30:38.700036+0000","last_undegraded":"2024-05-08T23:30:38.700036+0000","last_fullsized":"2024-05-08T23:30:38.700036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:02.623735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":72078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":70604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4830000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":70640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":78378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]}]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]}]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":654107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":661520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:30:43.049 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T23:30:43.050 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T23:30:43.050 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-08T23:30:43.050 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-08T23:30:43.050 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-08T23:30:43.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.157+0000 7fa17d0da700 1 Processor -- start 2024-05-08T23:30:43.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.157+0000 7fa17d0da700 1 -- start start 2024-05-08T23:30:43.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.157+0000 7fa17d0da700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa1780b3c20 0x7fa1781624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.157+0000 7fa17d0da700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa178070110 0x7fa1780705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.157+0000 7fa17d0da700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa178158960 con 0x7fa178071e70 2024-05-08T23:30:43.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.157+0000 7fa17d0da700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1780b1f60 con 0x7fa1780b3c20 2024-05-08T23:30:43.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.157+0000 7fa17d0da700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1780ae990 con 0x7fa178070110 2024-05-08T23:30:43.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.157+0000 7fa177fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa1780b3c20 0x7fa1781624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.157+0000 7fa177fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa1780b3c20 0x7fa1781624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32956/0 (socket says 172.21.15.81:32956) 2024-05-08T23:30:43.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.157+0000 7fa177fff700 1 -- 172.21.15.81:0/1563777989 learned_addr learned my addr 172.21.15.81:0/1563777989 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:43.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.157+0000 7fa17c8d9700 1 --2- 172.21.15.81:0/1563777989 >> v2:172.21.15.117:3300/0 conn(0x7fa178070110 0x7fa1780705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa177fff700 1 -- 172.21.15.81:0/1563777989 >> v2:172.21.15.117:3300/0 conn(0x7fa178070110 msgr2=0x7fa1780705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:43.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa177fff700 1 --2- 172.21.15.81:0/1563777989 >> v2:172.21.15.117:3300/0 conn(0x7fa178070110 0x7fa1780705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa176ffd700 1 -- 172.21.15.81:0/1563777989 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030001106 0 0) 0x7fa178158960 con 0x7fa178071e70 2024-05-08T23:30:43.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa177fff700 1 -- 172.21.15.81:0/1563777989 >> v1:172.21.15.117:6789/0 conn(0x7fa178071e70 legacy=0x7fa1780b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa177fff700 1 -- 172.21.15.81:0/1563777989 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1780b2170 con 0x7fa1780b3c20 2024-05-08T23:30:43.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa177fff700 1 --2- 172.21.15.81:0/1563777989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1780b3c20 0x7fa1781624b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa1780b51f0 tx=0x7fa16c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ff6c03481c0c29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:43.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa176ffd700 1 -- 172.21.15.81:0/1563777989 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c011830 con 0x7fa1780b3c20 2024-05-08T23:30:43.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa176ffd700 1 -- 172.21.15.81:0/1563777989 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16c011990 con 0x7fa1780b3c20 2024-05-08T23:30:43.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa176ffd700 1 -- 172.21.15.81:0/1563777989 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c011d20 con 0x7fa1780b3c20 2024-05-08T23:30:43.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa17d0da700 1 -- 172.21.15.81:0/1563777989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1780b3c20 msgr2=0x7fa1781624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa17d0da700 1 --2- 172.21.15.81:0/1563777989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1780b3c20 0x7fa1781624b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa1780b51f0 tx=0x7fa16c009bf0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa17d0da700 1 -- 172.21.15.81:0/1563777989 shutdown_connections 2024-05-08T23:30:43.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa17d0da700 1 --2- 172.21.15.81:0/1563777989 >> v2:172.21.15.117:3300/0 conn(0x7fa178070110 0x7fa1780705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa17d0da700 1 --2- 172.21.15.81:0/1563777989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1780b3c20 0x7fa1781624b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa17d0da700 1 -- 172.21.15.81:0/1563777989 >> 172.21.15.81:0/1563777989 conn(0x7fa17806c330 msgr2=0x7fa17806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:43.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa17d0da700 1 -- 172.21.15.81:0/1563777989 shutdown_connections 2024-05-08T23:30:43.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa17d0da700 1 -- 172.21.15.81:0/1563777989 wait complete. 2024-05-08T23:30:43.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa17d0da700 1 Processor -- start 2024-05-08T23:30:43.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.161+0000 7fa17d0da700 1 -- start start 2024-05-08T23:30:43.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa17d0da700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa178070110 0x7fa17814e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa17d0da700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa178071e70 0x7fa178164c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa17d0da700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1780b3c20 0x7fa1780ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa17d0da700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1781638d0 con 0x7fa178070110 2024-05-08T23:30:43.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa17d0da700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa178165fa0 con 0x7fa1780b3c20 2024-05-08T23:30:43.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa17d0da700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17814e650 con 0x7fa178071e70 2024-05-08T23:30:43.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa177fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa178070110 0x7fa17814e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa17c8d9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1780b3c20 0x7fa1780ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa177fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa178070110 0x7fa17814e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32962/0 (socket says 172.21.15.81:32962) 2024-05-08T23:30:43.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa177fff700 1 -- 172.21.15.81:0/1473633766 learned_addr learned my addr 172.21.15.81:0/1473633766 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:43.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa1777fe700 1 --2- 172.21.15.81:0/1473633766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa178071e70 0x7fa178164c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa177fff700 1 -- 172.21.15.81:0/1473633766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa178071e70 msgr2=0x7fa178164c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa177fff700 1 --2- 172.21.15.81:0/1473633766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa178071e70 0x7fa178164c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa177fff700 1 -- 172.21.15.81:0/1473633766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1780b3c20 msgr2=0x7fa1780ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa177fff700 1 --2- 172.21.15.81:0/1473633766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1780b3c20 0x7fa1780ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa177fff700 1 -- 172.21.15.81:0/1473633766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c009570 con 0x7fa178070110 2024-05-08T23:30:43.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa17c8d9700 1 --2- 172.21.15.81:0/1473633766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1780b3c20 0x7fa1780ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:43.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa177fff700 1 --2- 172.21.15.81:0/1473633766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa178070110 0x7fa17814e110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa16c000c00 tx=0x7fa16c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:43.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa174ff9700 1 -- 172.21.15.81:0/1473633766 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c004510 con 0x7fa178070110 2024-05-08T23:30:43.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa174ff9700 1 -- 172.21.15.81:0/1473633766 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16c01e070 con 0x7fa178070110 2024-05-08T23:30:43.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa17d0da700 1 -- 172.21.15.81:0/1473633766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa178165700 con 0x7fa178070110 2024-05-08T23:30:43.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa174ff9700 1 -- 172.21.15.81:0/1473633766 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c003900 con 0x7fa178070110 2024-05-08T23:30:43.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa17d0da700 1 -- 172.21.15.81:0/1473633766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa178165cc0 con 0x7fa178070110 2024-05-08T23:30:43.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.165+0000 7fa17d0da700 1 -- 172.21.15.81:0/1473633766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1780684b0 con 0x7fa178070110 2024-05-08T23:30:43.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.169+0000 7fa174ff9700 1 -- 172.21.15.81:0/1473633766 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa16c003a60 con 0x7fa178070110 2024-05-08T23:30:43.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.169+0000 7fa174ff9700 1 --2- 172.21.15.81:0/1473633766 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa164066730 0x7fa164068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.169+0000 7fa174ff9700 1 -- 172.21.15.81:0/1473633766 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa16c084c80 con 0x7fa178070110 2024-05-08T23:30:43.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.169+0000 7fa1777fe700 1 --2- 172.21.15.81:0/1473633766 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa164066730 0x7fa164068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.169+0000 7fa1777fe700 1 --2- 172.21.15.81:0/1473633766 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa164066730 0x7fa164068bf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa1781517a0 tx=0x7fa168009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:43.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.173+0000 7fa174ff9700 1 -- 172.21.15.81:0/1473633766 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa16c052080 con 0x7fa178070110 2024-05-08T23:30:43.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.385+0000 7fa17d0da700 1 -- 172.21.15.81:0/1473633766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa178071c50 con 0x7fa178070110 2024-05-08T23:30:43.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.393+0000 7fa174ff9700 1 -- 172.21.15.81:0/1473633766 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164885 (secure 0 0 0) 0x7fa16c015090 con 0x7fa178070110 2024-05-08T23:30:43.398 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:43.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.393+0000 7fa17d0da700 1 -- 172.21.15.81:0/1473633766 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa164066730 msgr2=0x7fa164068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.393+0000 7fa17d0da700 1 --2- 172.21.15.81:0/1473633766 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa164066730 0x7fa164068bf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa1781517a0 tx=0x7fa168009040 comp rx=0 tx=0).stop 2024-05-08T23:30:43.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.393+0000 7fa17d0da700 1 -- 172.21.15.81:0/1473633766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa178070110 msgr2=0x7fa17814e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.393+0000 7fa17d0da700 1 --2- 172.21.15.81:0/1473633766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa178070110 0x7fa17814e110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa16c000c00 tx=0x7fa16c004770 comp rx=0 tx=0).stop 2024-05-08T23:30:43.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.393+0000 7fa17d0da700 1 -- 172.21.15.81:0/1473633766 shutdown_connections 2024-05-08T23:30:43.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.393+0000 7fa17d0da700 1 --2- 172.21.15.81:0/1473633766 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa164066730 0x7fa164068bf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.393+0000 7fa17d0da700 1 --2- 172.21.15.81:0/1473633766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1780b3c20 0x7fa1780ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.393+0000 7fa17d0da700 1 --2- 172.21.15.81:0/1473633766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa178071e70 0x7fa178164c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.393+0000 7fa17d0da700 1 --2- 172.21.15.81:0/1473633766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa178070110 0x7fa17814e110 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.393+0000 7fa17d0da700 1 -- 172.21.15.81:0/1473633766 >> 172.21.15.81:0/1473633766 conn(0x7fa17806c330 msgr2=0x7fa1780ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:43.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.397+0000 7fa17d0da700 1 -- 172.21.15.81:0/1473633766 shutdown_connections 2024-05-08T23:30:43.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.397+0000 7fa17d0da700 1 -- 172.21.15.81:0/1473633766 wait complete. 2024-05-08T23:30:43.414 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":25965},{"type":"v1","addr":"172.21.15.81:6833","nonce":25965}]},"active_addr":"172.21.15.81:6833/25965","active_change":"2024-05-08T23:30:19.310402+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4099,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":2898491320}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":3028795337}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":1549664863}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":1365876826}]}]} 2024-05-08T23:30:43.417 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-08T23:30:43.417 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T23:30:43.417 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:30:43.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9addb85700 1 Processor -- start 2024-05-08T23:30:43.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9addb85700 1 -- start start 2024-05-08T23:30:43.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9addb85700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9ad81596a0 0x7f9ad80b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9addb85700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9ad815a190 0x7f9ad80709a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9addb85700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad80b3590 con 0x7f9ad80b4e50 2024-05-08T23:30:43.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9addb85700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad80b24f0 con 0x7f9ad815a190 2024-05-08T23:30:43.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9addb85700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad80b27d0 con 0x7f9ad81596a0 2024-05-08T23:30:43.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9add384700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9ad815a190 0x7f9ad80709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9add384700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9ad815a190 0x7f9ad80709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32968/0 (socket says 172.21.15.81:32968) 2024-05-08T23:30:43.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9add384700 1 -- 172.21.15.81:0/3733003294 learned_addr learned my addr 172.21.15.81:0/3733003294 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:43.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9adcb83700 1 --2- 172.21.15.81:0/3733003294 >> v2:172.21.15.117:3300/0 conn(0x7f9ad81596a0 0x7f9ad80b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9add384700 1 -- 172.21.15.81:0/3733003294 >> v2:172.21.15.117:3300/0 conn(0x7f9ad81596a0 msgr2=0x7f9ad80b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:43.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9add384700 1 --2- 172.21.15.81:0/3733003294 >> v2:172.21.15.117:3300/0 conn(0x7f9ad81596a0 0x7f9ad80b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9add384700 1 -- 172.21.15.81:0/3733003294 >> v1:172.21.15.117:6789/0 conn(0x7f9ad80b4e50 legacy=0x7f9ad8159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:43.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9add384700 1 -- 172.21.15.81:0/3733003294 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad80b2ab0 con 0x7f9ad815a190 2024-05-08T23:30:43.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9adcb83700 1 --2- 172.21.15.81:0/3733003294 >> v2:172.21.15.117:3300/0 conn(0x7f9ad81596a0 0x7f9ad80b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:43.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9add384700 1 --2- 172.21.15.81:0/3733003294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ad815a190 0x7f9ad80709a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9ad804f790 tx=0x7f9ad0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7147b93a7577a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:43.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9ad77fe700 1 -- 172.21.15.81:0/3733003294 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0004230 con 0x7f9ad815a190 2024-05-08T23:30:43.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9ad77fe700 1 -- 172.21.15.81:0/3733003294 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad0004390 con 0x7f9ad815a190 2024-05-08T23:30:43.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.529+0000 7f9ad77fe700 1 -- 172.21.15.81:0/3733003294 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad00036c0 con 0x7f9ad815a190 2024-05-08T23:30:43.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 -- 172.21.15.81:0/3733003294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ad815a190 msgr2=0x7f9ad80709a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 --2- 172.21.15.81:0/3733003294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ad815a190 0x7f9ad80709a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9ad804f790 tx=0x7f9ad0009bf0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 -- 172.21.15.81:0/3733003294 shutdown_connections 2024-05-08T23:30:43.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 --2- 172.21.15.81:0/3733003294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ad815a190 0x7f9ad80709a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 --2- 172.21.15.81:0/3733003294 >> v2:172.21.15.117:3300/0 conn(0x7f9ad81596a0 0x7f9ad80b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 -- 172.21.15.81:0/3733003294 >> 172.21.15.81:0/3733003294 conn(0x7f9ad806c330 msgr2=0x7f9ad806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:43.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 -- 172.21.15.81:0/3733003294 shutdown_connections 2024-05-08T23:30:43.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 -- 172.21.15.81:0/3733003294 wait complete. 2024-05-08T23:30:43.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 Processor -- start 2024-05-08T23:30:43.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 -- start start 2024-05-08T23:30:43.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ad80b4e50 0x7f9ad80b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ad81596a0 0x7f9ad8148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ad815a190 0x7f9ad813fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad80afcd0 con 0x7f9ad815a190 2024-05-08T23:30:43.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad81403c0 con 0x7f9ad80b4e50 2024-05-08T23:30:43.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9addb85700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad8150680 con 0x7f9ad81596a0 2024-05-08T23:30:43.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9add384700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ad815a190 0x7f9ad813fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9add384700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ad815a190 0x7f9ad813fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32970/0 (socket says 172.21.15.81:32970) 2024-05-08T23:30:43.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9add384700 1 -- 172.21.15.81:0/3020074552 learned_addr learned my addr 172.21.15.81:0/3020074552 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:43.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9adcb83700 1 --2- 172.21.15.81:0/3020074552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ad80b4e50 0x7f9ad80b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.533+0000 7f9ad7fff700 1 --2- 172.21.15.81:0/3020074552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ad81596a0 0x7f9ad8148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9add384700 1 -- 172.21.15.81:0/3020074552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ad81596a0 msgr2=0x7f9ad8148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9add384700 1 --2- 172.21.15.81:0/3020074552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ad81596a0 0x7f9ad8148f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9add384700 1 -- 172.21.15.81:0/3020074552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ad80b4e50 msgr2=0x7f9ad80b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9add384700 1 --2- 172.21.15.81:0/3020074552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ad80b4e50 0x7f9ad80b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9add384700 1 -- 172.21.15.81:0/3020074552 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad0009570 con 0x7f9ad815a190 2024-05-08T23:30:43.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9add384700 1 --2- 172.21.15.81:0/3020074552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ad815a190 0x7f9ad813fe80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80725c0 tx=0x7f9ad0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:43.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9ad57fa700 1 -- 172.21.15.81:0/3020074552 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0003cc0 con 0x7f9ad815a190 2024-05-08T23:30:43.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9ad57fa700 1 -- 172.21.15.81:0/3020074552 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad001f070 con 0x7f9ad815a190 2024-05-08T23:30:43.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9addb85700 1 -- 172.21.15.81:0/3020074552 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad81503a0 con 0x7f9ad815a190 2024-05-08T23:30:43.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9ad57fa700 1 -- 172.21.15.81:0/3020074552 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000b880 con 0x7f9ad815a190 2024-05-08T23:30:43.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9addb85700 1 -- 172.21.15.81:0/3020074552 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad80af960 con 0x7f9ad815a190 2024-05-08T23:30:43.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9addb85700 1 -- 172.21.15.81:0/3020074552 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad80afe40 con 0x7f9ad815a190 2024-05-08T23:30:43.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9ad57fa700 1 -- 172.21.15.81:0/3020074552 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9ad0021030 con 0x7f9ad815a190 2024-05-08T23:30:43.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9ad57fa700 1 --2- 172.21.15.81:0/3020074552 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9abc066730 0x7f9abc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9ad57fa700 1 -- 172.21.15.81:0/3020074552 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ad0085710 con 0x7f9ad815a190 2024-05-08T23:30:43.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.537+0000 7f9adcb83700 1 --2- 172.21.15.81:0/3020074552 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9abc066730 0x7f9abc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.541+0000 7f9adcb83700 1 --2- 172.21.15.81:0/3020074552 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9abc066730 0x7f9abc068bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9ad81f3d50 tx=0x7f9ac0007810 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:43.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.545+0000 7f9ad57fa700 1 -- 172.21.15.81:0/3020074552 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9ad0052b40 con 0x7f9ad815a190 2024-05-08T23:30:43.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.717+0000 7f9addb85700 1 -- 172.21.15.81:0/3020074552 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ad80684b0 con 0x7f9ad815a190 2024-05-08T23:30:43.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.721+0000 7f9ad57fa700 1 -- 172.21.15.81:0/3020074552 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f9ad0056160 con 0x7f9ad815a190 2024-05-08T23:30:43.725 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:43.725 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:30:43.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.721+0000 7f9addb85700 1 -- 172.21.15.81:0/3020074552 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9abc066730 msgr2=0x7f9abc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.721+0000 7f9addb85700 1 --2- 172.21.15.81:0/3020074552 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9abc066730 0x7f9abc068bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9ad81f3d50 tx=0x7f9ac0007810 comp rx=0 tx=0).stop 2024-05-08T23:30:43.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.721+0000 7f9addb85700 1 -- 172.21.15.81:0/3020074552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ad815a190 msgr2=0x7f9ad813fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.721+0000 7f9addb85700 1 --2- 172.21.15.81:0/3020074552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ad815a190 0x7f9ad813fe80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80725c0 tx=0x7f9ad0004770 comp rx=0 tx=0).stop 2024-05-08T23:30:43.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.721+0000 7f9addb85700 1 -- 172.21.15.81:0/3020074552 shutdown_connections 2024-05-08T23:30:43.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.721+0000 7f9addb85700 1 --2- 172.21.15.81:0/3020074552 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9abc066730 0x7f9abc068bf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.721+0000 7f9addb85700 1 --2- 172.21.15.81:0/3020074552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ad815a190 0x7f9ad813fe80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.721+0000 7f9addb85700 1 --2- 172.21.15.81:0/3020074552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ad81596a0 0x7f9ad8148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.721+0000 7f9addb85700 1 --2- 172.21.15.81:0/3020074552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ad80b4e50 0x7f9ad80b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.721+0000 7f9addb85700 1 -- 172.21.15.81:0/3020074552 >> 172.21.15.81:0/3020074552 conn(0x7f9ad806c330 msgr2=0x7f9ad80ccd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:43.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.721+0000 7f9addb85700 1 -- 172.21.15.81:0/3020074552 shutdown_connections 2024-05-08T23:30:43.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.721+0000 7f9addb85700 1 -- 172.21.15.81:0/3020074552 wait complete. 2024-05-08T23:30:43.740 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T23:30:43.740 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:30:43.853 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.849+0000 7f5b327bd700 1 Processor -- start 2024-05-08T23:30:43.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.849+0000 7f5b327bd700 1 -- start start 2024-05-08T23:30:43.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.849+0000 7f5b327bd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b2c0ce510 0x7f5b2c15a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.849+0000 7f5b327bd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5b2c15aaa0 0x7f5b2c13d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.849+0000 7f5b327bd700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2c0ce080 con 0x7f5b2c071ab0 2024-05-08T23:30:43.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.849+0000 7f5b327bd700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b2c0cd7c0 con 0x7f5b2c15aaa0 2024-05-08T23:30:43.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.849+0000 7f5b327bd700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c0cda30 con 0x7f5b2c0ce510 2024-05-08T23:30:43.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.849+0000 7f5b317bb700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5b2c071ab0 0x7f5b2c071e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55010/0 (socket says 172.21.15.81:55010) 2024-05-08T23:30:43.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.849+0000 7f5b317bb700 1 -- 172.21.15.81:0/3593791962 learned_addr learned my addr 172.21.15.81:0/3593791962 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:43.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.849+0000 7f5b30fba700 1 --2- 172.21.15.81:0/3593791962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b2c0ce510 0x7f5b2c15a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.849+0000 7f5b31fbc700 1 --2- 172.21.15.81:0/3593791962 >> v2:172.21.15.117:3300/0 conn(0x7f5b2c15aaa0 0x7f5b2c13d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.849+0000 7f5b23fff700 1 -- 172.21.15.81:0/3593791962 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996657852 0 0) 0x7f5b2c0ce080 con 0x7f5b2c071ab0 2024-05-08T23:30:43.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.849+0000 7f5b23fff700 1 -- 172.21.15.81:0/3593791962 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b180035d0 con 0x7f5b2c071ab0 2024-05-08T23:30:43.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b30fba700 1 -- 172.21.15.81:0/3593791962 >> v2:172.21.15.117:3300/0 conn(0x7f5b2c15aaa0 msgr2=0x7f5b2c13d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:43.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b30fba700 1 --2- 172.21.15.81:0/3593791962 >> v2:172.21.15.117:3300/0 conn(0x7f5b2c15aaa0 0x7f5b2c13d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b30fba700 1 -- 172.21.15.81:0/3593791962 >> v1:172.21.15.81:6789/0 conn(0x7f5b2c071ab0 legacy=0x7f5b2c071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b30fba700 1 -- 172.21.15.81:0/3593791962 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c0cdd10 con 0x7f5b2c0ce510 2024-05-08T23:30:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b31fbc700 1 --2- 172.21.15.81:0/3593791962 >> v2:172.21.15.117:3300/0 conn(0x7f5b2c15aaa0 0x7f5b2c13d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b30fba700 1 --2- 172.21.15.81:0/3593791962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b2c0ce510 0x7f5b2c15a560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c009850 tx=0x7f5b1c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85f609e82032ea04 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b23fff700 1 -- 172.21.15.81:0/3593791962 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c011830 con 0x7f5b2c0ce510 2024-05-08T23:30:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b23fff700 1 -- 172.21.15.81:0/3593791962 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c011990 con 0x7f5b2c0ce510 2024-05-08T23:30:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b23fff700 1 -- 172.21.15.81:0/3593791962 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c011d00 con 0x7f5b2c0ce510 2024-05-08T23:30:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 -- 172.21.15.81:0/3593791962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b2c0ce510 msgr2=0x7f5b2c15a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 --2- 172.21.15.81:0/3593791962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b2c0ce510 0x7f5b2c15a560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c009850 tx=0x7f5b1c009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 -- 172.21.15.81:0/3593791962 shutdown_connections 2024-05-08T23:30:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 --2- 172.21.15.81:0/3593791962 >> v2:172.21.15.117:3300/0 conn(0x7f5b2c15aaa0 0x7f5b2c13d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 --2- 172.21.15.81:0/3593791962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b2c0ce510 0x7f5b2c15a560 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 -- 172.21.15.81:0/3593791962 >> 172.21.15.81:0/3593791962 conn(0x7f5b2c06c330 msgr2=0x7f5b2c06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 -- 172.21.15.81:0/3593791962 shutdown_connections 2024-05-08T23:30:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 -- 172.21.15.81:0/3593791962 wait complete. 2024-05-08T23:30:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 Processor -- start 2024-05-08T23:30:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 -- start start 2024-05-08T23:30:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c071ab0 0x7f5b2c155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b2c0ce510 0x7f5b2c1541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b2c15aaa0 0x7f5b2c1427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c1563e0 con 0x7f5b2c15aaa0 2024-05-08T23:30:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c1566c0 con 0x7f5b2c071ab0 2024-05-08T23:30:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b327bd700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c142000 con 0x7f5b2c0ce510 2024-05-08T23:30:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b31fbc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b2c15aaa0 0x7f5b2c1427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b31fbc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b2c15aaa0 0x7f5b2c1427e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:32972/0 (socket says 172.21.15.81:32972) 2024-05-08T23:30:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b31fbc700 1 -- 172.21.15.81:0/2704209024 learned_addr learned my addr 172.21.15.81:0/2704209024 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b30fba700 1 --2- 172.21.15.81:0/2704209024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b2c0ce510 0x7f5b2c1541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.853+0000 7f5b317bb700 1 --2- 172.21.15.81:0/2704209024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c071ab0 0x7f5b2c155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b31fbc700 1 -- 172.21.15.81:0/2704209024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b2c0ce510 msgr2=0x7f5b2c1541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b31fbc700 1 --2- 172.21.15.81:0/2704209024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b2c0ce510 0x7f5b2c1541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b31fbc700 1 -- 172.21.15.81:0/2704209024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c071ab0 msgr2=0x7f5b2c155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:43.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b31fbc700 1 --2- 172.21.15.81:0/2704209024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c071ab0 0x7f5b2c155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:43.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b31fbc700 1 -- 172.21.15.81:0/2704209024 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b24009a70 con 0x7f5b2c15aaa0 2024-05-08T23:30:43.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b31fbc700 1 --2- 172.21.15.81:0/2704209024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b2c15aaa0 0x7f5b2c1427e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400efc0 tx=0x7f5b2400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:43.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b21ffb700 1 -- 172.21.15.81:0/2704209024 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2400cb40 con 0x7f5b2c15aaa0 2024-05-08T23:30:43.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b21ffb700 1 -- 172.21.15.81:0/2704209024 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2400f040 con 0x7f5b2c15aaa0 2024-05-08T23:30:43.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b327bd700 1 -- 172.21.15.81:0/2704209024 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1c0094b0 con 0x7f5b2c15aaa0 2024-05-08T23:30:43.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b21ffb700 1 -- 172.21.15.81:0/2704209024 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24003a40 con 0x7f5b2c15aaa0 2024-05-08T23:30:43.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b327bd700 1 -- 172.21.15.81:0/2704209024 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2c1569a0 con 0x7f5b2c15aaa0 2024-05-08T23:30:43.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b327bd700 1 -- 172.21.15.81:0/2704209024 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2c0684b0 con 0x7f5b2c15aaa0 2024-05-08T23:30:43.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b21ffb700 1 -- 172.21.15.81:0/2704209024 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5b24014480 con 0x7f5b2c15aaa0 2024-05-08T23:30:43.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b21ffb700 1 --2- 172.21.15.81:0/2704209024 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5b10066730 0x7f5b10068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:43.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b21ffb700 1 -- 172.21.15.81:0/2704209024 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b24084ba0 con 0x7f5b2c15aaa0 2024-05-08T23:30:43.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.857+0000 7f5b317bb700 1 --2- 172.21.15.81:0/2704209024 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5b10066730 0x7f5b10068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:43.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.861+0000 7f5b317bb700 1 --2- 172.21.15.81:0/2704209024 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5b10066730 0x7f5b10068bf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5b28005930 tx=0x7f5b28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:43.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:43.865+0000 7f5b21ffb700 1 -- 172.21.15.81:0/2704209024 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5b24051f10 con 0x7f5b2c15aaa0 2024-05-08T23:30:44.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.037+0000 7f5b327bd700 1 -- 172.21.15.81:0/2704209024 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b2c070110 con 0x7f5b2c15aaa0 2024-05-08T23:30:44.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.037+0000 7f5b21ffb700 1 -- 172.21.15.81:0/2704209024 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f5b24018090 con 0x7f5b2c15aaa0 2024-05-08T23:30:44.043 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:44.043 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:30:44.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.041+0000 7f5b327bd700 1 -- 172.21.15.81:0/2704209024 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5b10066730 msgr2=0x7f5b10068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.041+0000 7f5b327bd700 1 --2- 172.21.15.81:0/2704209024 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5b10066730 0x7f5b10068bf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5b28005930 tx=0x7f5b28009040 comp rx=0 tx=0).stop 2024-05-08T23:30:44.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.041+0000 7f5b327bd700 1 -- 172.21.15.81:0/2704209024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b2c15aaa0 msgr2=0x7f5b2c1427e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.041+0000 7f5b327bd700 1 --2- 172.21.15.81:0/2704209024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b2c15aaa0 0x7f5b2c1427e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400efc0 tx=0x7f5b2400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.041+0000 7f5b327bd700 1 -- 172.21.15.81:0/2704209024 shutdown_connections 2024-05-08T23:30:44.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.041+0000 7f5b327bd700 1 --2- 172.21.15.81:0/2704209024 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5b10066730 0x7f5b10068bf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.041+0000 7f5b327bd700 1 --2- 172.21.15.81:0/2704209024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b2c15aaa0 0x7f5b2c1427e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.041+0000 7f5b327bd700 1 --2- 172.21.15.81:0/2704209024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b2c0ce510 0x7f5b2c1541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.041+0000 7f5b327bd700 1 --2- 172.21.15.81:0/2704209024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c071ab0 0x7f5b2c155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.041+0000 7f5b327bd700 1 -- 172.21.15.81:0/2704209024 >> 172.21.15.81:0/2704209024 conn(0x7f5b2c06c330 msgr2=0x7f5b2c071330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.041+0000 7f5b327bd700 1 -- 172.21.15.81:0/2704209024 shutdown_connections 2024-05-08T23:30:44.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.041+0000 7f5b327bd700 1 -- 172.21.15.81:0/2704209024 wait complete. 2024-05-08T23:30:44.061 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T23:30:44.062 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T23:30:44.062 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T23:30:44.062 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T23:30:44.063 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T23:30:44.063 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T23:30:44.064 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-08T23:30:44.064 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-08T23:30:44.214 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.209+0000 7fad50b73700 1 Processor -- start 2024-05-08T23:30:44.214 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.209+0000 7fad50b73700 1 -- start start 2024-05-08T23:30:44.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.209+0000 7fd425c35700 1 Processor -- start 2024-05-08T23:30:44.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.209+0000 7fd425c35700 1 -- start start 2024-05-08T23:30:44.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.209+0000 7fad50b73700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad4c0b1f60 0x7fad4c06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.209+0000 7fad50b73700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad4c0b2740 con 0x7fad4c160c80 2024-05-08T23:30:44.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.209+0000 7fad50b73700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad4c06d990 con 0x7fad4c0ac8e0 2024-05-08T23:30:44.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.209+0000 7fad50b73700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c146d60 con 0x7fad4c0b1f60 2024-05-08T23:30:44.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.209+0000 7fad4b7fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fad4c160c80 0x7fad4c161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55016/0 (socket says 172.21.15.81:55016) 2024-05-08T23:30:44.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.209+0000 7fad4b7fe700 1 -- 172.21.15.81:0/3954392101 learned_addr learned my addr 172.21.15.81:0/3954392101 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:44.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.209+0000 7fad4affd700 1 --2- 172.21.15.81:0/3954392101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad4c0b1f60 0x7fad4c06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fad4affd700 1 -- 172.21.15.81:0/3954392101 >> v1:172.21.15.117:6789/0 conn(0x7fad4c0ac8e0 legacy=0x7fad4c0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fad4affd700 1 -- 172.21.15.81:0/3954392101 >> v1:172.21.15.81:6789/0 conn(0x7fad4c160c80 legacy=0x7fad4c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fad4affd700 1 -- 172.21.15.81:0/3954392101 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4c0b2460 con 0x7fad4c0b1f60 2024-05-08T23:30:44.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd425c35700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd420160c80 0x7fd420161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd425c35700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd4200ac8e0 0x7fd4200acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd425c35700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4200b2740 con 0x7fd4200b1f60 2024-05-08T23:30:44.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd425c35700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd42006d990 con 0x7fd4200ac8e0 2024-05-08T23:30:44.218 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fad4affd700 1 --2- 172.21.15.81:0/3954392101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad4c0b1f60 0x7fad4c06e4b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009880 tx=0x7fad3c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=817d233c773633b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd425c35700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd420146d60 con 0x7fd420160c80 2024-05-08T23:30:44.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd41ffff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd4200b1f60 0x7fd42006e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55032/0 (socket says 172.21.15.81:55032) 2024-05-08T23:30:44.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd41ffff700 1 -- 172.21.15.81:0/3509829953 learned_addr learned my addr 172.21.15.81:0/3509829953 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:44.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd424c33700 1 --2- 172.21.15.81:0/3509829953 >> v2:172.21.15.117:3300/0 conn(0x7fd420160c80 0x7fd420161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd425434700 1 --2- 172.21.15.81:0/3509829953 >> v2:172.21.15.81:3300/0 conn(0x7fd4200ac8e0 0x7fd4200acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd425434700 1 -- 172.21.15.81:0/3509829953 >> v2:172.21.15.117:3300/0 conn(0x7fd420160c80 msgr2=0x7fd420161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd425434700 1 --2- 172.21.15.81:0/3509829953 >> v2:172.21.15.117:3300/0 conn(0x7fd420160c80 0x7fd420161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd425434700 1 -- 172.21.15.81:0/3509829953 >> v1:172.21.15.81:6789/0 conn(0x7fd4200b1f60 legacy=0x7fd42006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd425434700 1 -- 172.21.15.81:0/3509829953 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4201615a0 con 0x7fd4200ac8e0 2024-05-08T23:30:44.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd425434700 1 --2- 172.21.15.81:0/3509829953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4200ac8e0 0x7fd4200acd30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4200b1bd0 tx=0x7fd410009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89855139f91c815f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fbfcf59e700 1 Processor -- start 2024-05-08T23:30:44.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fbfcf59e700 1 -- start start 2024-05-08T23:30:44.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7f3dbc83a700 1 Processor -- start 2024-05-08T23:30:44.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7f3dbc83a700 1 -- start start 2024-05-08T23:30:44.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fbfcf59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbfd0140130 0x7fbfd00b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fad4a7fc700 1 -- 172.21.15.81:0/3954392101 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c004230 con 0x7fad4c0b1f60 2024-05-08T23:30:44.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fad4a7fc700 1 -- 172.21.15.81:0/3954392101 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3c004390 con 0x7fad4c0b1f60 2024-05-08T23:30:44.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fad4a7fc700 1 -- 172.21.15.81:0/3954392101 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c0036c0 con 0x7fad4c0b1f60 2024-05-08T23:30:44.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fbfcf59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfd0142c10 0x7fbfd01613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fbfcf59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd00b5040 con 0x7fbfd01427a0 2024-05-08T23:30:44.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfcf59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd00b3fa0 con 0x7fbfd0140130 2024-05-08T23:30:44.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfcf59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd00b4280 con 0x7fbfd0142c10 2024-05-08T23:30:44.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd41f7fe700 1 -- 172.21.15.81:0/3509829953 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410004230 con 0x7fd4200ac8e0 2024-05-08T23:30:44.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd41f7fe700 1 -- 172.21.15.81:0/3509829953 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd410004390 con 0x7fd4200ac8e0 2024-05-08T23:30:44.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.213+0000 7fd41f7fe700 1 -- 172.21.15.81:0/3509829953 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4100036c0 con 0x7fd4200ac8e0 2024-05-08T23:30:44.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfce59c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbfd0140130 0x7fbfd00b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfce59c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbfd0140130 0x7fbfd00b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:44232/0 (socket says 172.21.15.81:44232) 2024-05-08T23:30:44.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfce59c700 1 -- 172.21.15.81:0/1037569337 learned_addr learned my addr 172.21.15.81:0/1037569337 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc46ab08700 1 Processor -- start 2024-05-08T23:30:44.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc46ab08700 1 -- start start 2024-05-08T23:30:44.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fad50b73700 1 -- 172.21.15.81:0/3954392101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad4c0b1f60 msgr2=0x7fad4c06e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fad50b73700 1 --2- 172.21.15.81:0/3954392101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad4c0b1f60 0x7fad4c06e4b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009880 tx=0x7fad3c009b90 comp rx=0 tx=0).stop 2024-05-08T23:30:44.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fad50b73700 1 -- 172.21.15.81:0/3954392101 shutdown_connections 2024-05-08T23:30:44.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fad50b73700 1 --2- 172.21.15.81:0/3954392101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad4c0b1f60 0x7fad4c06e4b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fad50b73700 1 -- 172.21.15.81:0/3954392101 >> 172.21.15.81:0/3954392101 conn(0x7fad4c0aebd0 msgr2=0x7fad4c0aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fd425c35700 1 -- 172.21.15.81:0/3509829953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4200ac8e0 msgr2=0x7fd4200acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fd425c35700 1 --2- 172.21.15.81:0/3509829953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4200ac8e0 0x7fd4200acd30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4200b1bd0 tx=0x7fd410009b30 comp rx=0 tx=0).stop 2024-05-08T23:30:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3dbc83a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3db80ce510 0x7f3db815a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3dbc83a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3db815aaa0 0x7f3db813d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3dbc83a700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db80ce080 con 0x7f3db8071ab0 2024-05-08T23:30:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3dbc83a700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db80cd7c0 con 0x7f3db815aaa0 2024-05-08T23:30:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3dbc83a700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db80cda30 con 0x7f3db80ce510 2024-05-08T23:30:44.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fad50b73700 1 -- 172.21.15.81:0/3954392101 shutdown_connections 2024-05-08T23:30:44.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3db77fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f3db8071ab0 0x7f3db8071e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55036/0 (socket says 172.21.15.81:55036) 2024-05-08T23:30:44.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3db77fe700 1 -- 172.21.15.81:0/2704298388 learned_addr learned my addr 172.21.15.81:0/2704298388 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:44.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fad50b73700 1 -- 172.21.15.81:0/3954392101 wait complete. 2024-05-08T23:30:44.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3db7fff700 1 --2- 172.21.15.81:0/2704298388 >> v2:172.21.15.81:3300/0 conn(0x7f3db815aaa0 0x7f3db813d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fd425c35700 1 -- 172.21.15.81:0/3509829953 shutdown_connections 2024-05-08T23:30:44.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fd425c35700 1 --2- 172.21.15.81:0/3509829953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4200ac8e0 0x7fd4200acd30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc46ab08700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc46415b1c0 0x7fc4640b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc46ab08700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4640b3e50 0x7fc46406d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc46ab08700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4640b1fd0 con 0x7fc46406dbb0 2024-05-08T23:30:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc46ab08700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc46406e020 con 0x7fc46415b1c0 2024-05-08T23:30:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc46ab08700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640b22b0 con 0x7fc4640b3e50 2024-05-08T23:30:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc4688a4700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc46415b1c0 0x7fc4640b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc4688a4700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc46415b1c0 0x7fc4640b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33000/0 (socket says 172.21.15.81:33000) 2024-05-08T23:30:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc4688a4700 1 -- 172.21.15.81:0/1655366077 learned_addr learned my addr 172.21.15.81:0/1655366077 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc463fff700 1 --2- 172.21.15.81:0/1655366077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4640b3e50 0x7fc46406d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc4688a4700 1 -- 172.21.15.81:0/1655366077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4640b3e50 msgr2=0x7fc46406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc4688a4700 1 --2- 172.21.15.81:0/1655366077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4640b3e50 0x7fc46406d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc4688a4700 1 -- 172.21.15.81:0/1655366077 >> v1:172.21.15.117:6789/0 conn(0x7fc46406dbb0 legacy=0x7fc46415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc4688a4700 1 -- 172.21.15.81:0/1655366077 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4640b2590 con 0x7fc46415b1c0 2024-05-08T23:30:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3db7fff700 1 -- 172.21.15.81:0/2704298388 >> v2:172.21.15.117:3300/0 conn(0x7f3db80ce510 msgr2=0x7f3db815a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3db7fff700 1 --2- 172.21.15.81:0/2704298388 >> v2:172.21.15.117:3300/0 conn(0x7f3db80ce510 0x7f3db815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3db7fff700 1 -- 172.21.15.81:0/2704298388 >> v1:172.21.15.81:6789/0 conn(0x7f3db8071ab0 legacy=0x7f3db8071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3db7fff700 1 -- 172.21.15.81:0/2704298388 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db80cdd10 con 0x7f3db815aaa0 2024-05-08T23:30:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3db7fff700 1 --2- 172.21.15.81:0/2704298388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3db815aaa0 0x7f3db813d500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3db806bb20 tx=0x7f3da8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6688b7179be8f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3db67fc700 1 -- 172.21.15.81:0/2704298388 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8011830 con 0x7f3db815aaa0 2024-05-08T23:30:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfced9d700 1 --2- 172.21.15.81:0/1037569337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfd0142c10 0x7fbfd01613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfce59c700 1 -- 172.21.15.81:0/1037569337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfd0142c10 msgr2=0x7fbfd01613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfce59c700 1 --2- 172.21.15.81:0/1037569337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfd0142c10 0x7fbfd01613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfce59c700 1 -- 172.21.15.81:0/1037569337 >> v1:172.21.15.117:6789/0 conn(0x7fbfd01427a0 legacy=0x7fbfd0160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfce59c700 1 -- 172.21.15.81:0/1037569337 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd00b4560 con 0x7fbfd0140130 2024-05-08T23:30:44.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfce59c700 1 --2- 172.21.15.81:0/1037569337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0140130 0x7fbfd00b2110 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009910 tx=0x7fbfc4009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f14abbe0fdd88b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfcd59a700 1 -- 172.21.15.81:0/1037569337 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc400ba60 con 0x7fbfd0140130 2024-05-08T23:30:44.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfcd59a700 1 -- 172.21.15.81:0/1037569337 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc400bbc0 con 0x7fbfd0140130 2024-05-08T23:30:44.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfcd59a700 1 -- 172.21.15.81:0/1037569337 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4009e60 con 0x7fbfd0140130 2024-05-08T23:30:44.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3db67fc700 1 -- 172.21.15.81:0/2704298388 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da8011990 con 0x7f3db815aaa0 2024-05-08T23:30:44.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3db67fc700 1 -- 172.21.15.81:0/2704298388 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8011d40 con 0x7f3db815aaa0 2024-05-08T23:30:44.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fd425c35700 1 --2- 172.21.15.81:0/3509829953 >> v2:172.21.15.117:3300/0 conn(0x7fd420160c80 0x7fd420161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fd425c35700 1 -- 172.21.15.81:0/3509829953 >> 172.21.15.81:0/3509829953 conn(0x7fd4200aebd0 msgr2=0x7fd4200aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc4688a4700 1 --2- 172.21.15.81:0/1655366077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc46415b1c0 0x7fc4640b3800 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009e50 tx=0x7fc45c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8897467fe4d038a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc4637fe700 1 -- 172.21.15.81:0/1655366077 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c00e070 con 0x7fc46415b1c0 2024-05-08T23:30:44.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfcf59e700 1 -- 172.21.15.81:0/1037569337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0140130 msgr2=0x7fbfd00b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfcf59e700 1 --2- 172.21.15.81:0/1037569337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0140130 0x7fbfd00b2110 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009910 tx=0x7fbfc4009c20 comp rx=0 tx=0).stop 2024-05-08T23:30:44.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfcf59e700 1 -- 172.21.15.81:0/1037569337 shutdown_connections 2024-05-08T23:30:44.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfcf59e700 1 --2- 172.21.15.81:0/1037569337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfd0142c10 0x7fbfd01613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfcf59e700 1 --2- 172.21.15.81:0/1037569337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0140130 0x7fbfd00b2110 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfcf59e700 1 -- 172.21.15.81:0/1037569337 >> 172.21.15.81:0/1037569337 conn(0x7fbfd00aebd0 msgr2=0x7fbfd00aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc4637fe700 1 -- 172.21.15.81:0/1655366077 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45c003a40 con 0x7fc46415b1c0 2024-05-08T23:30:44.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fc4637fe700 1 -- 172.21.15.81:0/1655366077 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c003df0 con 0x7fc46415b1c0 2024-05-08T23:30:44.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfcf59e700 1 -- 172.21.15.81:0/1037569337 shutdown_connections 2024-05-08T23:30:44.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fd425c35700 1 -- 172.21.15.81:0/3509829953 shutdown_connections 2024-05-08T23:30:44.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fd425c35700 1 -- 172.21.15.81:0/3509829953 wait complete. 2024-05-08T23:30:44.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fbfcf59e700 1 -- 172.21.15.81:0/1037569337 wait complete. 2024-05-08T23:30:44.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fad50b73700 1 Processor -- start 2024-05-08T23:30:44.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7fad50b73700 1 -- start start 2024-05-08T23:30:44.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3dbc83a700 1 -- 172.21.15.81:0/2704298388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3db815aaa0 msgr2=0x7f3db813d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.217+0000 7f3dbc83a700 1 --2- 172.21.15.81:0/2704298388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3db815aaa0 0x7f3db813d500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3db806bb20 tx=0x7f3da8009b30 comp rx=0 tx=0).stop 2024-05-08T23:30:44.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3dbc83a700 1 -- 172.21.15.81:0/2704298388 shutdown_connections 2024-05-08T23:30:44.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3dbc83a700 1 --2- 172.21.15.81:0/2704298388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3db815aaa0 0x7f3db813d500 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3dbc83a700 1 --2- 172.21.15.81:0/2704298388 >> v2:172.21.15.117:3300/0 conn(0x7f3db80ce510 0x7f3db815a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3dbc83a700 1 -- 172.21.15.81:0/2704298388 >> 172.21.15.81:0/2704298388 conn(0x7f3db806c330 msgr2=0x7f3db806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fc46ab08700 1 -- 172.21.15.81:0/1655366077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc46415b1c0 msgr2=0x7fc4640b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fc46ab08700 1 --2- 172.21.15.81:0/1655366077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc46415b1c0 0x7fc4640b3800 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009e50 tx=0x7fc45c00b330 comp rx=0 tx=0).stop 2024-05-08T23:30:44.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3dbc83a700 1 -- 172.21.15.81:0/2704298388 shutdown_connections 2024-05-08T23:30:44.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3dbc83a700 1 -- 172.21.15.81:0/2704298388 wait complete. 2024-05-08T23:30:44.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad50b73700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad4c0ac8e0 0x7fad4c1e8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad50b73700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad4c0b1f60 0x7fad4c142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad50b73700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad4c160c80 0x7fad4c145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad50b73700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c143940 con 0x7fad4c160c80 2024-05-08T23:30:44.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad50b73700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c143c20 con 0x7fad4c0ac8e0 2024-05-08T23:30:44.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad50b73700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c143f00 con 0x7fad4c0b1f60 2024-05-08T23:30:44.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f50d4818700 1 Processor -- start 2024-05-08T23:30:44.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f50d4818700 1 -- start start 2024-05-08T23:30:44.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad4affd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad4c0b1f60 0x7fad4c142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad4b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad4c0ac8e0 0x7fad4c1e8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad4affd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad4c0b1f60 0x7fad4c142f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:53106/0 (socket says 172.21.15.81:53106) 2024-05-08T23:30:44.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad4b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad4c0ac8e0 0x7fad4c1e8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:44244/0 (socket says 172.21.15.81:44244) 2024-05-08T23:30:44.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad4affd700 1 -- 172.21.15.81:0/1013428084 learned_addr learned my addr 172.21.15.81:0/1013428084 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad4bfff700 1 --2- 172.21.15.81:0/1013428084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad4c160c80 0x7fad4c145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfcf59e700 1 Processor -- start 2024-05-08T23:30:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fc46ab08700 1 -- 172.21.15.81:0/1655366077 shutdown_connections 2024-05-08T23:30:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fc46ab08700 1 --2- 172.21.15.81:0/1655366077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4640b3e50 0x7fc46406d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fc46ab08700 1 --2- 172.21.15.81:0/1655366077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc46415b1c0 0x7fc4640b3800 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fc46ab08700 1 -- 172.21.15.81:0/1655366077 >> 172.21.15.81:0/1655366077 conn(0x7fc4640b0cd0 msgr2=0x7fc4640b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fd425c35700 1 Processor -- start 2024-05-08T23:30:44.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fd425c35700 1 -- start start 2024-05-08T23:30:44.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfcf59e700 1 -- start start 2024-05-08T23:30:44.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fc46ab08700 1 -- 172.21.15.81:0/1655366077 shutdown_connections 2024-05-08T23:30:44.240 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3dbc83a700 1 Processor -- start 2024-05-08T23:30:44.240 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3dbc83a700 1 -- start start 2024-05-08T23:30:44.240 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad4affd700 1 -- 172.21.15.81:0/1013428084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad4c0ac8e0 msgr2=0x7fad4c1e8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad4affd700 1 --2- 172.21.15.81:0/1013428084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad4c0ac8e0 0x7fad4c1e8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad4affd700 1 -- 172.21.15.81:0/1013428084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad4c160c80 msgr2=0x7fad4c145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad4affd700 1 --2- 172.21.15.81:0/1013428084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad4c160c80 0x7fad4c145440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad4affd700 1 -- 172.21.15.81:0/1013428084 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c0094b0 con 0x7fad4c0b1f60 2024-05-08T23:30:44.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad4b7fe700 1 --2- 172.21.15.81:0/1013428084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad4c0ac8e0 0x7fad4c1e8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:44.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad4bfff700 1 --2- 172.21.15.81:0/1013428084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad4c160c80 0x7fad4c145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:44.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad4affd700 1 --2- 172.21.15.81:0/1013428084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad4c0b1f60 0x7fad4c142f80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fad3c000c00 tx=0x7fad3c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad33fff700 1 -- 172.21.15.81:0/1013428084 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c003da0 con 0x7fad4c0b1f60 2024-05-08T23:30:44.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3f71609700 1 Processor -- start 2024-05-08T23:30:44.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad33fff700 1 -- 172.21.15.81:0/1013428084 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3c020070 con 0x7fad4c0b1f60 2024-05-08T23:30:44.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad33fff700 1 -- 172.21.15.81:0/1013428084 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c011870 con 0x7fad4c0b1f60 2024-05-08T23:30:44.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad50b73700 1 -- 172.21.15.81:0/1013428084 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4c1441e0 con 0x7fad4c0b1f60 2024-05-08T23:30:44.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fad50b73700 1 -- 172.21.15.81:0/1013428084 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4c1e93c0 con 0x7fad4c0b1f60 2024-05-08T23:30:44.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfcf59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfd0140130 0x7fbfd013dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f50cf7fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f50d00cbe70 0x7f50d00ce4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55046/0 (socket says 172.21.15.81:55046) 2024-05-08T23:30:44.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfcf59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfd01427a0 0x7fbfd0141680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfcf59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0142c10 0x7fbfd01390e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfcf59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd0137d00 con 0x7fbfd01427a0 2024-05-08T23:30:44.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfcf59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd0138690 con 0x7fbfd0142c10 2024-05-08T23:30:44.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfcf59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd0138c50 con 0x7fbfd0140130 2024-05-08T23:30:44.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfced9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0142c10 0x7fbfd01390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfced9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0142c10 0x7fbfd01390e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:44246/0 (socket says 172.21.15.81:44246) 2024-05-08T23:30:44.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfced9d700 1 -- 172.21.15.81:0/3446920639 learned_addr learned my addr 172.21.15.81:0/3446920639 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfce59c700 1 --2- 172.21.15.81:0/3446920639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfd0140130 0x7fbfd013dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fd425c35700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4200ac8e0 0x7fd42013af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fd425c35700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4200b1f60 0x7fd4201e8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fd425c35700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420160c80 0x7fd420138f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fd425c35700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42013b4a0 con 0x7fd4200ac8e0 2024-05-08T23:30:44.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fd425c35700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4201e7b00 con 0x7fd420160c80 2024-05-08T23:30:44.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fd425c35700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4201e7de0 con 0x7fd4200b1f60 2024-05-08T23:30:44.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfced9d700 1 -- 172.21.15.81:0/3446920639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfd0140130 msgr2=0x7fbfd013dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfced9d700 1 --2- 172.21.15.81:0/3446920639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfd0140130 0x7fbfd013dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfced9d700 1 -- 172.21.15.81:0/3446920639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfd01427a0 msgr2=0x7fbfd0141680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:30:44.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfced9d700 1 --2- 172.21.15.81:0/3446920639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfd01427a0 0x7fbfd0141680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f50cf7fe700 1 -- 172.21.15.81:0/2777394498 learned_addr learned my addr 172.21.15.81:0/2777394498 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:44.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f50d4818700 1 --2- 172.21.15.81:0/2777394498 >> v2:172.21.15.117:3300/0 conn(0x7f50d015b5c0 0x7f50d015ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fbfced9d700 1 -- 172.21.15.81:0/3446920639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc4009570 con 0x7fbfd0142c10 2024-05-08T23:30:44.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fd424c33700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4200ac8e0 0x7fd42013af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fd424c33700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4200ac8e0 0x7fd42013af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33022/0 (socket says 172.21.15.81:33022) 2024-05-08T23:30:44.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fd424c33700 1 -- 172.21.15.81:0/3120480336 learned_addr learned my addr 172.21.15.81:0/3120480336 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.247 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f50d4818700 1 -- 172.21.15.81:0/2777394498 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d00cc250 con 0x7f50d00cbe70 2024-05-08T23:30:44.247 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f50d4818700 1 -- 172.21.15.81:0/2777394498 --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d015b250 con 0x7f50d006dcd0 2024-05-08T23:30:44.247 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f50d4818700 1 -- 172.21.15.81:0/2777394498 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d015bf60 con 0x7f50d015b5c0 2024-05-08T23:30:44.247 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3dbc83a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3db8071ab0 0x7f3db813e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.247 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3dbc83a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db80ce510 0x7f3db815c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.247 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3dbc83a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db815aaa0 0x7f3db813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.247 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3dbc83a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db8142720 con 0x7f3db8071ab0 2024-05-08T23:30:44.247 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3dbc83a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db8141850 con 0x7f3db80ce510 2024-05-08T23:30:44.247 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3dbc83a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db8141aa0 con 0x7f3db815aaa0 2024-05-08T23:30:44.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3db7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db815aaa0 0x7f3db813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3db7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db815aaa0 0x7f3db813b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:53130/0 (socket says 172.21.15.81:53130) 2024-05-08T23:30:44.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3db7fff700 1 -- 172.21.15.81:0/3193333368 learned_addr learned my addr 172.21.15.81:0/3193333368 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fc46ab08700 1 -- 172.21.15.81:0/1655366077 wait complete. 2024-05-08T23:30:44.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3f71609700 1 -- start start 2024-05-08T23:30:44.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fc46ab08700 1 Processor -- start 2024-05-08T23:30:44.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3f71609700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f6c0cdc20 0x7f3f6c06d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3f71609700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f6c0b1f60 0x7f3f6c0b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3f71609700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6c0b2d90 con 0x7f3f6c0ce000 2024-05-08T23:30:44.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3f71609700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f6c150680 con 0x7f3f6c0cdc20 2024-05-08T23:30:44.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3f71609700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c0b2910 con 0x7f3f6c0b1f60 2024-05-08T23:30:44.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fad50b73700 1 -- 172.21.15.81:0/1013428084 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad2c000fc0 con 0x7fad4c0b1f60 2024-05-08T23:30:44.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7f3db77fe700 1 --2- 172.21.15.81:0/3193333368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3db8071ab0 0x7f3db813e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.221+0000 7fd425434700 1 --2- 172.21.15.81:0/3120480336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420160c80 0x7fd420138f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fbfced9d700 1 --2- 172.21.15.81:0/3446920639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0142c10 0x7fbfd01390e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8007e60 tx=0x7fbfc800e3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.251 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3f6affd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f6c0cdc20 0x7f3f6c06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.251 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3f6affd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f6c0cdc20 0x7f3f6c06d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:44274/0 (socket says 172.21.15.81:44274) 2024-05-08T23:30:44.251 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3f6b7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f6c0b1f60 0x7f3f6c0b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.251 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3f6affd700 1 -- 172.21.15.81:0/1853904347 learned_addr learned my addr 172.21.15.81:0/1853904347 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd424c33700 1 -- 172.21.15.81:0/3120480336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4200b1f60 msgr2=0x7fd4201e8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd424c33700 1 --2- 172.21.15.81:0/3120480336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4200b1f60 0x7fd4201e8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd424c33700 1 -- 172.21.15.81:0/3120480336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420160c80 msgr2=0x7fd420138f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd424c33700 1 --2- 172.21.15.81:0/3120480336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420160c80 0x7fd420138f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd424c33700 1 -- 172.21.15.81:0/3120480336 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4100094b0 con 0x7fd4200ac8e0 2024-05-08T23:30:44.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3db6ffd700 1 --2- 172.21.15.81:0/3193333368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db80ce510 0x7f3db815c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3db77fe700 1 -- 172.21.15.81:0/3193333368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db815aaa0 msgr2=0x7f3db813b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3db77fe700 1 --2- 172.21.15.81:0/3193333368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db815aaa0 0x7f3db813b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3db77fe700 1 -- 172.21.15.81:0/3193333368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db80ce510 msgr2=0x7f3db815c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3db77fe700 1 --2- 172.21.15.81:0/3193333368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db80ce510 0x7f3db815c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3db77fe700 1 -- 172.21.15.81:0/3193333368 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da80094b0 con 0x7f3db8071ab0 2024-05-08T23:30:44.253 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3db6ffd700 1 --2- 172.21.15.81:0/3193333368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db80ce510 0x7f3db815c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:44.253 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fbfbeffd700 1 -- 172.21.15.81:0/3446920639 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc800ede0 con 0x7fbfd0142c10 2024-05-08T23:30:44.253 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fbfbeffd700 1 -- 172.21.15.81:0/3446920639 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc8014420 con 0x7fbfd0142c10 2024-05-08T23:30:44.253 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fbfbeffd700 1 -- 172.21.15.81:0/3446920639 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80147d0 con 0x7fbfd0142c10 2024-05-08T23:30:44.253 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fbfcf59e700 1 -- 172.21.15.81:0/3446920639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd0137ee0 con 0x7fbfd0142c10 2024-05-08T23:30:44.253 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f50ceffd700 1 --2- 172.21.15.81:0/2777394498 >> v2:172.21.15.117:3300/0 conn(0x7f50d015b5c0 0x7f50d015ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd425434700 1 --2- 172.21.15.81:0/3120480336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420160c80 0x7fd420138f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:44.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd424c33700 1 --2- 172.21.15.81:0/3120480336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4200ac8e0 0x7fd42013af60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd40800e7c0 tx=0x7fd40800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fbfcf59e700 1 -- 172.21.15.81:0/3446920639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd01388e0 con 0x7fbfd0142c10 2024-05-08T23:30:44.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3f6affd700 1 -- 172.21.15.81:0/1853904347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f6c0b1f60 msgr2=0x7f3f6c0b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3f6affd700 1 --2- 172.21.15.81:0/1853904347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f6c0b1f60 0x7f3f6c0b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3f6affd700 1 -- 172.21.15.81:0/1853904347 >> v1:172.21.15.117:6789/0 conn(0x7f3f6c0ce000 legacy=0x7f3f6c15a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3f6affd700 1 -- 172.21.15.81:0/1853904347 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c0ce4b0 con 0x7f3f6c0cdc20 2024-05-08T23:30:44.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3db77fe700 1 --2- 172.21.15.81:0/3193333368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3db8071ab0 0x7f3db813e4b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3db000caa0 tx=0x7f3db000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f50ceffd700 1 -- 172.21.15.81:0/2777394498 >> v1:172.21.15.117:6789/0 conn(0x7f50d006dcd0 legacy=0x7f50d015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f50ceffd700 1 -- 172.21.15.81:0/2777394498 >> v1:172.21.15.81:6789/0 conn(0x7f50d00cbe70 legacy=0x7f50d00ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f50ceffd700 1 -- 172.21.15.81:0/2777394498 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d00cd9e0 con 0x7f50d015b5c0 2024-05-08T23:30:44.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc46ab08700 1 -- start start 2024-05-08T23:30:44.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc46ab08700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc46406dbb0 0x7fc464145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc46ab08700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4640b3e50 0x7fc464143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc46ab08700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc46415b1c0 0x7fc464143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc46ab08700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc464156ee0 con 0x7fc46415b1c0 2024-05-08T23:30:44.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc46ab08700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc464145260 con 0x7fc4640b3e50 2024-05-08T23:30:44.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc46ab08700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46415e8e0 con 0x7fc46406dbb0 2024-05-08T23:30:44.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc463fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4640b3e50 0x7fc464143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc463fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4640b3e50 0x7fc464143a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:44290/0 (socket says 172.21.15.81:44290) 2024-05-08T23:30:44.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc463fff700 1 -- 172.21.15.81:0/3544753852 learned_addr learned my addr 172.21.15.81:0/3544753852 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd41d7fa700 1 -- 172.21.15.81:0/3120480336 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408013680 con 0x7fd4200ac8e0 2024-05-08T23:30:44.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd41d7fa700 1 -- 172.21.15.81:0/3120480336 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4080137e0 con 0x7fd4200ac8e0 2024-05-08T23:30:44.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd41d7fa700 1 -- 172.21.15.81:0/3120480336 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408013b90 con 0x7fd4200ac8e0 2024-05-08T23:30:44.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd425c35700 1 -- 172.21.15.81:0/3120480336 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4201e80c0 con 0x7fd4200ac8e0 2024-05-08T23:30:44.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd425c35700 1 -- 172.21.15.81:0/3120480336 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4201e8680 con 0x7fd4200ac8e0 2024-05-08T23:30:44.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3f6affd700 1 --2- 172.21.15.81:0/1853904347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f6c0cdc20 0x7f3f6c06d820 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c008910 tx=0x7f3f5c00d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70e053576b7f99d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f50ceffd700 1 --2- 172.21.15.81:0/2777394498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50d015b5c0 0x7f50d015ba20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009850 tx=0x7f50c0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb25802fc9726b08 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc4690a5700 1 --2- 172.21.15.81:0/3544753852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc46415b1c0 0x7fc464143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fbfcf59e700 1 -- 172.21.15.81:0/3446920639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbfd0143040 con 0x7fbfd0142c10 2024-05-08T23:30:44.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3d9ffff700 1 -- 172.21.15.81:0/3193333368 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0007480 con 0x7f3db8071ab0 2024-05-08T23:30:44.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd425c35700 1 -- 172.21.15.81:0/3120480336 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd42006d620 con 0x7fd4200ac8e0 2024-05-08T23:30:44.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3d9ffff700 1 -- 172.21.15.81:0/3193333368 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db00075e0 con 0x7f3db8071ab0 2024-05-08T23:30:44.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3d9ffff700 1 -- 172.21.15.81:0/3193333368 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0007970 con 0x7f3db8071ab0 2024-05-08T23:30:44.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3dbc83a700 1 -- 172.21.15.81:0/3193333368 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db8141d80 con 0x7f3db8071ab0 2024-05-08T23:30:44.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3dbc83a700 1 -- 172.21.15.81:0/3193333368 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db8142340 con 0x7f3db8071ab0 2024-05-08T23:30:44.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fad33fff700 1 -- 172.21.15.81:0/1013428084 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fad3c022030 con 0x7fad4c0b1f60 2024-05-08T23:30:44.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc4690a5700 1 -- 172.21.15.81:0/3544753852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc46406dbb0 msgr2=0x7fc464145d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc4690a5700 1 --2- 172.21.15.81:0/3544753852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc46406dbb0 0x7fc464145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f50ce7fc700 1 -- 172.21.15.81:0/2777394498 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0004230 con 0x7f50d015b5c0 2024-05-08T23:30:44.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc4690a5700 1 -- 172.21.15.81:0/3544753852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4640b3e50 msgr2=0x7fc464143a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc4690a5700 1 --2- 172.21.15.81:0/3544753852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4640b3e50 0x7fc464143a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc4690a5700 1 -- 172.21.15.81:0/3544753852 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c009b30 con 0x7fc46415b1c0 2024-05-08T23:30:44.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc463fff700 1 --2- 172.21.15.81:0/3544753852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4640b3e50 0x7fc464143a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:44.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f50ce7fc700 1 -- 172.21.15.81:0/2777394498 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c0004390 con 0x7f50d015b5c0 2024-05-08T23:30:44.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f50ce7fc700 1 -- 172.21.15.81:0/2777394498 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c00036c0 con 0x7f50d015b5c0 2024-05-08T23:30:44.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fc4690a5700 1 --2- 172.21.15.81:0/3544753852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc46415b1c0 0x7fc464143fd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc45800c410 tx=0x7fc45800c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd41d7fa700 1 -- 172.21.15.81:0/3120480336 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd408013cf0 con 0x7fd4200ac8e0 2024-05-08T23:30:44.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fad33fff700 1 --2- 172.21.15.81:0/1013428084 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fad38066760 0x7fad38068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fad33fff700 1 -- 172.21.15.81:0/1013428084 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad3c086080 con 0x7fad4c0b1f60 2024-05-08T23:30:44.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3f69ffb700 1 -- 172.21.15.81:0/1853904347 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c010070 con 0x7f3f6c0cdc20 2024-05-08T23:30:44.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3f69ffb700 1 -- 172.21.15.81:0/1853904347 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c00d9d0 con 0x7f3f6c0cdc20 2024-05-08T23:30:44.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7f3f69ffb700 1 -- 172.21.15.81:0/1853904347 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c00dd60 con 0x7f3f6c0cdc20 2024-05-08T23:30:44.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd41d7fa700 1 --2- 172.21.15.81:0/3120480336 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd4000667b0 0x7fd400068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fd41d7fa700 1 -- 172.21.15.81:0/3120480336 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd408083e10 con 0x7fd4200ac8e0 2024-05-08T23:30:44.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fad33fff700 1 --2- 172.21.15.81:0/1013428084 >> [v2:172.21.15.81:6805/26094,v1:172.21.15.81:6807/26094] conn(0x7fad3806e980 0x7fad38070da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fad33fff700 1 -- 172.21.15.81:0/1013428084 --> [v2:172.21.15.81:6805/26094,v1:172.21.15.81:6807/26094] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad38071450 con 0x7fad3806e980 2024-05-08T23:30:44.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.225+0000 7fad33fff700 1 -- 172.21.15.81:0/1013428084 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad3c086430 con 0x7fad4c0b1f60 2024-05-08T23:30:44.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fad4b7fe700 1 --2- 172.21.15.81:0/1013428084 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fad38066760 0x7fad38068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fad4bfff700 1 --2- 172.21.15.81:0/1013428084 >> [v2:172.21.15.81:6805/26094,v1:172.21.15.81:6807/26094] conn(0x7fad3806e980 0x7fad38070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fad4bfff700 1 --2- 172.21.15.81:0/1013428084 >> [v2:172.21.15.81:6805/26094,v1:172.21.15.81:6807/26094] conn(0x7fad3806e980 0x7fad38070da0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fbfbeffd700 1 -- 172.21.15.81:0/3446920639 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbfc8014930 con 0x7fbfd0142c10 2024-05-08T23:30:44.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f50d4818700 1 -- 172.21.15.81:0/2777394498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50d015b5c0 msgr2=0x7f50d015ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f50d4818700 1 --2- 172.21.15.81:0/2777394498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50d015b5c0 0x7f50d015ba20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009850 tx=0x7f50c0009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:44.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3f71609700 1 -- 172.21.15.81:0/1853904347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f6c0cdc20 msgr2=0x7f3f6c06d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3f71609700 1 --2- 172.21.15.81:0/1853904347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f6c0cdc20 0x7f3f6c06d820 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c008910 tx=0x7f3f5c00d330 comp rx=0 tx=0).stop 2024-05-08T23:30:44.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3dbc83a700 1 -- 172.21.15.81:0/3193333368 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3db815af30 con 0x7f3db8071ab0 2024-05-08T23:30:44.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fad4b7fe700 1 --2- 172.21.15.81:0/1013428084 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fad38066760 0x7fad38068c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fad44001dc0 tx=0x7fad44001d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fbfbeffd700 1 --2- 172.21.15.81:0/3446920639 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbfb0066830 0x7fbfb0068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f50d4818700 1 -- 172.21.15.81:0/2777394498 shutdown_connections 2024-05-08T23:30:44.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f50d4818700 1 --2- 172.21.15.81:0/2777394498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50d015b5c0 0x7f50d015ba20 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009850 tx=0x7f50c0009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:44.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f50d4818700 1 -- 172.21.15.81:0/2777394498 >> 172.21.15.81:0/2777394498 conn(0x7f50d00aebd0 msgr2=0x7f50d00aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fad33fff700 1 -- 172.21.15.81:0/1013428084 <== osd.3 v2:172.21.15.81:6805/26094 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fad38071450 con 0x7fad3806e980 2024-05-08T23:30:44.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fbfce59c700 1 --2- 172.21.15.81:0/3446920639 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbfb0066830 0x7fbfb0068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fbfbeffd700 1 -- 172.21.15.81:0/3446920639 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfc80855b0 con 0x7fbfd0142c10 2024-05-08T23:30:44.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fd41d7fa700 1 --2- 172.21.15.81:0/3120480336 >> [v2:172.21.15.117:6824/26390,v1:172.21.15.117:6825/26390] conn(0x7fd40006e9d0 0x7fd400070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fd41d7fa700 1 -- 172.21.15.81:0/3120480336 --> [v2:172.21.15.117:6824/26390,v1:172.21.15.117:6825/26390] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd4000714a0 con 0x7fd40006e9d0 2024-05-08T23:30:44.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fd41d7fa700 1 -- 172.21.15.81:0/3120480336 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd408086520 con 0x7fd4200ac8e0 2024-05-08T23:30:44.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fd41ffff700 1 --2- 172.21.15.81:0/3120480336 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd4000667b0 0x7fd400068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fd41ffff700 1 --2- 172.21.15.81:0/3120480336 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd4000667b0 0x7fd400068c70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd4180012e0 tx=0x7fd418001860 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f50d4818700 1 -- 172.21.15.81:0/2777394498 shutdown_connections 2024-05-08T23:30:44.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f50d4818700 1 -- 172.21.15.81:0/2777394498 wait complete. 2024-05-08T23:30:44.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f986759e700 1 Processor -- start 2024-05-08T23:30:44.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f986759e700 1 -- start start 2024-05-08T23:30:44.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fd425434700 1 --2- 172.21.15.81:0/3120480336 >> [v2:172.21.15.117:6824/26390,v1:172.21.15.117:6825/26390] conn(0x7fd40006e9d0 0x7fd400070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f986759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f986808a620 0x7f986806e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f986759e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f986806caa0 0x7f986806cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f986759e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986806c330 con 0x7f98680b3c40 2024-05-08T23:30:44.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f986759e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f986806c610 con 0x7f986806caa0 2024-05-08T23:30:44.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f986759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98680cc530 con 0x7f986808a620 2024-05-08T23:30:44.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f986659c700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f98680b3c40 0x7f98680b4020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55058/0 (socket says 172.21.15.81:55058) 2024-05-08T23:30:44.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f986659c700 1 -- 172.21.15.81:0/1150411626 learned_addr learned my addr 172.21.15.81:0/1150411626 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:44.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f9865d9b700 1 --2- 172.21.15.81:0/1150411626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f986808a620 0x7f986806e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f9865d9b700 1 -- 172.21.15.81:0/1150411626 >> v2:172.21.15.81:3300/0 conn(0x7f986806caa0 msgr2=0x7f986806cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f9866d9d700 1 --2- 172.21.15.81:0/1150411626 >> v2:172.21.15.81:3300/0 conn(0x7f986806caa0 0x7f986806cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f9865d9b700 1 --2- 172.21.15.81:0/1150411626 >> v2:172.21.15.81:3300/0 conn(0x7f986806caa0 0x7f986806cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f9865d9b700 1 -- 172.21.15.81:0/1150411626 >> v1:172.21.15.81:6789/0 conn(0x7f98680b3c40 legacy=0x7f98680b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f9865d9b700 1 -- 172.21.15.81:0/1150411626 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98680b2a10 con 0x7f986808a620 2024-05-08T23:30:44.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3f71609700 1 -- 172.21.15.81:0/1853904347 shutdown_connections 2024-05-08T23:30:44.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3f71609700 1 --2- 172.21.15.81:0/1853904347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f6c0b1f60 0x7f3f6c0b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3f71609700 1 --2- 172.21.15.81:0/1853904347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f6c0cdc20 0x7f3f6c06d820 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3f71609700 1 -- 172.21.15.81:0/1853904347 >> 172.21.15.81:0/1853904347 conn(0x7f3f6c0aebd0 msgr2=0x7f3f6c0aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f9866d9d700 1 --2- 172.21.15.81:0/1150411626 >> v2:172.21.15.81:3300/0 conn(0x7f986806caa0 0x7f986806cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:44.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fd425434700 1 --2- 172.21.15.81:0/3120480336 >> [v2:172.21.15.117:6824/26390,v1:172.21.15.117:6825/26390] conn(0x7fd40006e9d0 0x7fd400070df0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f9865d9b700 1 --2- 172.21.15.81:0/1150411626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f986808a620 0x7f986806e380 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f986000ad90 tx=0x7f986000c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=540ca3294326c34f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3d9ffff700 1 -- 172.21.15.81:0/3193333368 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3db001b3b0 con 0x7f3db8071ab0 2024-05-08T23:30:44.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3d9ffff700 1 --2- 172.21.15.81:0/3193333368 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3da4066780 0x7f3da4068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3d9ffff700 1 -- 172.21.15.81:0/3193333368 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3db0083970 con 0x7f3db8071ab0 2024-05-08T23:30:44.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3db6ffd700 1 --2- 172.21.15.81:0/3193333368 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3da4066780 0x7f3da4068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3d9ffff700 1 --2- 172.21.15.81:0/3193333368 >> [v2:172.21.15.81:6817/26092,v1:172.21.15.81:6819/26092] conn(0x7f3da406e9a0 0x7f3da4070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f50d4818700 1 Processor -- start 2024-05-08T23:30:44.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3d9ffff700 1 -- 172.21.15.81:0/3193333368 --> [v2:172.21.15.81:6817/26092,v1:172.21.15.81:6819/26092] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3da4071470 con 0x7f3da406e9a0 2024-05-08T23:30:44.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3d9ffff700 1 -- 172.21.15.81:0/3193333368 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3db00882a0 con 0x7f3db8071ab0 2024-05-08T23:30:44.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3db6ffd700 1 --2- 172.21.15.81:0/3193333368 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3da4066780 0x7f3da4068c40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3db806dfc0 tx=0x7f3dac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f3db7fff700 1 --2- 172.21.15.81:0/3193333368 >> [v2:172.21.15.81:6817/26092,v1:172.21.15.81:6819/26092] conn(0x7f3da406e9a0 0x7f3da4070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.274 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7f50d4818700 1 -- start start 2024-05-08T23:30:44.274 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f986559a700 1 -- 172.21.15.81:0/1150411626 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986000f070 con 0x7f986808a620 2024-05-08T23:30:44.274 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f986559a700 1 -- 172.21.15.81:0/1150411626 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986000cbd0 con 0x7f986808a620 2024-05-08T23:30:44.274 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f986559a700 1 -- 172.21.15.81:0/1150411626 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9860013430 con 0x7f986808a620 2024-05-08T23:30:44.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f3db7fff700 1 --2- 172.21.15.81:0/3193333368 >> [v2:172.21.15.81:6817/26092,v1:172.21.15.81:6819/26092] conn(0x7f3da406e9a0 0x7f3da4070dc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50d4818700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50d006dcd0 0x7f50d013a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50d4818700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50d00cbe70 0x7f50d0158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50d4818700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50d01552f0 0x7f50d0155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50d4818700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d0168a70 con 0x7f50d00cbe70 2024-05-08T23:30:44.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50d4818700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d0167e30 con 0x7f50d006dcd0 2024-05-08T23:30:44.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50d4818700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d01680d0 con 0x7f50d01552f0 2024-05-08T23:30:44.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f3f71609700 1 -- 172.21.15.81:0/1853904347 shutdown_connections 2024-05-08T23:30:44.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f3f71609700 1 -- 172.21.15.81:0/1853904347 wait complete. 2024-05-08T23:30:44.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f3f71609700 1 Processor -- start 2024-05-08T23:30:44.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fc4617fa700 1 -- 172.21.15.81:0/3544753852 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4580054e0 con 0x7fc46415b1c0 2024-05-08T23:30:44.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fc4617fa700 1 -- 172.21.15.81:0/3544753852 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45800ce30 con 0x7fc46415b1c0 2024-05-08T23:30:44.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fc46ab08700 1 -- 172.21.15.81:0/3544753852 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc464144650 con 0x7fc46415b1c0 2024-05-08T23:30:44.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fc4617fa700 1 -- 172.21.15.81:0/3544753852 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4580038e0 con 0x7fc46415b1c0 2024-05-08T23:30:44.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fc46ab08700 1 -- 172.21.15.81:0/3544753852 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc464144c10 con 0x7fc46415b1c0 2024-05-08T23:30:44.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fc46ab08700 1 -- 172.21.15.81:0/3544753852 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc444000fc0 con 0x7fc46415b1c0 2024-05-08T23:30:44.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7fc4617fa700 1 -- 172.21.15.81:0/3544753852 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc458003a40 con 0x7fc46415b1c0 2024-05-08T23:30:44.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7fc4617fa700 1 --2- 172.21.15.81:0/3544753852 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc450066780 0x7fc450068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7fc4617fa700 1 -- 172.21.15.81:0/3544753852 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc458083450 con 0x7fc46415b1c0 2024-05-08T23:30:44.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7fc4617fa700 1 --2- 172.21.15.81:0/3544753852 >> [v2:172.21.15.81:6816/26093,v1:172.21.15.81:6818/26093] conn(0x7fc45006e9a0 0x7fc450070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7fc4617fa700 1 -- 172.21.15.81:0/3544753852 --> [v2:172.21.15.81:6816/26093,v1:172.21.15.81:6818/26093] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc450071470 con 0x7fc45006e9a0 2024-05-08T23:30:44.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7fc4617fa700 1 -- 172.21.15.81:0/3544753852 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc458085b60 con 0x7fc46415b1c0 2024-05-08T23:30:44.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7fc4688a4700 1 --2- 172.21.15.81:0/3544753852 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc450066780 0x7fc450068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7fc463fff700 1 --2- 172.21.15.81:0/3544753852 >> [v2:172.21.15.81:6816/26093,v1:172.21.15.81:6818/26093] conn(0x7fc45006e9a0 0x7fc450070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7fc463fff700 1 --2- 172.21.15.81:0/3544753852 >> [v2:172.21.15.81:6816/26093,v1:172.21.15.81:6818/26093] conn(0x7fc45006e9a0 0x7fc450070dc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7fc4688a4700 1 --2- 172.21.15.81:0/3544753852 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc450066780 0x7fc450068c40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00b370 tx=0x7fc45c010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7fc4617fa700 1 -- 172.21.15.81:0/3544753852 <== osd.1 v2:172.21.15.81:6816/26093 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc450071470 con 0x7fc45006e9a0 2024-05-08T23:30:44.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f3d9ffff700 1 -- 172.21.15.81:0/3193333368 <== osd.2 v2:172.21.15.81:6817/26092 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f3da4071470 con 0x7f3da406e9a0 2024-05-08T23:30:44.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fbfbeffd700 1 --2- 172.21.15.81:0/3446920639 >> [v2:172.21.15.117:6808/26388,v1:172.21.15.117:6809/26388] conn(0x7fbfb006ea50 0x7fbfb0070e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fbfbeffd700 1 -- 172.21.15.81:0/3446920639 --> [v2:172.21.15.117:6808/26388,v1:172.21.15.117:6809/26388] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfb0071520 con 0x7fbfb006ea50 2024-05-08T23:30:44.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fbfbeffd700 1 -- 172.21.15.81:0/3446920639 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfc8085960 con 0x7fbfd0142c10 2024-05-08T23:30:44.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fbfce59c700 1 --2- 172.21.15.81:0/3446920639 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbfb0066830 0x7fbfb0068cf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4000c00 tx=0x7fbfc4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fbfcdd9b700 1 --2- 172.21.15.81:0/3446920639 >> [v2:172.21.15.117:6808/26388,v1:172.21.15.117:6809/26388] conn(0x7fbfb006ea50 0x7fbfb0070e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.229+0000 7fbfcdd9b700 1 --2- 172.21.15.81:0/3446920639 >> [v2:172.21.15.117:6808/26388,v1:172.21.15.117:6809/26388] conn(0x7fbfb006ea50 0x7fbfb0070e70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7fbfbeffd700 1 -- 172.21.15.81:0/3446920639 <== osd.4 v2:172.21.15.117:6808/26388 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fbfb0071520 con 0x7fbfb006ea50 2024-05-08T23:30:44.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50ceffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50d00cbe70 0x7f50d0158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50ceffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50d00cbe70 0x7f50d0158d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33056/0 (socket says 172.21.15.81:33056) 2024-05-08T23:30:44.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50ceffd700 1 -- 172.21.15.81:0/932689543 learned_addr learned my addr 172.21.15.81:0/932689543 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50cffff700 1 --2- 172.21.15.81:0/932689543 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50d01552f0 0x7f50d0155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50ceffd700 1 -- 172.21.15.81:0/932689543 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50d01552f0 msgr2=0x7f50d0155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50ceffd700 1 --2- 172.21.15.81:0/932689543 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50d01552f0 0x7f50d0155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50ceffd700 1 -- 172.21.15.81:0/932689543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50d006dcd0 msgr2=0x7f50d013a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50ceffd700 1 --2- 172.21.15.81:0/932689543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50d006dcd0 0x7f50d013a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50ceffd700 1 -- 172.21.15.81:0/932689543 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c00094b0 con 0x7f50d00cbe70 2024-05-08T23:30:44.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50ceffd700 1 --2- 172.21.15.81:0/932689543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50d00cbe70 0x7f50d0158d20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009ba0 tx=0x7f50c0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50b7fff700 1 -- 172.21.15.81:0/932689543 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0006ea0 con 0x7f50d00cbe70 2024-05-08T23:30:44.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50b7fff700 1 -- 172.21.15.81:0/932689543 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c0003e20 con 0x7f50d00cbe70 2024-05-08T23:30:44.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50b7fff700 1 -- 172.21.15.81:0/932689543 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0011a60 con 0x7f50d00cbe70 2024-05-08T23:30:44.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50d4818700 1 -- 172.21.15.81:0/932689543 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d01683b0 con 0x7f50d00cbe70 2024-05-08T23:30:44.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50d4818700 1 -- 172.21.15.81:0/932689543 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d0159260 con 0x7f50d00cbe70 2024-05-08T23:30:44.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f50d4818700 1 -- 172.21.15.81:0/932689543 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f50d00cd690 con 0x7f50d00cbe70 2024-05-08T23:30:44.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7fd41d7fa700 1 -- 172.21.15.81:0/3120480336 <== osd.6 v2:172.21.15.117:6824/26390 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd4000714a0 con 0x7fd40006e9d0 2024-05-08T23:30:44.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f986759e700 1 -- 172.21.15.81:0/1150411626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f986808a620 msgr2=0x7f986806e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f986759e700 1 --2- 172.21.15.81:0/1150411626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f986808a620 0x7f986806e380 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f986000ad90 tx=0x7f986000c330 comp rx=0 tx=0).stop 2024-05-08T23:30:44.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f986759e700 1 -- 172.21.15.81:0/1150411626 shutdown_connections 2024-05-08T23:30:44.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f986759e700 1 --2- 172.21.15.81:0/1150411626 >> v2:172.21.15.81:3300/0 conn(0x7f986806caa0 0x7f986806cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f986759e700 1 --2- 172.21.15.81:0/1150411626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f986808a620 0x7f986806e380 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.233+0000 7f986759e700 1 -- 172.21.15.81:0/1150411626 >> 172.21.15.81:0/1150411626 conn(0x7f98680aebd0 msgr2=0x7f98680aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f71609700 1 -- start start 2024-05-08T23:30:44.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f986759e700 1 -- 172.21.15.81:0/1150411626 shutdown_connections 2024-05-08T23:30:44.282 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f71609700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f6c0b1f60 0x7f3f6c1507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.282 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f50b7fff700 1 -- 172.21.15.81:0/932689543 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f50c0011bc0 con 0x7f50d00cbe70 2024-05-08T23:30:44.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f71609700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f6c0cdc20 0x7f3f6c15a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f71609700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f6c0ce000 0x7f3f6c15ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f71609700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c151e90 con 0x7f3f6c0cdc20 2024-05-08T23:30:44.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f71609700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c14be20 con 0x7f3f6c0ce000 2024-05-08T23:30:44.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f71609700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c151280 con 0x7f3f6c0b1f60 2024-05-08T23:30:44.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f6a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f6c0cdc20 0x7f3f6c15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f6affd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f6c0b1f60 0x7f3f6c1507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f6a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f6c0cdc20 0x7f3f6c15a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33066/0 (socket says 172.21.15.81:33066) 2024-05-08T23:30:44.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f6a7fc700 1 -- 172.21.15.81:0/4032232254 learned_addr learned my addr 172.21.15.81:0/4032232254 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f6b7fe700 1 --2- 172.21.15.81:0/4032232254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f6c0ce000 0x7f3f6c15ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f50b7fff700 1 --2- 172.21.15.81:0/932689543 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f50bc066730 0x7f50bc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f50b7fff700 1 -- 172.21.15.81:0/932689543 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50c0085640 con 0x7f50d00cbe70 2024-05-08T23:30:44.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f6affd700 1 -- 172.21.15.81:0/4032232254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f6c0ce000 msgr2=0x7f3f6c15ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f6affd700 1 --2- 172.21.15.81:0/4032232254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f6c0ce000 0x7f3f6c15ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f6affd700 1 -- 172.21.15.81:0/4032232254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f6c0cdc20 msgr2=0x7f3f6c15a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f6affd700 1 --2- 172.21.15.81:0/4032232254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f6c0cdc20 0x7f3f6c15a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f6affd700 1 -- 172.21.15.81:0/4032232254 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c0085f0 con 0x7f3f6c0b1f60 2024-05-08T23:30:44.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f50b7fff700 1 --2- 172.21.15.81:0/932689543 >> [v2:172.21.15.117:6816/26391,v1:172.21.15.117:6817/26391] conn(0x7f50bc06e950 0x7f50bc070d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f50b7fff700 1 -- 172.21.15.81:0/932689543 --> [v2:172.21.15.117:6816/26391,v1:172.21.15.117:6817/26391] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f50bc071420 con 0x7f50bc06e950 2024-05-08T23:30:44.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f50b7fff700 1 -- 172.21.15.81:0/932689543 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50c00859f0 con 0x7f50d00cbe70 2024-05-08T23:30:44.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f50cf7fe700 1 --2- 172.21.15.81:0/932689543 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f50bc066730 0x7f50bc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f3f6affd700 1 --2- 172.21.15.81:0/4032232254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f6c0b1f60 0x7f3f6c1507c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c019000 tx=0x7f3f5c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f50cffff700 1 --2- 172.21.15.81:0/932689543 >> [v2:172.21.15.117:6816/26391,v1:172.21.15.117:6817/26391] conn(0x7f50bc06e950 0x7f50bc070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f50cf7fe700 1 --2- 172.21.15.81:0/932689543 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f50bc066730 0x7f50bc068bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f50c8002210 tx=0x7f50c80017b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.237+0000 7f50cffff700 1 --2- 172.21.15.81:0/932689543 >> [v2:172.21.15.117:6816/26391,v1:172.21.15.117:6817/26391] conn(0x7f50bc06e950 0x7f50bc070d70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f986759e700 1 -- 172.21.15.81:0/1150411626 wait complete. 2024-05-08T23:30:44.287 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f3f537fe700 1 -- 172.21.15.81:0/4032232254 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c008db0 con 0x7f3f6c0b1f60 2024-05-08T23:30:44.287 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f3f537fe700 1 -- 172.21.15.81:0/4032232254 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c0036c0 con 0x7f3f6c0b1f60 2024-05-08T23:30:44.287 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f3f71609700 1 -- 172.21.15.81:0/4032232254 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6c151560 con 0x7f3f6c0b1f60 2024-05-08T23:30:44.287 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f3f71609700 1 -- 172.21.15.81:0/4032232254 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6c151b20 con 0x7f3f6c0b1f60 2024-05-08T23:30:44.287 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f50b7fff700 1 -- 172.21.15.81:0/932689543 <== osd.5 v2:172.21.15.117:6816/26391 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f50bc071420 con 0x7f50bc06e950 2024-05-08T23:30:44.288 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f3f537fe700 1 -- 172.21.15.81:0/4032232254 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c003a50 con 0x7f3f6c0b1f60 2024-05-08T23:30:44.288 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f3f537fe700 1 -- 172.21.15.81:0/4032232254 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3f5c003bb0 con 0x7f3f6c0b1f60 2024-05-08T23:30:44.288 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f986759e700 1 Processor -- start 2024-05-08T23:30:44.288 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f986759e700 1 -- start start 2024-05-08T23:30:44.289 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f3f537fe700 1 --2- 172.21.15.81:0/4032232254 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f580667b0 0x7f3f58068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.289 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f3f537fe700 1 -- 172.21.15.81:0/4032232254 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f5c0851d0 con 0x7f3f6c0b1f60 2024-05-08T23:30:44.289 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f986759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f986806caa0 0x7f986815f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.289 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f986759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f986808a620 0x7f9868089e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.289 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f986759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98680b3c40 0x7f98680cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.289 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f986759e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9868164420 con 0x7f986808a620 2024-05-08T23:30:44.289 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f986759e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986815fdd0 con 0x7f98680b3c40 2024-05-08T23:30:44.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f986759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986808a3a0 con 0x7f986806caa0 2024-05-08T23:30:44.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f3f6a7fc700 1 --2- 172.21.15.81:0/4032232254 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f580667b0 0x7f3f58068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f9865d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f986808a620 0x7f9868089e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f9866d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98680b3c40 0x7f98680cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f986659c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f986806caa0 0x7f986815f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f9865d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f986808a620 0x7f9868089e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33078/0 (socket says 172.21.15.81:33078) 2024-05-08T23:30:44.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.241+0000 7f9865d9b700 1 -- 172.21.15.81:0/1165257819 learned_addr learned my addr 172.21.15.81:0/1165257819 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.245+0000 7f9865d9b700 1 -- 172.21.15.81:0/1165257819 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f986806caa0 msgr2=0x7f986815f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.245+0000 7f9865d9b700 1 --2- 172.21.15.81:0/1165257819 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f986806caa0 0x7f986815f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.245+0000 7f9865d9b700 1 -- 172.21.15.81:0/1165257819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98680b3c40 msgr2=0x7f98680cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.245+0000 7f9865d9b700 1 --2- 172.21.15.81:0/1165257819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98680b3c40 0x7f98680cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.245+0000 7f9865d9b700 1 -- 172.21.15.81:0/1165257819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986000aa70 con 0x7f986808a620 2024-05-08T23:30:44.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.245+0000 7f9865d9b700 1 --2- 172.21.15.81:0/1165257819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f986808a620 0x7f9868089e60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9860000f80 tx=0x7f9860004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.245+0000 7f3f6a7fc700 1 --2- 172.21.15.81:0/4032232254 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f580667b0 0x7f3f58068c70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c072550 tx=0x7f3f60009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.249+0000 7f3f517fa700 1 --2- 172.21.15.81:0/4032232254 >> [v2:172.21.15.81:6800/26091,v1:172.21.15.81:6801/26091] conn(0x7f3f6c150d00 0x7f3f6c060db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.249+0000 7f3f517fa700 1 -- 172.21.15.81:0/4032232254 --> [v2:172.21.15.81:6800/26091,v1:172.21.15.81:6801/26091] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3f6c06d390 con 0x7f3f6c150d00 2024-05-08T23:30:44.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.249+0000 7f3f6b7fe700 1 --2- 172.21.15.81:0/4032232254 >> [v2:172.21.15.81:6800/26091,v1:172.21.15.81:6801/26091] conn(0x7f3f6c150d00 0x7f3f6c060db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.249+0000 7f3f6b7fe700 1 --2- 172.21.15.81:0/4032232254 >> [v2:172.21.15.81:6800/26091,v1:172.21.15.81:6801/26091] conn(0x7f3f6c150d00 0x7f3f6c060db0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.249+0000 7fad50b73700 1 -- 172.21.15.81:0/1013428084 --> [v2:172.21.15.81:6805/26094,v1:172.21.15.81:6807/26094] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad2c002d60 con 0x7fad3806e980 2024-05-08T23:30:44.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.249+0000 7f3f537fe700 1 -- 172.21.15.81:0/4032232254 <== osd.0 v2:172.21.15.81:6800/26091 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f3f6400f040 con 0x7f3f6c150d00 2024-05-08T23:30:44.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.249+0000 7fad33fff700 1 -- 172.21.15.81:0/1013428084 <== osd.3 v2:172.21.15.81:6805/26094 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fad2c002d60 con 0x7fad3806e980 2024-05-08T23:30:44.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fad50b73700 1 -- 172.21.15.81:0/1013428084 >> [v2:172.21.15.81:6805/26094,v1:172.21.15.81:6807/26094] conn(0x7fad3806e980 msgr2=0x7fad38070da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.294 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fad50b73700 1 --2- 172.21.15.81:0/1013428084 >> [v2:172.21.15.81:6805/26094,v1:172.21.15.81:6807/26094] conn(0x7fad3806e980 0x7fad38070da0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.294 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7f9856ffd700 1 -- 172.21.15.81:0/1165257819 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9860004880 con 0x7f986808a620 2024-05-08T23:30:44.294 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7f9856ffd700 1 -- 172.21.15.81:0/1165257819 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9860013430 con 0x7f986808a620 2024-05-08T23:30:44.294 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7f9856ffd700 1 -- 172.21.15.81:0/1165257819 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98600137e0 con 0x7f986808a620 2024-05-08T23:30:44.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fad50b73700 1 -- 172.21.15.81:0/1013428084 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fad38066760 msgr2=0x7fad38068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fad50b73700 1 --2- 172.21.15.81:0/1013428084 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fad38066760 0x7fad38068c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fad44001dc0 tx=0x7fad44001d50 comp rx=0 tx=0).stop 2024-05-08T23:30:44.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fad50b73700 1 -- 172.21.15.81:0/1013428084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad4c0b1f60 msgr2=0x7fad4c142f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fad50b73700 1 --2- 172.21.15.81:0/1013428084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad4c0b1f60 0x7fad4c142f80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fad3c000c00 tx=0x7fad3c004770 comp rx=0 tx=0).stop 2024-05-08T23:30:44.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fad4bfff700 1 -- 172.21.15.81:0/1013428084 reap_dead start 2024-05-08T23:30:44.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fd425c35700 1 -- 172.21.15.81:0/3120480336 --> [v2:172.21.15.117:6824/26390,v1:172.21.15.117:6825/26390] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd42006d340 con 0x7fd40006e9d0 2024-05-08T23:30:44.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fad50b73700 1 -- 172.21.15.81:0/1013428084 shutdown_connections 2024-05-08T23:30:44.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fad50b73700 1 -- 172.21.15.81:0/1013428084 >> 172.21.15.81:0/1013428084 conn(0x7fad4c0aebd0 msgr2=0x7fad4c0b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7f986759e700 1 -- 172.21.15.81:0/1165257819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98681638a0 con 0x7f986808a620 2024-05-08T23:30:44.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7f986759e700 1 -- 172.21.15.81:0/1165257819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9868163e60 con 0x7f986808a620 2024-05-08T23:30:44.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fd41d7fa700 1 -- 172.21.15.81:0/3120480336 <== osd.6 v2:172.21.15.117:6824/26390 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd410006c80 con 0x7fd40006e9d0 2024-05-08T23:30:44.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fad50b73700 1 -- 172.21.15.81:0/1013428084 shutdown_connections 2024-05-08T23:30:44.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fad50b73700 1 -- 172.21.15.81:0/1013428084 wait complete. 2024-05-08T23:30:44.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7f3dbc83a700 1 -- 172.21.15.81:0/3193333368 --> [v2:172.21.15.81:6817/26092,v1:172.21.15.81:6819/26092] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3db815b2c0 con 0x7f3da406e9a0 2024-05-08T23:30:44.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7f986759e700 1 -- 172.21.15.81:0/1165257819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f986806e3f0 con 0x7f986808a620 2024-05-08T23:30:44.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fd425c35700 1 -- 172.21.15.81:0/3120480336 >> [v2:172.21.15.117:6824/26390,v1:172.21.15.117:6825/26390] conn(0x7fd40006e9d0 msgr2=0x7fd400070df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fd425c35700 1 --2- 172.21.15.81:0/3120480336 >> [v2:172.21.15.117:6824/26390,v1:172.21.15.117:6825/26390] conn(0x7fd40006e9d0 0x7fd400070df0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7f3d9ffff700 1 -- 172.21.15.81:0/3193333368 <== osd.2 v2:172.21.15.81:6817/26092 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3da8022040 con 0x7f3da406e9a0 2024-05-08T23:30:44.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fd425c35700 1 -- 172.21.15.81:0/3120480336 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd4000667b0 msgr2=0x7fd400068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fd425c35700 1 --2- 172.21.15.81:0/3120480336 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd4000667b0 0x7fd400068c70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd4180012e0 tx=0x7fd418001860 comp rx=0 tx=0).stop 2024-05-08T23:30:44.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fd425c35700 1 -- 172.21.15.81:0/3120480336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4200ac8e0 msgr2=0x7fd42013af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fd425c35700 1 --2- 172.21.15.81:0/3120480336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4200ac8e0 0x7fd42013af60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd40800e7c0 tx=0x7fd40800ead0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fd425434700 1 -- 172.21.15.81:0/3120480336 reap_dead start 2024-05-08T23:30:44.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fd425c35700 1 -- 172.21.15.81:0/3120480336 shutdown_connections 2024-05-08T23:30:44.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fd425c35700 1 -- 172.21.15.81:0/3120480336 >> 172.21.15.81:0/3120480336 conn(0x7fd4200aebd0 msgr2=0x7fd420070780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fd425c35700 1 -- 172.21.15.81:0/3120480336 shutdown_connections 2024-05-08T23:30:44.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7f9856ffd700 1 -- 172.21.15.81:0/1165257819 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9860013940 con 0x7f986808a620 2024-05-08T23:30:44.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.257+0000 7fd425c35700 1 -- 172.21.15.81:0/3120480336 wait complete. 2024-05-08T23:30:44.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7fbfcf59e700 1 -- 172.21.15.81:0/3446920639 --> [v2:172.21.15.117:6808/26388,v1:172.21.15.117:6809/26388] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbfd00b4b20 con 0x7fbfb006ea50 2024-05-08T23:30:44.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7f9856ffd700 1 --2- 172.21.15.81:0/1165257819 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9848066780 0x7f9848068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7f9856ffd700 1 -- 172.21.15.81:0/1165257819 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9860086360 con 0x7f986808a620 2024-05-08T23:30:44.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7f9856ffd700 1 --2- 172.21.15.81:0/1165257819 >> [v2:172.21.15.117:6800/26389,v1:172.21.15.117:6801/26389] conn(0x7f984806e9a0 0x7f9848070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7f9856ffd700 1 -- 172.21.15.81:0/1165257819 --> [v2:172.21.15.117:6800/26389,v1:172.21.15.117:6801/26389] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9848071470 con 0x7f984806e9a0 2024-05-08T23:30:44.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7f9856ffd700 1 -- 172.21.15.81:0/1165257819 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9860086800 con 0x7f986808a620 2024-05-08T23:30:44.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7fbfbeffd700 1 -- 172.21.15.81:0/3446920639 <== osd.4 v2:172.21.15.117:6808/26388 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbfc0005090 con 0x7fbfb006ea50 2024-05-08T23:30:44.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7f9866d9d700 1 --2- 172.21.15.81:0/1165257819 >> [v2:172.21.15.117:6800/26389,v1:172.21.15.117:6801/26389] conn(0x7f984806e9a0 0x7f9848070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7fbfbcff9700 1 -- 172.21.15.81:0/3446920639 >> [v2:172.21.15.117:6808/26388,v1:172.21.15.117:6809/26388] conn(0x7fbfb006ea50 msgr2=0x7fbfb0070e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7fbfbcff9700 1 --2- 172.21.15.81:0/3446920639 >> [v2:172.21.15.117:6808/26388,v1:172.21.15.117:6809/26388] conn(0x7fbfb006ea50 0x7fbfb0070e70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7fbfbcff9700 1 -- 172.21.15.81:0/3446920639 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbfb0066830 msgr2=0x7fbfb0068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7fbfbcff9700 1 --2- 172.21.15.81:0/3446920639 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbfb0066830 0x7fbfb0068cf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4000c00 tx=0x7fbfc4008030 comp rx=0 tx=0).stop 2024-05-08T23:30:44.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7fbfbcff9700 1 -- 172.21.15.81:0/3446920639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0142c10 msgr2=0x7fbfd01390e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7fbfbcff9700 1 --2- 172.21.15.81:0/3446920639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0142c10 0x7fbfd01390e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8007e60 tx=0x7fbfc800e3c0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7f9866d9d700 1 --2- 172.21.15.81:0/1165257819 >> [v2:172.21.15.117:6800/26389,v1:172.21.15.117:6801/26389] conn(0x7f984806e9a0 0x7f9848070dc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7fbfced9d700 1 -- 172.21.15.81:0/3446920639 reap_dead start 2024-05-08T23:30:44.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7fbfbcff9700 1 -- 172.21.15.81:0/3446920639 shutdown_connections 2024-05-08T23:30:44.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7fbfbcff9700 1 -- 172.21.15.81:0/3446920639 >> 172.21.15.81:0/3446920639 conn(0x7fbfd00aebd0 msgr2=0x7fbfd006d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7fbfbcff9700 1 -- 172.21.15.81:0/3446920639 shutdown_connections 2024-05-08T23:30:44.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7fbfbcff9700 1 -- 172.21.15.81:0/3446920639 wait complete. 2024-05-08T23:30:44.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.261+0000 7f9856ffd700 1 -- 172.21.15.81:0/1165257819 <== osd.7 v2:172.21.15.117:6800/26389 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f9848071470 con 0x7f984806e9a0 2024-05-08T23:30:44.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.265+0000 7f986659c700 1 --2- 172.21.15.81:0/1165257819 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9848066780 0x7f9848068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.269+0000 7f986659c700 1 --2- 172.21.15.81:0/1165257819 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9848066780 0x7f9848068c40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9858004950 tx=0x7f9858008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.269+0000 7f3dbc83a700 1 -- 172.21.15.81:0/3193333368 >> [v2:172.21.15.81:6817/26092,v1:172.21.15.81:6819/26092] conn(0x7f3da406e9a0 msgr2=0x7f3da4070dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.269+0000 7f3dbc83a700 1 --2- 172.21.15.81:0/3193333368 >> [v2:172.21.15.81:6817/26092,v1:172.21.15.81:6819/26092] conn(0x7f3da406e9a0 0x7f3da4070dc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.269+0000 7f3dbc83a700 1 -- 172.21.15.81:0/3193333368 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3da4066780 msgr2=0x7f3da4068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.269+0000 7f3dbc83a700 1 --2- 172.21.15.81:0/3193333368 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3da4066780 0x7f3da4068c40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3db806dfc0 tx=0x7f3dac008040 comp rx=0 tx=0).stop 2024-05-08T23:30:44.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.269+0000 7f3dbc83a700 1 -- 172.21.15.81:0/3193333368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3db8071ab0 msgr2=0x7f3db813e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.269+0000 7f3dbc83a700 1 --2- 172.21.15.81:0/3193333368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3db8071ab0 0x7f3db813e4b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3db000caa0 tx=0x7f3db000cad0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.269+0000 7f3db7fff700 1 -- 172.21.15.81:0/3193333368 reap_dead start 2024-05-08T23:30:44.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.269+0000 7f3dbc83a700 1 -- 172.21.15.81:0/3193333368 shutdown_connections 2024-05-08T23:30:44.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.269+0000 7f3dbc83a700 1 -- 172.21.15.81:0/3193333368 >> 172.21.15.81:0/3193333368 conn(0x7f3db806c330 msgr2=0x7f3db813c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.269+0000 7f3dbc83a700 1 -- 172.21.15.81:0/3193333368 shutdown_connections 2024-05-08T23:30:44.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.269+0000 7f3dbc83a700 1 -- 172.21.15.81:0/3193333368 wait complete. 2024-05-08T23:30:44.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.269+0000 7fc46ab08700 1 -- 172.21.15.81:0/3544753852 --> [v2:172.21.15.81:6816/26093,v1:172.21.15.81:6818/26093] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc444002d60 con 0x7fc45006e9a0 2024-05-08T23:30:44.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.269+0000 7fc4617fa700 1 -- 172.21.15.81:0/3544753852 <== osd.1 v2:172.21.15.81:6816/26093 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc444002d60 con 0x7fc45006e9a0 2024-05-08T23:30:44.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.273+0000 7f50d4818700 1 -- 172.21.15.81:0/932689543 --> [v2:172.21.15.117:6816/26391,v1:172.21.15.117:6817/26391] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f50d00cdfa0 con 0x7f50bc06e950 2024-05-08T23:30:44.309 INFO:teuthology.orchestra.run.smithi081.stdout:64424509446 2024-05-08T23:30:44.309 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T23:30:44.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.273+0000 7f50b7fff700 1 -- 172.21.15.81:0/932689543 <== osd.5 v2:172.21.15.117:6816/26391 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f50c4004b40 con 0x7f50bc06e950 2024-05-08T23:30:44.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.273+0000 7f50b5ffb700 1 -- 172.21.15.81:0/932689543 >> [v2:172.21.15.117:6816/26391,v1:172.21.15.117:6817/26391] conn(0x7f50bc06e950 msgr2=0x7f50bc070d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.273+0000 7f50b5ffb700 1 --2- 172.21.15.81:0/932689543 >> [v2:172.21.15.117:6816/26391,v1:172.21.15.117:6817/26391] conn(0x7f50bc06e950 0x7f50bc070d70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.273+0000 7f50b5ffb700 1 -- 172.21.15.81:0/932689543 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f50bc066730 msgr2=0x7f50bc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.273+0000 7f50b5ffb700 1 --2- 172.21.15.81:0/932689543 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f50bc066730 0x7f50bc068bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f50c8002210 tx=0x7f50c80017b0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.273+0000 7f50b5ffb700 1 -- 172.21.15.81:0/932689543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50d00cbe70 msgr2=0x7f50d0158d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.273+0000 7f50b5ffb700 1 --2- 172.21.15.81:0/932689543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50d00cbe70 0x7f50d0158d20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009ba0 tx=0x7f50c0004770 comp rx=0 tx=0).stop 2024-05-08T23:30:44.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.273+0000 7f50cffff700 1 -- 172.21.15.81:0/932689543 reap_dead start 2024-05-08T23:30:44.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.273+0000 7f50b5ffb700 1 -- 172.21.15.81:0/932689543 shutdown_connections 2024-05-08T23:30:44.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.273+0000 7f50b5ffb700 1 -- 172.21.15.81:0/932689543 >> 172.21.15.81:0/932689543 conn(0x7f50d00aebd0 msgr2=0x7f50d015c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.273+0000 7f50b5ffb700 1 -- 172.21.15.81:0/932689543 shutdown_connections 2024-05-08T23:30:44.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.273+0000 7f50b5ffb700 1 -- 172.21.15.81:0/932689543 wait complete. 2024-05-08T23:30:44.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.277+0000 7f3f517fa700 1 -- 172.21.15.81:0/4032232254 --> [v2:172.21.15.81:6800/26091,v1:172.21.15.81:6801/26091] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3f6c06cd40 con 0x7f3f6c150d00 2024-05-08T23:30:44.312 INFO:teuthology.orchestra.run.smithi081.stdout:64424509448 2024-05-08T23:30:44.313 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T23:30:44.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.277+0000 7fc46ab08700 1 -- 172.21.15.81:0/3544753852 >> [v2:172.21.15.81:6816/26093,v1:172.21.15.81:6818/26093] conn(0x7fc45006e9a0 msgr2=0x7fc450070dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.277+0000 7fc46ab08700 1 --2- 172.21.15.81:0/3544753852 >> [v2:172.21.15.81:6816/26093,v1:172.21.15.81:6818/26093] conn(0x7fc45006e9a0 0x7fc450070dc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.277+0000 7fc46ab08700 1 -- 172.21.15.81:0/3544753852 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc450066780 msgr2=0x7fc450068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.277+0000 7fc46ab08700 1 --2- 172.21.15.81:0/3544753852 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc450066780 0x7fc450068c40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00b370 tx=0x7fc45c010000 comp rx=0 tx=0).stop 2024-05-08T23:30:44.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.277+0000 7fc46ab08700 1 -- 172.21.15.81:0/3544753852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc46415b1c0 msgr2=0x7fc464143fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.277+0000 7fc46ab08700 1 --2- 172.21.15.81:0/3544753852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc46415b1c0 0x7fc464143fd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc45800c410 tx=0x7fc45800c720 comp rx=0 tx=0).stop 2024-05-08T23:30:44.314 INFO:teuthology.orchestra.run.smithi081.stdout:64424509446 2024-05-08T23:30:44.314 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T23:30:44.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.277+0000 7fc4690a5700 1 -- 172.21.15.81:0/3544753852 reap_dead start 2024-05-08T23:30:44.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.277+0000 7fc46ab08700 1 -- 172.21.15.81:0/3544753852 shutdown_connections 2024-05-08T23:30:44.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.277+0000 7fc46ab08700 1 -- 172.21.15.81:0/3544753852 >> 172.21.15.81:0/3544753852 conn(0x7fc4640b0cd0 msgr2=0x7fc4640aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.277+0000 7fc46ab08700 1 -- 172.21.15.81:0/3544753852 shutdown_connections 2024-05-08T23:30:44.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.277+0000 7fc46ab08700 1 -- 172.21.15.81:0/3544753852 wait complete. 2024-05-08T23:30:44.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f986759e700 1 -- 172.21.15.81:0/1165257819 --> [v2:172.21.15.117:6800/26389,v1:172.21.15.117:6801/26389] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f98680ac8e0 con 0x7f984806e9a0 2024-05-08T23:30:44.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f3f537fe700 1 -- 172.21.15.81:0/4032232254 <== osd.0 v2:172.21.15.81:6800/26091 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3f64011480 con 0x7f3f6c150d00 2024-05-08T23:30:44.317 INFO:teuthology.orchestra.run.smithi081.stdout:64424509446 2024-05-08T23:30:44.317 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T23:30:44.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f9856ffd700 1 -- 172.21.15.81:0/1165257819 <== osd.7 v2:172.21.15.117:6800/26389 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f98680ac8e0 con 0x7f984806e9a0 2024-05-08T23:30:44.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f3f517fa700 1 -- 172.21.15.81:0/4032232254 >> [v2:172.21.15.81:6800/26091,v1:172.21.15.81:6801/26091] conn(0x7f3f6c150d00 msgr2=0x7f3f6c060db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f3f517fa700 1 --2- 172.21.15.81:0/4032232254 >> [v2:172.21.15.81:6800/26091,v1:172.21.15.81:6801/26091] conn(0x7f3f6c150d00 0x7f3f6c060db0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f3f517fa700 1 -- 172.21.15.81:0/4032232254 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f580667b0 msgr2=0x7f3f58068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f3f517fa700 1 --2- 172.21.15.81:0/4032232254 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f580667b0 0x7f3f58068c70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c072550 tx=0x7f3f60009040 comp rx=0 tx=0).stop 2024-05-08T23:30:44.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f3f517fa700 1 -- 172.21.15.81:0/4032232254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f6c0b1f60 msgr2=0x7f3f6c1507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f3f517fa700 1 --2- 172.21.15.81:0/4032232254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f6c0b1f60 0x7f3f6c1507c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c019000 tx=0x7f3f5c004770 comp rx=0 tx=0).stop 2024-05-08T23:30:44.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f3f6b7fe700 1 -- 172.21.15.81:0/4032232254 reap_dead start 2024-05-08T23:30:44.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f3f517fa700 1 -- 172.21.15.81:0/4032232254 shutdown_connections 2024-05-08T23:30:44.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f3f517fa700 1 -- 172.21.15.81:0/4032232254 >> 172.21.15.81:0/4032232254 conn(0x7f3f6c0aebd0 msgr2=0x7f3f6c0cc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f3f517fa700 1 -- 172.21.15.81:0/4032232254 shutdown_connections 2024-05-08T23:30:44.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f3f517fa700 1 -- 172.21.15.81:0/4032232254 wait complete. 2024-05-08T23:30:44.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f986759e700 1 -- 172.21.15.81:0/1165257819 >> [v2:172.21.15.117:6800/26389,v1:172.21.15.117:6801/26389] conn(0x7f984806e9a0 msgr2=0x7f9848070dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f986759e700 1 --2- 172.21.15.81:0/1165257819 >> [v2:172.21.15.117:6800/26389,v1:172.21.15.117:6801/26389] conn(0x7f984806e9a0 0x7f9848070dc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f986759e700 1 -- 172.21.15.81:0/1165257819 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9848066780 msgr2=0x7f9848068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f986759e700 1 --2- 172.21.15.81:0/1165257819 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9848066780 0x7f9848068c40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9858004950 tx=0x7f9858008040 comp rx=0 tx=0).stop 2024-05-08T23:30:44.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f986759e700 1 -- 172.21.15.81:0/1165257819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f986808a620 msgr2=0x7f9868089e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f986759e700 1 --2- 172.21.15.81:0/1165257819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f986808a620 0x7f9868089e60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9860000f80 tx=0x7f9860004af0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f9866d9d700 1 -- 172.21.15.81:0/1165257819 reap_dead start 2024-05-08T23:30:44.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f986759e700 1 -- 172.21.15.81:0/1165257819 shutdown_connections 2024-05-08T23:30:44.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f986759e700 1 -- 172.21.15.81:0/1165257819 >> 172.21.15.81:0/1165257819 conn(0x7f98680aebd0 msgr2=0x7f98680ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.285+0000 7f986759e700 1 -- 172.21.15.81:0/1165257819 shutdown_connections 2024-05-08T23:30:44.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.289+0000 7f986759e700 1 -- 172.21.15.81:0/1165257819 wait complete. 2024-05-08T23:30:44.321 INFO:teuthology.orchestra.run.smithi081.stdout:64424509448 2024-05-08T23:30:44.321 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T23:30:44.322 INFO:teuthology.orchestra.run.smithi081.stdout:64424509446 2024-05-08T23:30:44.323 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T23:30:44.323 INFO:teuthology.orchestra.run.smithi081.stdout:60129542150 2024-05-08T23:30:44.323 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T23:30:44.324 INFO:teuthology.orchestra.run.smithi081.stdout:60129542150 2024-05-08T23:30:44.325 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T23:30:44.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f74f8d00700 1 Processor -- start 2024-05-08T23:30:44.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f74f8d00700 1 -- start start 2024-05-08T23:30:44.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f74f8d00700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74f40b4310 0x7f74f4165ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7faaa0d3a700 1 Processor -- start 2024-05-08T23:30:44.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7faaa0d3a700 1 -- start start 2024-05-08T23:30:44.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f74f8d00700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f74f406e3a0 0x7f74f406c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f74f8d00700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f4162700 con 0x7f74f40cb3e0 2024-05-08T23:30:44.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f74f8d00700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f4161dd0 con 0x7f74f406e3a0 2024-05-08T23:30:44.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f74f8d00700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f41620b0 con 0x7f74f40b4310 2024-05-08T23:30:44.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f74f37fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f74f40cb3e0 0x7f74f40b3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55070/0 (socket says 172.21.15.81:55070) 2024-05-08T23:30:44.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f846d6f2700 1 Processor -- start 2024-05-08T23:30:44.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f846d6f2700 1 -- start start 2024-05-08T23:30:44.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f846d6f2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84680b3c40 0x7f846806e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f846d6f2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f846808a620 0x7f84680b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f846d6f2700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846806c330 con 0x7f846816d820 2024-05-08T23:30:44.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f74f37fe700 1 -- 172.21.15.81:0/2370269837 learned_addr learned my addr 172.21.15.81:0/2370269837 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:44.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f74f2ffd700 1 --2- 172.21.15.81:0/2370269837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74f40b4310 0x7f74f4165ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f74f3fff700 1 --2- 172.21.15.81:0/2370269837 >> v2:172.21.15.117:3300/0 conn(0x7f74f406e3a0 0x7f74f406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f846d6f2700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846806c610 con 0x7f846808a620 2024-05-08T23:30:44.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f846d6f2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84680cc530 con 0x7f84680b3c40 2024-05-08T23:30:44.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f84677fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f846808a620 0x7f84680b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f84677fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f846808a620 0x7f84680b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33086/0 (socket says 172.21.15.81:33086) 2024-05-08T23:30:44.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f84677fe700 1 -- 172.21.15.81:0/3798004156 learned_addr learned my addr 172.21.15.81:0/3798004156 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.501+0000 7f8467fff700 1 --2- 172.21.15.81:0/3798004156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84680b3c40 0x7f846806e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f84677fe700 1 -- 172.21.15.81:0/3798004156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84680b3c40 msgr2=0x7f846806e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f84677fe700 1 --2- 172.21.15.81:0/3798004156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84680b3c40 0x7f846806e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f84677fe700 1 -- 172.21.15.81:0/3798004156 >> v1:172.21.15.117:6789/0 conn(0x7f846816d820 legacy=0x7f846816dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f84677fe700 1 -- 172.21.15.81:0/3798004156 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84680b4560 con 0x7f846808a620 2024-05-08T23:30:44.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f27fc700 1 -- 172.21.15.81:0/2370269837 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653464199 0 0) 0x7f74f4162700 con 0x7f74f40cb3e0 2024-05-08T23:30:44.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f27fc700 1 -- 172.21.15.81:0/2370269837 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74dc0035d0 con 0x7f74f40cb3e0 2024-05-08T23:30:44.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f3fff700 1 -- 172.21.15.81:0/2370269837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74f40b4310 msgr2=0x7f74f4165ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f3fff700 1 --2- 172.21.15.81:0/2370269837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74f40b4310 0x7f74f4165ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f3fff700 1 -- 172.21.15.81:0/2370269837 >> v1:172.21.15.81:6789/0 conn(0x7f74f40cb3e0 legacy=0x7f74f40b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f3fff700 1 -- 172.21.15.81:0/2370269837 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f4162390 con 0x7f74f406e3a0 2024-05-08T23:30:44.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faaa0d3a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faa9c0cb3e0 0x7faa9c165ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faaa0d3a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa9c06e3a0 0x7faa9c0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faaa0d3a700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa9c0b4590 con 0x7faa9c0b4140 2024-05-08T23:30:44.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faaa0d3a700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa9c162700 con 0x7faa9c0cb3e0 2024-05-08T23:30:44.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faaa0d3a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c161dd0 con 0x7faa9c06e3a0 2024-05-08T23:30:44.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f84677fe700 1 --2- 172.21.15.81:0/3798004156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f846808a620 0x7f84680b4020 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8460008ec0 tx=0x7f846000b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d1f49c2cdfdf5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faa9affd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa9c06e3a0 0x7faa9c0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faa9affd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa9c06e3a0 0x7faa9c0b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:53216/0 (socket says 172.21.15.81:53216) 2024-05-08T23:30:44.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faa9affd700 1 -- 172.21.15.81:0/2395049805 learned_addr learned my addr 172.21.15.81:0/2395049805 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f3fff700 1 --2- 172.21.15.81:0/2370269837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74f406e3a0 0x7f74f406c330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009850 tx=0x7f74e4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8460e71fda4a40ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faa9b7fe700 1 --2- 172.21.15.81:0/2395049805 >> v2:172.21.15.117:3300/0 conn(0x7faa9c0cb3e0 0x7faa9c165ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc14d64700 1 Processor -- start 2024-05-08T23:30:44.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc14d64700 1 -- start start 2024-05-08T23:30:44.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f27fc700 1 -- 172.21.15.81:0/2370269837 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4011830 con 0x7f74f406e3a0 2024-05-08T23:30:44.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f8466ffd700 1 -- 172.21.15.81:0/3798004156 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846000bea0 con 0x7f846808a620 2024-05-08T23:30:44.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f8466ffd700 1 -- 172.21.15.81:0/3798004156 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84600203b0 con 0x7f846808a620 2024-05-08T23:30:44.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f8466ffd700 1 -- 172.21.15.81:0/3798004156 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460020740 con 0x7f846808a620 2024-05-08T23:30:44.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f27fc700 1 -- 172.21.15.81:0/2370269837 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e4011990 con 0x7f74f406e3a0 2024-05-08T23:30:44.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f27fc700 1 -- 172.21.15.81:0/2370269837 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4011d00 con 0x7f74f406e3a0 2024-05-08T23:30:44.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f9377e80700 1 Processor -- start 2024-05-08T23:30:44.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faa9affd700 1 -- 172.21.15.81:0/2395049805 >> v2:172.21.15.117:3300/0 conn(0x7faa9c0cb3e0 msgr2=0x7faa9c165ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faa9affd700 1 --2- 172.21.15.81:0/2395049805 >> v2:172.21.15.117:3300/0 conn(0x7faa9c0cb3e0 0x7faa9c165ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faa9affd700 1 -- 172.21.15.81:0/2395049805 >> v1:172.21.15.117:6789/0 conn(0x7faa9c0b4140 legacy=0x7faa9c0b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faa9affd700 1 -- 172.21.15.81:0/2395049805 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c1620b0 con 0x7faa9c06e3a0 2024-05-08T23:30:44.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faa9affd700 1 --2- 172.21.15.81:0/2395049805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa9c06e3a0 0x7faa9c0b3c00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009910 tx=0x7faa8c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a652e1ec7da4339 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc14d64700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efc1006e3a0 0x7efc10165ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc14d64700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc10162700 con 0x7efc100cb3e0 2024-05-08T23:30:44.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc14d64700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc10161dd0 con 0x7efc100b4310 2024-05-08T23:30:44.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc14d64700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc101620b0 con 0x7efc1006e3a0 2024-05-08T23:30:44.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0f7fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7efc100cb3e0 0x7efc100b3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55080/0 (socket says 172.21.15.81:55080) 2024-05-08T23:30:44.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0f7fe700 1 -- 172.21.15.81:0/3942131103 learned_addr learned my addr 172.21.15.81:0/3942131103 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:44.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f846d6f2700 1 -- 172.21.15.81:0/3798004156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f846808a620 msgr2=0x7f84680b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f846d6f2700 1 --2- 172.21.15.81:0/3798004156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f846808a620 0x7f84680b4020 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8460008ec0 tx=0x7f846000b400 comp rx=0 tx=0).stop 2024-05-08T23:30:44.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faa9a7fc700 1 -- 172.21.15.81:0/2395049805 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c00ba60 con 0x7faa9c06e3a0 2024-05-08T23:30:44.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f9377e80700 1 -- start start 2024-05-08T23:30:44.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f35e9a18700 1 Processor -- start 2024-05-08T23:30:44.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f35e9a18700 1 -- start start 2024-05-08T23:30:44.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f8d00700 1 -- 172.21.15.81:0/2370269837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74f406e3a0 msgr2=0x7f74f406c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f8d00700 1 --2- 172.21.15.81:0/2370269837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74f406e3a0 0x7f74f406c330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009850 tx=0x7f74e4009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:44.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f8d00700 1 -- 172.21.15.81:0/2370269837 shutdown_connections 2024-05-08T23:30:44.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f8d00700 1 --2- 172.21.15.81:0/2370269837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74f406e3a0 0x7f74f406c330 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f8d00700 1 --2- 172.21.15.81:0/2370269837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74f40b4310 0x7f74f4165ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f8d00700 1 -- 172.21.15.81:0/2370269837 >> 172.21.15.81:0/2370269837 conn(0x7f74f40aebd0 msgr2=0x7f74f40aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0e7fc700 1 -- 172.21.15.81:0/3942131103 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780372170 0 0) 0x7efc10162700 con 0x7efc100cb3e0 2024-05-08T23:30:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0e7fc700 1 -- 172.21.15.81:0/3942131103 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbf80035d0 con 0x7efc100cb3e0 2024-05-08T23:30:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0e7fc700 1 -- 172.21.15.81:0/3942131103 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1869811961 0 0) 0x7efbf80035d0 con 0x7efc100cb3e0 2024-05-08T23:30:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0e7fc700 1 -- 172.21.15.81:0/3942131103 >> v2:172.21.15.117:3300/0 conn(0x7efc1006e3a0 msgr2=0x7efc10165ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0e7fc700 1 --2- 172.21.15.81:0/3942131103 >> v2:172.21.15.117:3300/0 conn(0x7efc1006e3a0 0x7efc10165ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0e7fc700 1 -- 172.21.15.81:0/3942131103 >> v1:172.21.15.117:6789/0 conn(0x7efc100b4310 legacy=0x7efc1006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0e7fc700 1 -- 172.21.15.81:0/3942131103 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc10162390 con 0x7efc100cb3e0 2024-05-08T23:30:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0e7fc700 1 -- 172.21.15.81:0/3942131103 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7efc00001670 con 0x7efc100cb3e0 2024-05-08T23:30:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0e7fc700 1 -- 172.21.15.81:0/3942131103 >> v1:172.21.15.81:6789/0 conn(0x7efc100cb3e0 legacy=0x7efc100b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0e7fc700 1 --2- 172.21.15.81:0/3942131103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf8003a20 0x7efbf8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0e7fc700 1 --2- 172.21.15.81:0/3942131103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efbf8004870 0x7efbf8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0e7fc700 1 --2- 172.21.15.81:0/3942131103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efbf8005430 0x7efbf800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0e7fc700 1 -- 172.21.15.81:0/3942131103 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc101620b0 con 0x7efbf8005430 2024-05-08T23:30:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0e7fc700 1 -- 172.21.15.81:0/3942131103 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf800c2d0 con 0x7efbf8003a20 2024-05-08T23:30:44.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0e7fc700 1 -- 172.21.15.81:0/3942131103 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf800c490 con 0x7efbf8004870 2024-05-08T23:30:44.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0f7fe700 1 --2- 172.21.15.81:0/3942131103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf8003a20 0x7efbf8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0ffff700 1 --2- 172.21.15.81:0/3942131103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efbf8005430 0x7efbf800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f846d6f2700 1 -- 172.21.15.81:0/3798004156 shutdown_connections 2024-05-08T23:30:44.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f846d6f2700 1 --2- 172.21.15.81:0/3798004156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f846808a620 0x7f84680b4020 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f846d6f2700 1 --2- 172.21.15.81:0/3798004156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84680b3c40 0x7f846806e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f846d6f2700 1 -- 172.21.15.81:0/3798004156 >> 172.21.15.81:0/3798004156 conn(0x7f84680aebd0 msgr2=0x7f84680aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7efc0effd700 1 --2- 172.21.15.81:0/3942131103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efbf8004870 0x7efbf8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faa9a7fc700 1 -- 172.21.15.81:0/2395049805 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8c00bbc0 con 0x7faa9c06e3a0 2024-05-08T23:30:44.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faa9a7fc700 1 -- 172.21.15.81:0/2395049805 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c009e60 con 0x7faa9c06e3a0 2024-05-08T23:30:44.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faaa0d3a700 1 -- 172.21.15.81:0/2395049805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa9c06e3a0 msgr2=0x7faa9c0b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faaa0d3a700 1 --2- 172.21.15.81:0/2395049805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa9c06e3a0 0x7faa9c0b3c00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009910 tx=0x7faa8c009c20 comp rx=0 tx=0).stop 2024-05-08T23:30:44.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faaa0d3a700 1 -- 172.21.15.81:0/2395049805 shutdown_connections 2024-05-08T23:30:44.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faaa0d3a700 1 --2- 172.21.15.81:0/2395049805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa9c06e3a0 0x7faa9c0b3c00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faaa0d3a700 1 --2- 172.21.15.81:0/2395049805 >> v2:172.21.15.117:3300/0 conn(0x7faa9c0cb3e0 0x7faa9c165ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f9377e80700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f93700cb3b0 0x7f93701478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7faaa0d3a700 1 -- 172.21.15.81:0/2395049805 >> 172.21.15.81:0/2395049805 conn(0x7faa9c0aebd0 msgr2=0x7faa9c0aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f846d6f2700 1 -- 172.21.15.81:0/3798004156 shutdown_connections 2024-05-08T23:30:44.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.505+0000 7f74f8d00700 1 -- 172.21.15.81:0/2370269837 shutdown_connections 2024-05-08T23:30:44.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f846d6f2700 1 -- 172.21.15.81:0/3798004156 wait complete. 2024-05-08T23:30:44.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e9a18700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f35e40b4310 0x7f35e4165ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9377e80700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93701656a0 con 0x7f9370144d40 2024-05-08T23:30:44.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9377e80700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9370165970 con 0x7f9370165240 2024-05-08T23:30:44.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9377e80700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9370161c50 con 0x7f93700cb3b0 2024-05-08T23:30:44.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e9a18700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e4162700 con 0x7f35e40cb3e0 2024-05-08T23:30:44.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e9a18700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e4161dd0 con 0x7f35e406e3a0 2024-05-08T23:30:44.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e9a18700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e41620b0 con 0x7f35e40b4310 2024-05-08T23:30:44.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7faaa0d3a700 1 -- 172.21.15.81:0/2395049805 shutdown_connections 2024-05-08T23:30:44.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e8a16700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f35e40cb3e0 0x7f35e40b3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55094/0 (socket says 172.21.15.81:55094) 2024-05-08T23:30:44.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e8a16700 1 -- 172.21.15.81:0/1001290954 learned_addr learned my addr 172.21.15.81:0/1001290954 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:44.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7faaa0d3a700 1 -- 172.21.15.81:0/2395049805 wait complete. 2024-05-08T23:30:44.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc0ffff700 1 -- 172.21.15.81:0/3942131103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efbf8004870 msgr2=0x7efbf8004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc0ffff700 1 --2- 172.21.15.81:0/3942131103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efbf8004870 0x7efbf8004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc0ffff700 1 -- 172.21.15.81:0/3942131103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf8003a20 msgr2=0x7efbf8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc0ffff700 1 --2- 172.21.15.81:0/3942131103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf8003a20 0x7efbf8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc0ffff700 1 -- 172.21.15.81:0/3942131103 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf80035d0 con 0x7efbf8005430 2024-05-08T23:30:44.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc0ffff700 1 -- 172.21.15.81:0/3942131103 reap_dead start 2024-05-08T23:30:44.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba5076b700 1 Processor -- start 2024-05-08T23:30:44.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc0ffff700 1 --2- 172.21.15.81:0/3942131103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efbf8005430 0x7efbf800ba30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efc08009e60 tx=0x7efbf80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75080770f04c036 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba5076b700 1 -- start start 2024-05-08T23:30:44.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e3fff700 1 --2- 172.21.15.81:0/1001290954 >> v2:172.21.15.117:3300/0 conn(0x7f35e40b4310 0x7f35e4165ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc0e7fc700 1 -- 172.21.15.81:0/3942131103 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc080098d0 con 0x7efbf8005430 2024-05-08T23:30:44.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc0e7fc700 1 -- 172.21.15.81:0/3942131103 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc08003c40 con 0x7efbf8005430 2024-05-08T23:30:44.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f937641d700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f9370144d40 0x7f937006c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55104/0 (socket says 172.21.15.81:55104) 2024-05-08T23:30:44.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f937641d700 1 -- 172.21.15.81:0/347422548 learned_addr learned my addr 172.21.15.81:0/347422548 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:44.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9374c1a700 1 -- 172.21.15.81:0/347422548 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582174748 0 0) 0x7f93701656a0 con 0x7f9370144d40 2024-05-08T23:30:44.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9374c1a700 1 -- 172.21.15.81:0/347422548 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f935c0035d0 con 0x7f9370144d40 2024-05-08T23:30:44.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9374c1a700 1 -- 172.21.15.81:0/347422548 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1754674978 0 0) 0x7f935c0035d0 con 0x7f9370144d40 2024-05-08T23:30:44.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9374c1a700 1 -- 172.21.15.81:0/347422548 >> v2:172.21.15.81:3300/0 conn(0x7f93700cb3b0 msgr2=0x7f93701478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9374c1a700 1 --2- 172.21.15.81:0/347422548 >> v2:172.21.15.81:3300/0 conn(0x7f93700cb3b0 0x7f93701478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9374c1a700 1 -- 172.21.15.81:0/347422548 >> v1:172.21.15.117:6789/0 conn(0x7f9370165240 legacy=0x7f9370162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9374c1a700 1 -- 172.21.15.81:0/347422548 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9370165c50 con 0x7f9370144d40 2024-05-08T23:30:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9374c1a700 1 -- 172.21.15.81:0/347422548 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f93680019f0 con 0x7f9370144d40 2024-05-08T23:30:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9374c1a700 1 -- 172.21.15.81:0/347422548 >> v1:172.21.15.81:6789/0 conn(0x7f9370144d40 legacy=0x7f937006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9374c1a700 1 --2- 172.21.15.81:0/347422548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f935c003a20 0x7f935c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9374c1a700 1 --2- 172.21.15.81:0/347422548 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f935c004870 0x7f935c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9374c1a700 1 --2- 172.21.15.81:0/347422548 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f935c005380 0x7f935c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9374c1a700 1 -- 172.21.15.81:0/347422548 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c00c290 con 0x7f935c004870 2024-05-08T23:30:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9374c1a700 1 -- 172.21.15.81:0/347422548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c00c400 con 0x7f935c003a20 2024-05-08T23:30:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9374c1a700 1 -- 172.21.15.81:0/347422548 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c00c5c0 con 0x7f935c005380 2024-05-08T23:30:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc0e7fc700 1 -- 172.21.15.81:0/3942131103 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0800b8a0 con 0x7efbf8005430 2024-05-08T23:30:44.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9375c1c700 1 --2- 172.21.15.81:0/347422548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f935c003a20 0x7f935c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc14d64700 1 -- 172.21.15.81:0/3942131103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efbf8005430 msgr2=0x7efbf800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc14d64700 1 --2- 172.21.15.81:0/3942131103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efbf8005430 0x7efbf800ba30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efc08009e60 tx=0x7efbf80058a0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663672792 0 0) 0x7f35e4161dd0 con 0x7f35e406e3a0 2024-05-08T23:30:44.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35cc0035d0 con 0x7f35e406e3a0 2024-05-08T23:30:44.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f937541b700 1 --2- 172.21.15.81:0/347422548 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f935c004870 0x7f935c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc14d64700 1 -- 172.21.15.81:0/3942131103 shutdown_connections 2024-05-08T23:30:44.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc14d64700 1 --2- 172.21.15.81:0/3942131103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efbf8005430 0x7efbf800ba30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc14d64700 1 -- 172.21.15.81:0/3942131103 >> 172.21.15.81:0/3942131103 conn(0x7efc100aebd0 msgr2=0x7efc100aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba5076b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fba480b1f60 0x7fba4806e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba5076b700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba480b2740 con 0x7fba48160c80 2024-05-08T23:30:44.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520714469 0 0) 0x7f35e4162700 con 0x7f35e40cb3e0 2024-05-08T23:30:44.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35e4161dd0 con 0x7f35e40cb3e0 2024-05-08T23:30:44.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7faaa0d3a700 1 Processor -- start 2024-05-08T23:30:44.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4e507700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fba48160c80 0x7fba48161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55116/0 (socket says 172.21.15.81:55116) 2024-05-08T23:30:44.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4e507700 1 -- 172.21.15.81:0/208389576 learned_addr learned my addr 172.21.15.81:0/208389576 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:44.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4dd06700 1 --2- 172.21.15.81:0/208389576 >> v2:172.21.15.81:3300/0 conn(0x7fba480b1f60 0x7fba4806e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f937641d700 1 --2- 172.21.15.81:0/347422548 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f935c005380 0x7f935c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba5076b700 1 -- 172.21.15.81:0/208389576 --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba4806d990 con 0x7fba480ac8e0 2024-05-08T23:30:44.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba5076b700 1 -- 172.21.15.81:0/208389576 --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba48146d60 con 0x7fba480b1f60 2024-05-08T23:30:44.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4d505700 1 -- 172.21.15.81:0/208389576 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223481798 0 0) 0x7fba480b2740 con 0x7fba48160c80 2024-05-08T23:30:44.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4d505700 1 -- 172.21.15.81:0/208389576 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba340035d0 con 0x7fba48160c80 2024-05-08T23:30:44.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1422336605 0 0) 0x7f35cc0035d0 con 0x7f35e406e3a0 2024-05-08T23:30:44.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 >> v2:172.21.15.117:3300/0 conn(0x7f35e40b4310 msgr2=0x7f35e4165ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 --2- 172.21.15.81:0/1001290954 >> v2:172.21.15.117:3300/0 conn(0x7f35e40b4310 0x7f35e4165ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 >> v1:172.21.15.81:6789/0 conn(0x7f35e40cb3e0 legacy=0x7f35e40b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4162390 con 0x7f35e406e3a0 2024-05-08T23:30:44.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f35dc0019f0 con 0x7f35e406e3a0 2024-05-08T23:30:44.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 >> v1:172.21.15.117:6789/0 conn(0x7f35e406e3a0 legacy=0x7f35e406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 --2- 172.21.15.81:0/1001290954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35cc003c00 0x7f35cc004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 --2- 172.21.15.81:0/1001290954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35cc004870 0x7f35cc004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 --2- 172.21.15.81:0/1001290954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35cc005300 0x7f35cc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e41620b0 con 0x7f35cc004870 2024-05-08T23:30:44.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc14d64700 1 -- 172.21.15.81:0/3942131103 shutdown_connections 2024-05-08T23:30:44.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc00c250 con 0x7f35cc003c00 2024-05-08T23:30:44.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc00c410 con 0x7f35cc005300 2024-05-08T23:30:44.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7efc14d64700 1 -- 172.21.15.81:0/3942131103 wait complete. 2024-05-08T23:30:44.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f846d6f2700 1 Processor -- start 2024-05-08T23:30:44.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f846d6f2700 1 -- start start 2024-05-08T23:30:44.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f74f8d00700 1 -- 172.21.15.81:0/2370269837 wait complete. 2024-05-08T23:30:44.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f74f8d00700 1 Processor -- start 2024-05-08T23:30:44.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f74f8d00700 1 -- start start 2024-05-08T23:30:44.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f74f8d00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74f406e3a0 0x7f74f415bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4d505700 1 -- 172.21.15.81:0/208389576 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 123564151 0 0) 0x7fba340035d0 con 0x7fba48160c80 2024-05-08T23:30:44.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4d505700 1 -- 172.21.15.81:0/208389576 >> v2:172.21.15.81:3300/0 conn(0x7fba480b1f60 msgr2=0x7fba4806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4d505700 1 --2- 172.21.15.81:0/208389576 >> v2:172.21.15.81:3300/0 conn(0x7fba480b1f60 0x7fba4806e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4d505700 1 -- 172.21.15.81:0/208389576 >> v1:172.21.15.117:6789/0 conn(0x7fba480ac8e0 legacy=0x7fba480acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f74f8d00700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74f40b4310 0x7f74f4159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f74f8d00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74f40cb3e0 0x7f74f4207980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f846d6f2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f846808a620 0x7f84681f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f846d6f2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84680b3c40 0x7f8468071a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f846d6f2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f846816d820 0x7f8468213f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f846d6f2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8468214e70 con 0x7f846816d820 2024-05-08T23:30:44.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f846d6f2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84682148b0 con 0x7f84680b3c40 2024-05-08T23:30:44.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f846d6f2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8468214b90 con 0x7f846808a620 2024-05-08T23:30:44.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4d505700 1 -- 172.21.15.81:0/208389576 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba480b2460 con 0x7fba48160c80 2024-05-08T23:30:44.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4d505700 1 -- 172.21.15.81:0/208389576 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7fba44001670 con 0x7fba48160c80 2024-05-08T23:30:44.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4d505700 1 -- 172.21.15.81:0/208389576 >> v1:172.21.15.81:6789/0 conn(0x7fba48160c80 legacy=0x7fba48161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4d505700 1 --2- 172.21.15.81:0/208389576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba34003a20 0x7fba34003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4d505700 1 --2- 172.21.15.81:0/208389576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba34004420 0x7fba340048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4d505700 1 --2- 172.21.15.81:0/208389576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba34005190 0x7fba3400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4d505700 1 -- 172.21.15.81:0/208389576 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba48146d60 con 0x7fba34005190 2024-05-08T23:30:44.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4d505700 1 -- 172.21.15.81:0/208389576 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3400c0d0 con 0x7fba34003a20 2024-05-08T23:30:44.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4d505700 1 -- 172.21.15.81:0/208389576 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3400c290 con 0x7fba34004420 2024-05-08T23:30:44.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9375c1c700 1 -- 172.21.15.81:0/347422548 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f935c005380 msgr2=0x7f935c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9375c1c700 1 --2- 172.21.15.81:0/347422548 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f935c005380 0x7f935c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9375c1c700 1 -- 172.21.15.81:0/347422548 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f935c004870 msgr2=0x7f935c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f8467fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f846808a620 0x7f84681f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9375c1c700 1 --2- 172.21.15.81:0/347422548 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f935c004870 0x7f935c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f9375c1c700 1 -- 172.21.15.81:0/347422548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9370165c50 con 0x7f935c003a20 2024-05-08T23:30:44.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7faaa0d3a700 1 -- start start 2024-05-08T23:30:44.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f937641d700 1 --2- 172.21.15.81:0/347422548 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f935c005380 0x7f935c00ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:44.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f8467fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f846808a620 0x7f84681f3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:53266/0 (socket says 172.21.15.81:53266) 2024-05-08T23:30:44.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f8467fff700 1 -- 172.21.15.81:0/1558298486 learned_addr learned my addr 172.21.15.81:0/1558298486 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7faaa0d3a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa9c06e3a0 0x7faa9c154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f937641d700 1 -- 172.21.15.81:0/347422548 reap_dead start 2024-05-08T23:30:44.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7fba4dd06700 1 --2- 172.21.15.81:0/208389576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba34004420 0x7fba340048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f74f8d00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f4208010 con 0x7f74f406e3a0 2024-05-08T23:30:44.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f74f8d00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f42082f0 con 0x7f74f40cb3e0 2024-05-08T23:30:44.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f74f8d00700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f42085d0 con 0x7f74f40b4310 2024-05-08T23:30:44.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faaa0d3a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa9c0b4140 0x7faa9c159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faaa0d3a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa9c0cb3e0 0x7faa9c0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f74f2ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74f40b4310 0x7f74f4159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f74f2ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74f40b4310 0x7f74f4159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:53254/0 (socket says 172.21.15.81:53254) 2024-05-08T23:30:44.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f74f2ffd700 1 -- 172.21.15.81:0/2833606116 learned_addr learned my addr 172.21.15.81:0/2833606116 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f74f37fe700 1 --2- 172.21.15.81:0/2833606116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74f406e3a0 0x7f74f415bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f74f3fff700 1 --2- 172.21.15.81:0/2833606116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74f40cb3e0 0x7f74f4207980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f74f37fe700 1 -- 172.21.15.81:0/2833606116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74f40b4310 msgr2=0x7f74f4159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f74f37fe700 1 --2- 172.21.15.81:0/2833606116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74f40b4310 0x7f74f4159f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f74f37fe700 1 -- 172.21.15.81:0/2833606116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74f40cb3e0 msgr2=0x7f74f4207980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f74f37fe700 1 --2- 172.21.15.81:0/2833606116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74f40cb3e0 0x7f74f4207980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f74f37fe700 1 -- 172.21.15.81:0/2833606116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e40094b0 con 0x7f74f406e3a0 2024-05-08T23:30:44.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fba4ed08700 1 --2- 172.21.15.81:0/208389576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba34005190 0x7fba3400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faaa0d3a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c158bd0 con 0x7faa9c0b4140 2024-05-08T23:30:44.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faaa0d3a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c155530 con 0x7faa9c0cb3e0 2024-05-08T23:30:44.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faaa0d3a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c164850 con 0x7faa9c06e3a0 2024-05-08T23:30:44.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f84677fe700 1 --2- 172.21.15.81:0/1558298486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84680b3c40 0x7f8468071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fba4e507700 1 --2- 172.21.15.81:0/208389576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba34003a20 0x7fba34003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f74f37fe700 1 --2- 172.21.15.81:0/2833606116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74f406e3a0 0x7f74f415bc10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f74e800c4d0 tx=0x7f74e800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f9375c1c700 1 --2- 172.21.15.81:0/347422548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f935c003a20 0x7f935c003e60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f936c009930 tx=0x7f935c0058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9e57be54abf5ec7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f8467fff700 1 -- 172.21.15.81:0/1558298486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84680b3c40 msgr2=0x7f8468071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f8467fff700 1 --2- 172.21.15.81:0/1558298486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84680b3c40 0x7f8468071a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f8467fff700 1 -- 172.21.15.81:0/1558298486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f846816d820 msgr2=0x7f8468213f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:30:44.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f8467fff700 1 --2- 172.21.15.81:0/1558298486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f846816d820 0x7f8468213f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f8467fff700 1 -- 172.21.15.81:0/1558298486 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8460008b20 con 0x7f846808a620 2024-05-08T23:30:44.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faa9affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa9c0b4140 0x7faa9c159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faa9bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa9c0cb3e0 0x7faa9c0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faa9affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa9c0b4140 0x7faa9c159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33152/0 (socket says 172.21.15.81:33152) 2024-05-08T23:30:44.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faa9affd700 1 -- 172.21.15.81:0/2949169489 learned_addr learned my addr 172.21.15.81:0/2949169489 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faa9b7fe700 1 --2- 172.21.15.81:0/2949169489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa9c06e3a0 0x7faa9c154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f74dbfff700 1 -- 172.21.15.81:0/2833606116 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e8005d90 con 0x7f74f406e3a0 2024-05-08T23:30:44.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f9374c1a700 1 -- 172.21.15.81:0/347422548 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c0093a0 con 0x7f935c003a20 2024-05-08T23:30:44.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f74dbfff700 1 -- 172.21.15.81:0/2833606116 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e800ce80 con 0x7f74f406e3a0 2024-05-08T23:30:44.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f74f8d00700 1 -- 172.21.15.81:0/2833606116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f42088b0 con 0x7f74f406e3a0 2024-05-08T23:30:44.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc14d64700 1 Processor -- start 2024-05-08T23:30:44.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc14d64700 1 -- start start 2024-05-08T23:30:44.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faa9affd700 1 -- 172.21.15.81:0/2949169489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa9c06e3a0 msgr2=0x7faa9c154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faa9affd700 1 --2- 172.21.15.81:0/2949169489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa9c06e3a0 0x7faa9c154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faa9affd700 1 -- 172.21.15.81:0/2949169489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa9c0cb3e0 msgr2=0x7faa9c0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faa9affd700 1 --2- 172.21.15.81:0/2949169489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa9c0cb3e0 0x7faa9c0cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faa9affd700 1 -- 172.21.15.81:0/2949169489 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c009570 con 0x7faa9c0b4140 2024-05-08T23:30:44.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faa9affd700 1 --2- 172.21.15.81:0/2949169489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa9c0b4140 0x7faa9c159890 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faa8c000c00 tx=0x7faa8c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f9374c1a700 1 -- 172.21.15.81:0/347422548 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936c004250 con 0x7f935c003a20 2024-05-08T23:30:44.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f9374c1a700 1 -- 172.21.15.81:0/347422548 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c0036a0 con 0x7f935c003a20 2024-05-08T23:30:44.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fba4ed08700 1 -- 172.21.15.81:0/208389576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba34004420 msgr2=0x7fba340048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fba4ed08700 1 --2- 172.21.15.81:0/208389576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba34004420 0x7fba340048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fba4ed08700 1 -- 172.21.15.81:0/208389576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba34003a20 msgr2=0x7fba34003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fba4ed08700 1 --2- 172.21.15.81:0/208389576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba34003a20 0x7fba34003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fba4ed08700 1 -- 172.21.15.81:0/208389576 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba480b2460 con 0x7fba34005190 2024-05-08T23:30:44.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fba4ed08700 1 -- 172.21.15.81:0/208389576 reap_dead start 2024-05-08T23:30:44.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fba4ed08700 1 --2- 172.21.15.81:0/208389576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba34005190 0x7fba3400ba30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba380018f0 tx=0x7fba340058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80de3653760a7cb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f8467fff700 1 --2- 172.21.15.81:0/1558298486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f846808a620 0x7f84681f3270 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f845c00d860 tx=0x7f845c00dc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fb465693700 1 Processor -- start 2024-05-08T23:30:44.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f74f8d00700 1 -- 172.21.15.81:0/2833606116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f4154b40 con 0x7f74f406e3a0 2024-05-08T23:30:44.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f74dbfff700 1 -- 172.21.15.81:0/2833606116 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e80072f0 con 0x7f74f406e3a0 2024-05-08T23:30:44.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.509+0000 7f35e8a16700 1 --2- 172.21.15.81:0/1001290954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35cc003c00 0x7f35cc004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f35e8a16700 1 -- 172.21.15.81:0/1001290954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35cc005300 msgr2=0x7f35cc00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f35e8a16700 1 --2- 172.21.15.81:0/1001290954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35cc005300 0x7f35cc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f35e8a16700 1 -- 172.21.15.81:0/1001290954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35cc004870 msgr2=0x7f35cc004cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:44.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f35e8a16700 1 --2- 172.21.15.81:0/1001290954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35cc004870 0x7f35cc004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f35e8a16700 1 -- 172.21.15.81:0/1001290954 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc0035d0 con 0x7f35cc003c00 2024-05-08T23:30:44.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f35e8a16700 1 --2- 172.21.15.81:0/1001290954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35cc003c00 0x7f35cc004000 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f35d4001a80 tx=0x7f35cc0058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8a161bfb3aef033 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f35e9217700 1 -- 172.21.15.81:0/1001290954 reap_dead start 2024-05-08T23:30:44.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc14d64700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf8005430 0x7efc10155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc14d64700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc101554e0 0x7efc10154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc14d64700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc10160e30 0x7efc101612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc14d64700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc10155de0 con 0x7efc10160e30 2024-05-08T23:30:44.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc14d64700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc101617f0 con 0x7efbf8005430 2024-05-08T23:30:44.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc14d64700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc10163c80 con 0x7efc101554e0 2024-05-08T23:30:44.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc0ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc10160e30 0x7efc101612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc0ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc10160e30 0x7efc101612b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33162/0 (socket says 172.21.15.81:33162) 2024-05-08T23:30:44.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc0ffff700 1 -- 172.21.15.81:0/522458045 learned_addr learned my addr 172.21.15.81:0/522458045 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc0f7fe700 1 --2- 172.21.15.81:0/522458045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf8005430 0x7efc10155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc0effd700 1 --2- 172.21.15.81:0/522458045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc101554e0 0x7efc10154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc0ffff700 1 -- 172.21.15.81:0/522458045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc101554e0 msgr2=0x7efc10154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc0ffff700 1 --2- 172.21.15.81:0/522458045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc101554e0 0x7efc10154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc0ffff700 1 -- 172.21.15.81:0/522458045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf8005430 msgr2=0x7efc10155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc0ffff700 1 --2- 172.21.15.81:0/522458045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf8005430 0x7efc10155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc0ffff700 1 -- 172.21.15.81:0/522458045 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc10163f60 con 0x7efc10160e30 2024-05-08T23:30:44.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f8464ff9700 1 -- 172.21.15.81:0/1558298486 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c00de00 con 0x7f846808a620 2024-05-08T23:30:44.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f8464ff9700 1 -- 172.21.15.81:0/1558298486 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845c00c850 con 0x7f846808a620 2024-05-08T23:30:44.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f8464ff9700 1 -- 172.21.15.81:0/1558298486 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c00cbc0 con 0x7f846808a620 2024-05-08T23:30:44.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fba4d505700 1 -- 172.21.15.81:0/208389576 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba38001390 con 0x7fba34005190 2024-05-08T23:30:44.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fba4d505700 1 -- 172.21.15.81:0/208389576 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba380052d0 con 0x7fba34005190 2024-05-08T23:30:44.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fba4d505700 1 -- 172.21.15.81:0/208389576 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba380036a0 con 0x7fba34005190 2024-05-08T23:30:44.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fba5076b700 1 -- 172.21.15.81:0/208389576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba34005190 msgr2=0x7fba3400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fba5076b700 1 --2- 172.21.15.81:0/208389576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba34005190 0x7fba3400ba30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba380018f0 tx=0x7fba340058a0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f846d6f2700 1 -- 172.21.15.81:0/1558298486 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84681f2a70 con 0x7f846808a620 2024-05-08T23:30:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f846d6f2700 1 -- 172.21.15.81:0/1558298486 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84681f37b0 con 0x7f846808a620 2024-05-08T23:30:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f9377e80700 1 -- 172.21.15.81:0/347422548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f935c003a20 msgr2=0x7f935c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f9377e80700 1 --2- 172.21.15.81:0/347422548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f935c003a20 0x7f935c003e60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f936c009930 tx=0x7f935c0058a0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f9377e80700 1 -- 172.21.15.81:0/347422548 shutdown_connections 2024-05-08T23:30:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f9377e80700 1 --2- 172.21.15.81:0/347422548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f935c003a20 0x7f935c003e60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f9377e80700 1 -- 172.21.15.81:0/347422548 >> 172.21.15.81:0/347422548 conn(0x7f937006a360 msgr2=0x7f937006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4004b50 con 0x7f35cc003c00 2024-05-08T23:30:44.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d4004cb0 con 0x7f35cc003c00 2024-05-08T23:30:44.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fb465693700 1 -- start start 2024-05-08T23:30:44.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f35e37fe700 1 -- 172.21.15.81:0/1001290954 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4007310 con 0x7f35cc003c00 2024-05-08T23:30:44.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fb465693700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb4600b3c20 0x7fb4601624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fb465693700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb460070110 0x7fb4600705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fb465693700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb460158960 con 0x7fb460071e70 2024-05-08T23:30:44.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fb465693700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4600b1f60 con 0x7fb460070110 2024-05-08T23:30:44.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fb465693700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4600ae990 con 0x7fb4600b3c20 2024-05-08T23:30:44.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faa83fff700 1 -- 172.21.15.81:0/2949169489 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c004510 con 0x7faa9c0b4140 2024-05-08T23:30:44.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faa83fff700 1 -- 172.21.15.81:0/2949169489 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8c01f070 con 0x7faa9c0b4140 2024-05-08T23:30:44.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faa83fff700 1 -- 172.21.15.81:0/2949169489 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c003900 con 0x7faa9c0b4140 2024-05-08T23:30:44.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f35e9a18700 1 -- 172.21.15.81:0/1001290954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35cc003c00 msgr2=0x7f35cc004000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f35e9a18700 1 --2- 172.21.15.81:0/1001290954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35cc003c00 0x7f35cc004000 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f35d4001a80 tx=0x7f35cc0058a0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fb464e92700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb460070110 0x7fb4600705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fb464e92700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb460070110 0x7fb4600705a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33166/0 (socket says 172.21.15.81:33166) 2024-05-08T23:30:44.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7fb464e92700 1 -- 172.21.15.81:0/786103464 learned_addr learned my addr 172.21.15.81:0/786103464 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f74dbfff700 1 -- 172.21.15.81:0/2833606116 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f74e8007450 con 0x7f74f406e3a0 2024-05-08T23:30:44.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f74f8d00700 1 -- 172.21.15.81:0/2833606116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f40b0900 con 0x7f74f406e3a0 2024-05-08T23:30:44.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7f74dbfff700 1 --2- 172.21.15.81:0/2833606116 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f74e0066780 0x7f74e0068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f74dbfff700 1 -- 172.21.15.81:0/2833606116 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74e80830a0 con 0x7f74f406e3a0 2024-05-08T23:30:44.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f74f2ffd700 1 --2- 172.21.15.81:0/2833606116 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f74e0066780 0x7f74e0068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc0ffff700 1 --2- 172.21.15.81:0/522458045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc10160e30 0x7efc101612b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efc10158be0 tx=0x7efc08004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efbf7fff700 1 -- 172.21.15.81:0/522458045 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc080158b0 con 0x7efc10160e30 2024-05-08T23:30:44.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc14d64700 1 -- 172.21.15.81:0/522458045 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc10164240 con 0x7efc10160e30 2024-05-08T23:30:44.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efc14d64700 1 -- 172.21.15.81:0/522458045 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc100ac020 con 0x7efc10160e30 2024-05-08T23:30:44.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efbf7fff700 1 -- 172.21.15.81:0/522458045 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc08015a30 con 0x7efc10160e30 2024-05-08T23:30:44.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7efbf7fff700 1 -- 172.21.15.81:0/522458045 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc08015de0 con 0x7efc10160e30 2024-05-08T23:30:44.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faaa0d3a700 1 -- 172.21.15.81:0/2949169489 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9c158db0 con 0x7faa9c0b4140 2024-05-08T23:30:44.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.513+0000 7faaa0d3a700 1 -- 172.21.15.81:0/2949169489 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9c159370 con 0x7faa9c0b4140 2024-05-08T23:30:44.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f9377e80700 1 -- 172.21.15.81:0/347422548 shutdown_connections 2024-05-08T23:30:44.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fba5076b700 1 -- 172.21.15.81:0/208389576 shutdown_connections 2024-05-08T23:30:44.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fba5076b700 1 --2- 172.21.15.81:0/208389576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba34005190 0x7fba3400ba30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fba5076b700 1 -- 172.21.15.81:0/208389576 >> 172.21.15.81:0/208389576 conn(0x7fba480aebd0 msgr2=0x7fba480aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fba5076b700 1 -- 172.21.15.81:0/208389576 shutdown_connections 2024-05-08T23:30:44.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f35e9a18700 1 -- 172.21.15.81:0/1001290954 shutdown_connections 2024-05-08T23:30:44.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f35e9a18700 1 --2- 172.21.15.81:0/1001290954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35cc003c00 0x7f35cc004000 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f35d4001a80 tx=0x7f35cc0058a0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f35e9a18700 1 -- 172.21.15.81:0/1001290954 >> 172.21.15.81:0/1001290954 conn(0x7f35e40aebd0 msgr2=0x7f35e40aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fba5076b700 1 -- 172.21.15.81:0/208389576 wait complete. 2024-05-08T23:30:44.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fb45ffff700 1 --2- 172.21.15.81:0/786103464 >> v2:172.21.15.117:3300/0 conn(0x7fb4600b3c20 0x7fb4601624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7faaa0d3a700 1 -- 172.21.15.81:0/2949169489 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9c0b0900 con 0x7faa9c0b4140 2024-05-08T23:30:44.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f8464ff9700 1 -- 172.21.15.81:0/1558298486 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f845c00cd20 con 0x7f846808a620 2024-05-08T23:30:44.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fb45ffff700 1 -- 172.21.15.81:0/786103464 >> v2:172.21.15.81:3300/0 conn(0x7fb460070110 msgr2=0x7fb4600705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:44.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fb45ffff700 1 --2- 172.21.15.81:0/786103464 >> v2:172.21.15.81:3300/0 conn(0x7fb460070110 0x7fb4600705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fb45ffff700 1 -- 172.21.15.81:0/786103464 >> v1:172.21.15.81:6789/0 conn(0x7fb460071e70 legacy=0x7fb4600b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fb45ffff700 1 -- 172.21.15.81:0/786103464 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4600b2170 con 0x7fb4600b3c20 2024-05-08T23:30:44.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f74f2ffd700 1 --2- 172.21.15.81:0/2833606116 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f74e0066780 0x7f74e0068c40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f74f4161420 tx=0x7f74ec00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fb45ffff700 1 --2- 172.21.15.81:0/786103464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4600b3c20 0x7fb4601624b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb46006eba0 tx=0x7fb45800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2886a2b81c2fde0e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f8464ff9700 1 --2- 172.21.15.81:0/1558298486 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8454066730 0x7f8454068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f8464ff9700 1 -- 172.21.15.81:0/1558298486 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f845c083750 con 0x7f846808a620 2024-05-08T23:30:44.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f84677fe700 1 --2- 172.21.15.81:0/1558298486 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8454066730 0x7f8454068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f35e9a18700 1 -- 172.21.15.81:0/1001290954 shutdown_connections 2024-05-08T23:30:44.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fba5076b700 1 Processor -- start 2024-05-08T23:30:44.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f84677fe700 1 --2- 172.21.15.81:0/1558298486 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8454066730 0x7f8454068bf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f846000b0f0 tx=0x7f846000b000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fba5076b700 1 -- start start 2024-05-08T23:30:44.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f846d6f2700 1 -- 172.21.15.81:0/1558298486 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84681f39a0 con 0x7f846808a620 2024-05-08T23:30:44.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7efc14d64700 1 -- 172.21.15.81:0/522458045 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf0005310 con 0x7efc10160e30 2024-05-08T23:30:44.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f9377e80700 1 -- 172.21.15.81:0/347422548 wait complete. 2024-05-08T23:30:44.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fb45effd700 1 -- 172.21.15.81:0/786103464 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800e070 con 0x7fb4600b3c20 2024-05-08T23:30:44.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f35e9a18700 1 -- 172.21.15.81:0/1001290954 wait complete. 2024-05-08T23:30:44.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fb45effd700 1 -- 172.21.15.81:0/786103464 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45800bb20 con 0x7fb4600b3c20 2024-05-08T23:30:44.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fb45effd700 1 -- 172.21.15.81:0/786103464 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458012430 con 0x7fb4600b3c20 2024-05-08T23:30:44.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f35e9a18700 1 Processor -- start 2024-05-08T23:30:44.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f9377e80700 1 Processor -- start 2024-05-08T23:30:44.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fba5076b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba34005190 0x7fba48160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f35e9a18700 1 -- start start 2024-05-08T23:30:44.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fba5076b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba48156d30 0x7fba48159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fba5076b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba4815a170 0x7fba481377d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fba5076b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba48158730 con 0x7fba34005190 2024-05-08T23:30:44.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fba5076b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba48158450 con 0x7fba48156d30 2024-05-08T23:30:44.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7fba5076b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4815a990 con 0x7fba4815a170 2024-05-08T23:30:44.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.517+0000 7f9377e80700 1 -- start start 2024-05-08T23:30:44.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fba4e507700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba34005190 0x7fba48160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fba4e507700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba34005190 0x7fba48160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33174/0 (socket says 172.21.15.81:33174) 2024-05-08T23:30:44.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fba4e507700 1 -- 172.21.15.81:0/2586334215 learned_addr learned my addr 172.21.15.81:0/2586334215 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fba4dd06700 1 --2- 172.21.15.81:0/2586334215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba48156d30 0x7fba48159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fba4ed08700 1 --2- 172.21.15.81:0/2586334215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba4815a170 0x7fba481377d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fba4e507700 1 -- 172.21.15.81:0/2586334215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba4815a170 msgr2=0x7fba481377d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fba4e507700 1 --2- 172.21.15.81:0/2586334215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba4815a170 0x7fba481377d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fba4e507700 1 -- 172.21.15.81:0/2586334215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba48156d30 msgr2=0x7fba48159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fba4e507700 1 --2- 172.21.15.81:0/2586334215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba48156d30 0x7fba48159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fba4e507700 1 -- 172.21.15.81:0/2586334215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4815ac70 con 0x7fba34005190 2024-05-08T23:30:44.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 -- 172.21.15.81:0/786103464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4600b3c20 msgr2=0x7fb4601624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 --2- 172.21.15.81:0/786103464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4600b3c20 0x7fb4601624b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb46006eba0 tx=0x7fb45800b330 comp rx=0 tx=0).stop 2024-05-08T23:30:44.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 -- 172.21.15.81:0/786103464 shutdown_connections 2024-05-08T23:30:44.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 --2- 172.21.15.81:0/786103464 >> v2:172.21.15.81:3300/0 conn(0x7fb460070110 0x7fb4600705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 --2- 172.21.15.81:0/786103464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4600b3c20 0x7fb4601624b0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb46006eba0 tx=0x7fb45800b330 comp rx=0 tx=0).stop 2024-05-08T23:30:44.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 -- 172.21.15.81:0/786103464 >> 172.21.15.81:0/786103464 conn(0x7fb46006c330 msgr2=0x7fb46006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e9a18700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e41484b0 0x7f35e4151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e9a18700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35e4148890 0x7f35e4143f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e9a18700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35e4144440 0x7f35e415f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e9a18700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e4152680 con 0x7f35e4148890 2024-05-08T23:30:44.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e9a18700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e41517d0 con 0x7f35e41484b0 2024-05-08T23:30:44.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e9a18700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e4151a00 con 0x7f35e4144440 2024-05-08T23:30:44.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fba4e507700 1 --2- 172.21.15.81:0/2586334215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba34005190 0x7fba48160080 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fba44007310 tx=0x7fba44007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 -- 172.21.15.81:0/786103464 shutdown_connections 2024-05-08T23:30:44.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 -- 172.21.15.81:0/786103464 wait complete. 2024-05-08T23:30:44.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e8a16700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e41484b0 0x7f35e4151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35e4148890 0x7f35e4143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e8a16700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e41484b0 0x7f35e4151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:44466/0 (socket says 172.21.15.81:44466) 2024-05-08T23:30:44.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35e4148890 0x7f35e4143f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33190/0 (socket says 172.21.15.81:33190) 2024-05-08T23:30:44.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e8a16700 1 -- 172.21.15.81:0/3524721248 learned_addr learned my addr 172.21.15.81:0/3524721248 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e9217700 1 --2- 172.21.15.81:0/3524721248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35e4144440 0x7f35e415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f9377e80700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f935c003a20 0x7f9370141220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f9377e80700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93701609f0 0x7f9370160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e3fff700 1 -- 172.21.15.81:0/3524721248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35e4144440 msgr2=0x7f35e415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e3fff700 1 --2- 172.21.15.81:0/3524721248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35e4144440 0x7f35e415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e3fff700 1 -- 172.21.15.81:0/3524721248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e41484b0 msgr2=0x7f35e4151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e3fff700 1 --2- 172.21.15.81:0/3524721248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e41484b0 0x7f35e4151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e3fff700 1 -- 172.21.15.81:0/3524721248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4151ce0 con 0x7f35e4148890 2024-05-08T23:30:44.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f9377e80700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f937015af90 0x7f937015b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f9377e80700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93701429b0 con 0x7f93701609f0 2024-05-08T23:30:44.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f9377e80700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9370141760 con 0x7f937015af90 2024-05-08T23:30:44.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f9377e80700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93701613b0 con 0x7f935c003a20 2024-05-08T23:30:44.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f9375c1c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f935c003a20 0x7f9370141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f937541b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93701609f0 0x7f9370160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f9375c1c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f935c003a20 0x7f9370141220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:53324/0 (socket says 172.21.15.81:53324) 2024-05-08T23:30:44.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f9375c1c700 1 -- 172.21.15.81:0/3882969880 learned_addr learned my addr 172.21.15.81:0/3882969880 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e3fff700 1 --2- 172.21.15.81:0/3524721248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35e4148890 0x7f35e4143f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f35e40692d0 tx=0x7f35d800e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f937541b700 1 -- 172.21.15.81:0/3882969880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f935c003a20 msgr2=0x7f9370141220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f937541b700 1 --2- 172.21.15.81:0/3882969880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f935c003a20 0x7f9370141220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f937541b700 1 -- 172.21.15.81:0/3882969880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f937015af90 msgr2=0x7f937015b410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:30:44.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f937541b700 1 --2- 172.21.15.81:0/3882969880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f937015af90 0x7f937015b410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f937541b700 1 -- 172.21.15.81:0/3882969880 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9370159af0 con 0x7f93701609f0 2024-05-08T23:30:44.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f9375c1c700 1 --2- 172.21.15.81:0/3882969880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f935c003a20 0x7f9370141220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:44.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f937541b700 1 --2- 172.21.15.81:0/3882969880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93701609f0 0x7f9370160e70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f936000f870 tx=0x7f936000fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 Processor -- start 2024-05-08T23:30:44.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 -- start start 2024-05-08T23:30:44.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb460070110 0x7fb46006c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb460071e70 0x7fb46013a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb46013a5d0 0x7fb4601641c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb460144d20 con 0x7fb460070110 2024-05-08T23:30:44.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb460164f30 con 0x7fb46013a5d0 2024-05-08T23:30:44.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb465693700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb460165210 con 0x7fb460071e70 2024-05-08T23:30:44.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb45ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb460070110 0x7fb46006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb45ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb460070110 0x7fb46006c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33204/0 (socket says 172.21.15.81:33204) 2024-05-08T23:30:44.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb45ffff700 1 -- 172.21.15.81:0/3470069854 learned_addr learned my addr 172.21.15.81:0/3470069854 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:44.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb464e92700 1 --2- 172.21.15.81:0/3470069854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb46013a5d0 0x7fb4601641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb45f7fe700 1 --2- 172.21.15.81:0/3470069854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb460071e70 0x7fb46013a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e17fa700 1 -- 172.21.15.81:0/3524721248 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d800c070 con 0x7f35e4148890 2024-05-08T23:30:44.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f35e9a18700 1 -- 172.21.15.81:0/3524721248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e4151fc0 con 0x7f35e4148890 2024-05-08T23:30:44.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7f35e9a18700 1 -- 172.21.15.81:0/3524721248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e41448c0 con 0x7f35e4148890 2024-05-08T23:30:44.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7f74dbfff700 1 -- 172.21.15.81:0/2833606116 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f74e8088060 con 0x7f74f406e3a0 2024-05-08T23:30:44.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7f93667fc700 1 -- 172.21.15.81:0/3882969880 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360018070 con 0x7f93701609f0 2024-05-08T23:30:44.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7f9377e80700 1 -- 172.21.15.81:0/3882969880 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9370142c00 con 0x7f93701609f0 2024-05-08T23:30:44.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb45ffff700 1 -- 172.21.15.81:0/3470069854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb460071e70 msgr2=0x7fb46013a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb45ffff700 1 --2- 172.21.15.81:0/3470069854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb460071e70 0x7fb46013a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb45ffff700 1 -- 172.21.15.81:0/3470069854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb46013a5d0 msgr2=0x7fb4601641c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb45ffff700 1 --2- 172.21.15.81:0/3470069854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb46013a5d0 0x7fb4601641c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.521+0000 7fb45ffff700 1 -- 172.21.15.81:0/3470069854 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb458009a70 con 0x7fb460070110 2024-05-08T23:30:44.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7f9377e80700 1 -- 172.21.15.81:0/3882969880 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93701431c0 con 0x7f93701609f0 2024-05-08T23:30:44.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7f93667fc700 1 -- 172.21.15.81:0/3882969880 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9360003be0 con 0x7f93701609f0 2024-05-08T23:30:44.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7f93667fc700 1 -- 172.21.15.81:0/3882969880 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360003d50 con 0x7f93701609f0 2024-05-08T23:30:44.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7f9377e80700 1 -- 172.21.15.81:0/3882969880 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9358005310 con 0x7f93701609f0 2024-05-08T23:30:44.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7faa83fff700 1 -- 172.21.15.81:0/2949169489 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faa8c003a60 con 0x7faa9c0b4140 2024-05-08T23:30:44.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7f35e17fa700 1 -- 172.21.15.81:0/3524721248 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d8004020 con 0x7f35e4148890 2024-05-08T23:30:44.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7f35e17fa700 1 -- 172.21.15.81:0/3524721248 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d8004390 con 0x7f35e4148890 2024-05-08T23:30:44.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7efbf7fff700 1 -- 172.21.15.81:0/522458045 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efc0802a440 con 0x7efc10160e30 2024-05-08T23:30:44.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7faa83fff700 1 --2- 172.21.15.81:0/2949169489 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa88066730 0x7faa88068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7faa83fff700 1 -- 172.21.15.81:0/2949169489 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa8c021070 con 0x7faa9c0b4140 2024-05-08T23:30:44.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7faa9b7fe700 1 --2- 172.21.15.81:0/2949169489 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa88066730 0x7faa88068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7f35e9a18700 1 -- 172.21.15.81:0/3524721248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e40b0900 con 0x7f35e4148890 2024-05-08T23:30:44.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7efbf7fff700 1 --2- 172.21.15.81:0/522458045 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efbfc066780 0x7efbfc068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7efc0f7fe700 1 --2- 172.21.15.81:0/522458045 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efbfc066780 0x7efbfc068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7efbf7fff700 1 -- 172.21.15.81:0/522458045 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc08011070 con 0x7efc10160e30 2024-05-08T23:30:44.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7faa9b7fe700 1 --2- 172.21.15.81:0/2949169489 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa88066730 0x7faa88068bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa90009930 tx=0x7faa90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.525+0000 7efc0f7fe700 1 --2- 172.21.15.81:0/522458045 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efbfc066780 0x7efbfc068c40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efc00000c00 tx=0x7efc00001c20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.529+0000 7fba3effd700 1 -- 172.21.15.81:0/2586334215 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba44011580 con 0x7fba34005190 2024-05-08T23:30:44.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.529+0000 7fba3effd700 1 -- 172.21.15.81:0/2586334215 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba440116e0 con 0x7fba34005190 2024-05-08T23:30:44.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.529+0000 7fba3effd700 1 -- 172.21.15.81:0/2586334215 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba44011a90 con 0x7fba34005190 2024-05-08T23:30:44.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.529+0000 7fba5076b700 1 -- 172.21.15.81:0/2586334215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba4815af50 con 0x7fba34005190 2024-05-08T23:30:44.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.529+0000 7fb45ffff700 1 --2- 172.21.15.81:0/3470069854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb460070110 0x7fb46006c980 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb45800b370 tx=0x7fb458004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.529+0000 7f8464ff9700 1 -- 172.21.15.81:0/1558298486 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f845c050c00 con 0x7f846808a620 2024-05-08T23:30:44.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.529+0000 7fba5076b700 1 -- 172.21.15.81:0/2586334215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4815a5f0 con 0x7fba34005190 2024-05-08T23:30:44.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.529+0000 7efbf7fff700 1 -- 172.21.15.81:0/522458045 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efc08052860 con 0x7efc10160e30 2024-05-08T23:30:44.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.533+0000 7f35e17fa700 1 -- 172.21.15.81:0/3524721248 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f35d80036c0 con 0x7f35e4148890 2024-05-08T23:30:44.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.533+0000 7f35e17fa700 1 --2- 172.21.15.81:0/3524721248 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f35d0066730 0x7f35d0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.533+0000 7f93667fc700 1 -- 172.21.15.81:0/3882969880 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9360006880 con 0x7f93701609f0 2024-05-08T23:30:44.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.533+0000 7f35e8a16700 1 --2- 172.21.15.81:0/3524721248 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f35d0066730 0x7f35d0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.533+0000 7f93667fc700 1 --2- 172.21.15.81:0/3882969880 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9354066760 0x7f9354068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.533+0000 7f9375c1c700 1 --2- 172.21.15.81:0/3882969880 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9354066760 0x7f9354068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.533+0000 7f35e8a16700 1 --2- 172.21.15.81:0/3524721248 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f35d0066730 0x7f35d0068bf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f35d4001eb0 tx=0x7f35d400a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.533+0000 7f9375c1c700 1 --2- 172.21.15.81:0/3882969880 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9354066760 0x7f9354068c20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f936c000c00 tx=0x7f936c01d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.537+0000 7fb45cff9700 1 -- 172.21.15.81:0/3470069854 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580042c0 con 0x7fb460070110 2024-05-08T23:30:44.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.537+0000 7fb465693700 1 -- 172.21.15.81:0/3470069854 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4601654f0 con 0x7fb460070110 2024-05-08T23:30:44.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.537+0000 7faa83fff700 1 -- 172.21.15.81:0/2949169489 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faa8c052fd0 con 0x7faa9c0b4140 2024-05-08T23:30:44.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.537+0000 7fb465693700 1 -- 172.21.15.81:0/3470069854 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46013aa60 con 0x7fb460070110 2024-05-08T23:30:44.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.537+0000 7f93667fc700 1 -- 172.21.15.81:0/3882969880 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9360055c50 con 0x7f93701609f0 2024-05-08T23:30:44.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.537+0000 7fba3effd700 1 -- 172.21.15.81:0/2586334215 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fba44011bf0 con 0x7fba34005190 2024-05-08T23:30:44.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.537+0000 7fba3effd700 1 --2- 172.21.15.81:0/2586334215 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fba2c066730 0x7fba2c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.537+0000 7fba3effd700 1 -- 172.21.15.81:0/2586334215 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba440841e0 con 0x7fba34005190 2024-05-08T23:30:44.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.537+0000 7fba4dd06700 1 --2- 172.21.15.81:0/2586334215 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fba2c066730 0x7fba2c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.537+0000 7fba4dd06700 1 --2- 172.21.15.81:0/2586334215 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fba2c066730 0x7fba2c068bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fba480ac810 tx=0x7fba4000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.541+0000 7f35e17fa700 1 -- 172.21.15.81:0/3524721248 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35d8055420 con 0x7f35e4148890 2024-05-08T23:30:44.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.541+0000 7fb45cff9700 1 -- 172.21.15.81:0/3470069854 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb458021070 con 0x7fb460070110 2024-05-08T23:30:44.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.541+0000 7fb45cff9700 1 -- 172.21.15.81:0/3470069854 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458004620 con 0x7fb460070110 2024-05-08T23:30:44.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.541+0000 7fba5076b700 1 -- 172.21.15.81:0/2586334215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba480684b0 con 0x7fba34005190 2024-05-08T23:30:44.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.541+0000 7f35e17fa700 1 -- 172.21.15.81:0/3524721248 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f35d8051060 con 0x7f35e4148890 2024-05-08T23:30:44.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.545+0000 7f93667fc700 1 -- 172.21.15.81:0/3882969880 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9360051890 con 0x7f93701609f0 2024-05-08T23:30:44.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.545+0000 7fb465693700 1 -- 172.21.15.81:0/3470069854 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb440005310 con 0x7fb460070110 2024-05-08T23:30:44.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.549+0000 7fb45cff9700 1 -- 172.21.15.81:0/3470069854 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb458023030 con 0x7fb460070110 2024-05-08T23:30:44.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.549+0000 7fb45cff9700 1 --2- 172.21.15.81:0/3470069854 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb44c066730 0x7fb44c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:44.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.549+0000 7fb45f7fe700 1 --2- 172.21.15.81:0/3470069854 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb44c066730 0x7fb44c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:44.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.549+0000 7fb45f7fe700 1 --2- 172.21.15.81:0/3470069854 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb44c066730 0x7fb44c068bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb46014a690 tx=0x7fb454009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:44.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.553+0000 7fb45cff9700 1 -- 172.21.15.81:0/3470069854 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4580579d0 con 0x7fb460070110 2024-05-08T23:30:44.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.553+0000 7fba3effd700 1 -- 172.21.15.81:0/2586334215 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fba440514d0 con 0x7fba34005190 2024-05-08T23:30:44.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.561+0000 7fb45cff9700 1 -- 172.21.15.81:0/3470069854 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb458053610 con 0x7fb460070110 2024-05-08T23:30:44.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.833+0000 7f74f8d00700 1 -- 172.21.15.81:0/2833606116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f74f40684b0 con 0x7f74f406e3a0 2024-05-08T23:30:44.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.833+0000 7f74dbfff700 1 -- 172.21.15.81:0/2833606116 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f74e80505a0 con 0x7f74f406e3a0 2024-05-08T23:30:44.841 INFO:teuthology.orchestra.run.smithi081.stdout:64424509445 2024-05-08T23:30:44.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.841+0000 7f74d9ffb700 1 -- 172.21.15.81:0/2833606116 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f74e0066780 msgr2=0x7f74e0068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.841+0000 7f74d9ffb700 1 --2- 172.21.15.81:0/2833606116 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f74e0066780 0x7f74e0068c40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f74f4161420 tx=0x7f74ec00b040 comp rx=0 tx=0).stop 2024-05-08T23:30:44.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.841+0000 7f74d9ffb700 1 -- 172.21.15.81:0/2833606116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74f406e3a0 msgr2=0x7f74f415bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.841+0000 7f74d9ffb700 1 --2- 172.21.15.81:0/2833606116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74f406e3a0 0x7f74f415bc10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f74e800c4d0 tx=0x7f74e800c7e0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.841+0000 7f74d9ffb700 1 -- 172.21.15.81:0/2833606116 shutdown_connections 2024-05-08T23:30:44.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.841+0000 7f74d9ffb700 1 --2- 172.21.15.81:0/2833606116 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f74e0066780 0x7f74e0068c40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.841+0000 7f74d9ffb700 1 --2- 172.21.15.81:0/2833606116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74f40cb3e0 0x7f74f4207980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.841+0000 7f74d9ffb700 1 --2- 172.21.15.81:0/2833606116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74f40b4310 0x7f74f4159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.841+0000 7f74d9ffb700 1 --2- 172.21.15.81:0/2833606116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74f406e3a0 0x7f74f415bc10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.841+0000 7f74d9ffb700 1 -- 172.21.15.81:0/2833606116 >> 172.21.15.81:0/2833606116 conn(0x7f74f40aebd0 msgr2=0x7f74f4070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.841+0000 7f74d9ffb700 1 -- 172.21.15.81:0/2833606116 shutdown_connections 2024-05-08T23:30:44.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.841+0000 7f74d9ffb700 1 -- 172.21.15.81:0/2833606116 wait complete. 2024-05-08T23:30:44.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.845+0000 7f846d6f2700 1 -- 172.21.15.81:0/1558298486 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f84681f2f90 con 0x7f846808a620 2024-05-08T23:30:44.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.845+0000 7f8464ff9700 1 -- 172.21.15.81:0/1558298486 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f84681f2f90 con 0x7f846808a620 2024-05-08T23:30:44.853 INFO:teuthology.orchestra.run.smithi081.stdout:64424509447 2024-05-08T23:30:44.853 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.849+0000 7efc14d64700 1 -- 172.21.15.81:0/522458045 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efbf0005890 con 0x7efc10160e30 2024-05-08T23:30:44.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.849+0000 7efbf7fff700 1 -- 172.21.15.81:0/522458045 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efc08025090 con 0x7efc10160e30 2024-05-08T23:30:44.854 INFO:teuthology.orchestra.run.smithi081.stdout:64424509445 2024-05-08T23:30:44.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7f844e7fc700 1 -- 172.21.15.81:0/1558298486 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8454066730 msgr2=0x7f8454068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7f844e7fc700 1 --2- 172.21.15.81:0/1558298486 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8454066730 0x7f8454068bf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f846000b0f0 tx=0x7f846000b000 comp rx=0 tx=0).stop 2024-05-08T23:30:44.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7f844e7fc700 1 -- 172.21.15.81:0/1558298486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f846808a620 msgr2=0x7f84681f3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7f844e7fc700 1 --2- 172.21.15.81:0/1558298486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f846808a620 0x7f84681f3270 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f845c00d860 tx=0x7f845c00dc20 comp rx=0 tx=0).stop 2024-05-08T23:30:44.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7f844e7fc700 1 -- 172.21.15.81:0/1558298486 shutdown_connections 2024-05-08T23:30:44.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7f844e7fc700 1 --2- 172.21.15.81:0/1558298486 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8454066730 0x7f8454068bf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7f844e7fc700 1 --2- 172.21.15.81:0/1558298486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f846816d820 0x7f8468213f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7f844e7fc700 1 --2- 172.21.15.81:0/1558298486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84680b3c40 0x7f8468071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7f844e7fc700 1 --2- 172.21.15.81:0/1558298486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f846808a620 0x7f84681f3270 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7f844e7fc700 1 -- 172.21.15.81:0/1558298486 >> 172.21.15.81:0/1558298486 conn(0x7f84680aebd0 msgr2=0x7f84680702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7f844e7fc700 1 -- 172.21.15.81:0/1558298486 shutdown_connections 2024-05-08T23:30:44.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7f844e7fc700 1 -- 172.21.15.81:0/1558298486 wait complete. 2024-05-08T23:30:44.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7efc14d64700 1 -- 172.21.15.81:0/522458045 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efbfc066780 msgr2=0x7efbfc068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7efc14d64700 1 --2- 172.21.15.81:0/522458045 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efbfc066780 0x7efbfc068c40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efc00000c00 tx=0x7efc00001c20 comp rx=0 tx=0).stop 2024-05-08T23:30:44.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7efc14d64700 1 -- 172.21.15.81:0/522458045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc10160e30 msgr2=0x7efc101612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7efc14d64700 1 --2- 172.21.15.81:0/522458045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc10160e30 0x7efc101612b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efc10158be0 tx=0x7efc08004c00 comp rx=0 tx=0).stop 2024-05-08T23:30:44.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7efc14d64700 1 -- 172.21.15.81:0/522458045 shutdown_connections 2024-05-08T23:30:44.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7efc14d64700 1 --2- 172.21.15.81:0/522458045 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efbfc066780 0x7efbfc068c40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7efc14d64700 1 --2- 172.21.15.81:0/522458045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc10160e30 0x7efc101612b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.860 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2024-05-08T23:30:44.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7efc14d64700 1 --2- 172.21.15.81:0/522458045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc101554e0 0x7efc10154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7efc14d64700 1 --2- 172.21.15.81:0/522458045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbf8005430 0x7efc10155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7efc14d64700 1 -- 172.21.15.81:0/522458045 >> 172.21.15.81:0/522458045 conn(0x7efc100aebd0 msgr2=0x7efc1015ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7efc14d64700 1 -- 172.21.15.81:0/522458045 shutdown_connections 2024-05-08T23:30:44.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.853+0000 7efc14d64700 1 -- 172.21.15.81:0/522458045 wait complete. 2024-05-08T23:30:44.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.857+0000 7faaa0d3a700 1 -- 172.21.15.81:0/2949169489 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faa9c159590 con 0x7faa9c0b4140 2024-05-08T23:30:44.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.857+0000 7faa83fff700 1 -- 172.21.15.81:0/2949169489 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faa8c015090 con 0x7faa9c0b4140 2024-05-08T23:30:44.862 INFO:teuthology.orchestra.run.smithi081.stdout:60129542149 2024-05-08T23:30:44.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.861+0000 7faaa0d3a700 1 -- 172.21.15.81:0/2949169489 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa88066730 msgr2=0x7faa88068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.861+0000 7faaa0d3a700 1 --2- 172.21.15.81:0/2949169489 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa88066730 0x7faa88068bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa90009930 tx=0x7faa90008040 comp rx=0 tx=0).stop 2024-05-08T23:30:44.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.861+0000 7faaa0d3a700 1 -- 172.21.15.81:0/2949169489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa9c0b4140 msgr2=0x7faa9c159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.861+0000 7faaa0d3a700 1 --2- 172.21.15.81:0/2949169489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa9c0b4140 0x7faa9c159890 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faa8c000c00 tx=0x7faa8c004770 comp rx=0 tx=0).stop 2024-05-08T23:30:44.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.861+0000 7faaa0d3a700 1 -- 172.21.15.81:0/2949169489 shutdown_connections 2024-05-08T23:30:44.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.861+0000 7faaa0d3a700 1 --2- 172.21.15.81:0/2949169489 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa88066730 0x7faa88068bf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.861+0000 7faaa0d3a700 1 --2- 172.21.15.81:0/2949169489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa9c0cb3e0 0x7faa9c0cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.861+0000 7faaa0d3a700 1 --2- 172.21.15.81:0/2949169489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa9c0b4140 0x7faa9c159890 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.861+0000 7faaa0d3a700 1 --2- 172.21.15.81:0/2949169489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa9c06e3a0 0x7faa9c154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.861+0000 7faaa0d3a700 1 -- 172.21.15.81:0/2949169489 >> 172.21.15.81:0/2949169489 conn(0x7faa9c0aebd0 msgr2=0x7faa9c070e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.861+0000 7faaa0d3a700 1 -- 172.21.15.81:0/2949169489 shutdown_connections 2024-05-08T23:30:44.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.861+0000 7faaa0d3a700 1 -- 172.21.15.81:0/2949169489 wait complete. 2024-05-08T23:30:44.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.861+0000 7f35e9a18700 1 -- 172.21.15.81:0/3524721248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f35e4144ae0 con 0x7f35e4148890 2024-05-08T23:30:44.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.865+0000 7f35e17fa700 1 -- 172.21.15.81:0/3524721248 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f35d8017020 con 0x7f35e4148890 2024-05-08T23:30:44.868 INFO:teuthology.orchestra.run.smithi081.stdout:64424509445 2024-05-08T23:30:44.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.865+0000 7f9377e80700 1 -- 172.21.15.81:0/3882969880 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9358005890 con 0x7f93701609f0 2024-05-08T23:30:44.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.865+0000 7f93667fc700 1 -- 172.21.15.81:0/3882969880 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9360054eb0 con 0x7f93701609f0 2024-05-08T23:30:44.870 INFO:teuthology.orchestra.run.smithi081.stdout:64424509447 2024-05-08T23:30:44.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f35e9a18700 1 -- 172.21.15.81:0/3524721248 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f35d0066730 msgr2=0x7f35d0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f35e9a18700 1 --2- 172.21.15.81:0/3524721248 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f35d0066730 0x7f35d0068bf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f35d4001eb0 tx=0x7f35d400a000 comp rx=0 tx=0).stop 2024-05-08T23:30:44.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f35e9a18700 1 -- 172.21.15.81:0/3524721248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35e4148890 msgr2=0x7f35e4143f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f35e9a18700 1 --2- 172.21.15.81:0/3524721248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35e4148890 0x7f35e4143f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f35e40692d0 tx=0x7f35d800e840 comp rx=0 tx=0).stop 2024-05-08T23:30:44.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.5 2024-05-08T23:30:44.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f35e9a18700 1 -- 172.21.15.81:0/3524721248 shutdown_connections 2024-05-08T23:30:44.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f35e9a18700 1 --2- 172.21.15.81:0/3524721248 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f35d0066730 0x7f35d0068bf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f35e9a18700 1 --2- 172.21.15.81:0/3524721248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35e4144440 0x7f35e415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f35e9a18700 1 --2- 172.21.15.81:0/3524721248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35e4148890 0x7f35e4143f00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f35e9a18700 1 --2- 172.21.15.81:0/3524721248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e41484b0 0x7f35e4151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f35e9a18700 1 -- 172.21.15.81:0/3524721248 >> 172.21.15.81:0/3524721248 conn(0x7f35e40aebd0 msgr2=0x7f35e4071280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f35e9a18700 1 -- 172.21.15.81:0/3524721248 shutdown_connections 2024-05-08T23:30:44.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f9377e80700 1 -- 172.21.15.81:0/3882969880 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9354066760 msgr2=0x7f9354068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f9377e80700 1 --2- 172.21.15.81:0/3882969880 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9354066760 0x7f9354068c20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f936c000c00 tx=0x7f936c01d040 comp rx=0 tx=0).stop 2024-05-08T23:30:44.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f9377e80700 1 -- 172.21.15.81:0/3882969880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93701609f0 msgr2=0x7f9370160e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f9377e80700 1 --2- 172.21.15.81:0/3882969880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93701609f0 0x7f9370160e70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f936000f870 tx=0x7f936000fb80 comp rx=0 tx=0).stop 2024-05-08T23:30:44.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f35e9a18700 1 -- 172.21.15.81:0/3524721248 wait complete. 2024-05-08T23:30:44.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f9377e80700 1 -- 172.21.15.81:0/3882969880 shutdown_connections 2024-05-08T23:30:44.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f9377e80700 1 --2- 172.21.15.81:0/3882969880 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9354066760 0x7f9354068c20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f9377e80700 1 --2- 172.21.15.81:0/3882969880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f937015af90 0x7f937015b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f9377e80700 1 --2- 172.21.15.81:0/3882969880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93701609f0 0x7f9370160e70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f9377e80700 1 --2- 172.21.15.81:0/3882969880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f935c003a20 0x7f9370141220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f9377e80700 1 -- 172.21.15.81:0/3882969880 >> 172.21.15.81:0/3882969880 conn(0x7f937006a360 msgr2=0x7f93700b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f9377e80700 1 -- 172.21.15.81:0/3882969880 shutdown_connections 2024-05-08T23:30:44.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.869+0000 7f9377e80700 1 -- 172.21.15.81:0/3882969880 wait complete. 2024-05-08T23:30:44.876 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.2 2024-05-08T23:30:44.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.873+0000 7fba5076b700 1 -- 172.21.15.81:0/2586334215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fba4806c5e0 con 0x7fba34005190 2024-05-08T23:30:44.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.873+0000 7fba3effd700 1 -- 172.21.15.81:0/2586334215 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba44054af0 con 0x7fba34005190 2024-05-08T23:30:44.878 INFO:teuthology.orchestra.run.smithi081.stdout:64424509445 2024-05-08T23:30:44.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.0 2024-05-08T23:30:44.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.877+0000 7fba5076b700 1 -- 172.21.15.81:0/2586334215 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fba2c066730 msgr2=0x7fba2c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.877+0000 7fba5076b700 1 --2- 172.21.15.81:0/2586334215 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fba2c066730 0x7fba2c068bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fba480ac810 tx=0x7fba4000b040 comp rx=0 tx=0).stop 2024-05-08T23:30:44.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.877+0000 7fba5076b700 1 -- 172.21.15.81:0/2586334215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba34005190 msgr2=0x7fba48160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.877+0000 7fba5076b700 1 --2- 172.21.15.81:0/2586334215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba34005190 0x7fba48160080 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fba44007310 tx=0x7fba44007620 comp rx=0 tx=0).stop 2024-05-08T23:30:44.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.877+0000 7fba5076b700 1 -- 172.21.15.81:0/2586334215 shutdown_connections 2024-05-08T23:30:44.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.877+0000 7fba5076b700 1 --2- 172.21.15.81:0/2586334215 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fba2c066730 0x7fba2c068bf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.877+0000 7fba5076b700 1 --2- 172.21.15.81:0/2586334215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba4815a170 0x7fba481377d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.877+0000 7fba5076b700 1 --2- 172.21.15.81:0/2586334215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba48156d30 0x7fba48159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.877+0000 7fba5076b700 1 --2- 172.21.15.81:0/2586334215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba34005190 0x7fba48160080 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.877+0000 7fba5076b700 1 -- 172.21.15.81:0/2586334215 >> 172.21.15.81:0/2586334215 conn(0x7fba480aebd0 msgr2=0x7fba48070fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.877+0000 7fba5076b700 1 -- 172.21.15.81:0/2586334215 shutdown_connections 2024-05-08T23:30:44.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.877+0000 7fba5076b700 1 -- 172.21.15.81:0/2586334215 wait complete. 2024-05-08T23:30:44.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.877+0000 7fb465693700 1 -- 172.21.15.81:0/3470069854 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb440005890 con 0x7fb460070110 2024-05-08T23:30:44.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.877+0000 7fb45cff9700 1 -- 172.21.15.81:0/3470069854 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb458056c30 con 0x7fb460070110 2024-05-08T23:30:44.885 INFO:teuthology.orchestra.run.smithi081.stdout:60129542149 2024-05-08T23:30:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.881+0000 7fb465693700 1 -- 172.21.15.81:0/3470069854 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb44c066730 msgr2=0x7fb44c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.881+0000 7fb465693700 1 --2- 172.21.15.81:0/3470069854 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb44c066730 0x7fb44c068bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb46014a690 tx=0x7fb454009040 comp rx=0 tx=0).stop 2024-05-08T23:30:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.881+0000 7fb465693700 1 -- 172.21.15.81:0/3470069854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb460070110 msgr2=0x7fb46006c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.881+0000 7fb465693700 1 --2- 172.21.15.81:0/3470069854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb460070110 0x7fb46006c980 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb45800b370 tx=0x7fb458004af0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.881+0000 7fb465693700 1 -- 172.21.15.81:0/3470069854 shutdown_connections 2024-05-08T23:30:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.881+0000 7fb465693700 1 --2- 172.21.15.81:0/3470069854 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb44c066730 0x7fb44c068bf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.881+0000 7fb465693700 1 --2- 172.21.15.81:0/3470069854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb46013a5d0 0x7fb4601641c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.881+0000 7fb465693700 1 --2- 172.21.15.81:0/3470069854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb460071e70 0x7fb46013a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.881+0000 7fb465693700 1 --2- 172.21.15.81:0/3470069854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb460070110 0x7fb46006c980 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:44.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.881+0000 7fb465693700 1 -- 172.21.15.81:0/3470069854 >> 172.21.15.81:0/3470069854 conn(0x7fb46006c330 msgr2=0x7fb4600b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:44.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.881+0000 7fb465693700 1 -- 172.21.15.81:0/3470069854 shutdown_connections 2024-05-08T23:30:44.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:44.881+0000 7fb465693700 1 -- 172.21.15.81:0/3470069854 wait complete. 2024-05-08T23:30:44.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.6 2024-05-08T23:30:44.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.3 2024-05-08T23:30:44.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.1 2024-05-08T23:30:44.899 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.7 2024-05-08T23:30:45.861 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T23:30:45.873 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T23:30:45.877 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T23:30:45.883 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T23:30:45.890 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T23:30:45.890 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T23:30:45.897 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T23:30:45.901 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T23:30:46.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:45.997+0000 7f73d67da700 1 Processor -- start 2024-05-08T23:30:46.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:45.997+0000 7f73d67da700 1 -- start start 2024-05-08T23:30:46.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73d67da700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f73d00b4310 0x7f73d0165ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73d67da700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d006e3a0 0x7f73d006c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73d67da700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d0162700 con 0x7f73d00cb3e0 2024-05-08T23:30:46.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73d67da700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d0161dd0 con 0x7f73d00b4310 2024-05-08T23:30:46.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73d67da700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d01620b0 con 0x7f73d006e3a0 2024-05-08T23:30:46.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73cffff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f73d00cb3e0 0x7f73d00b3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38656/0 (socket says 172.21.15.81:38656) 2024-05-08T23:30:46.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73cffff700 1 -- 172.21.15.81:0/2288054578 learned_addr learned my addr 172.21.15.81:0/2288054578 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:46.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73d4d77700 1 --2- 172.21.15.81:0/2288054578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d006e3a0 0x7f73d006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73ceffd700 1 -- 172.21.15.81:0/2288054578 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44201822 0 0) 0x7f73d0162700 con 0x7f73d00cb3e0 2024-05-08T23:30:46.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73cf7fe700 1 --2- 172.21.15.81:0/2288054578 >> v2:172.21.15.81:3300/0 conn(0x7f73d00b4310 0x7f73d0165ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73d4d77700 1 -- 172.21.15.81:0/2288054578 >> v2:172.21.15.81:3300/0 conn(0x7f73d00b4310 msgr2=0x7f73d0165ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73d4d77700 1 --2- 172.21.15.81:0/2288054578 >> v2:172.21.15.81:3300/0 conn(0x7f73d00b4310 0x7f73d0165ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73d4d77700 1 -- 172.21.15.81:0/2288054578 >> v1:172.21.15.81:6789/0 conn(0x7f73d00cb3e0 legacy=0x7f73d00b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73d4d77700 1 -- 172.21.15.81:0/2288054578 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d0162390 con 0x7f73d006e3a0 2024-05-08T23:30:46.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73cf7fe700 1 --2- 172.21.15.81:0/2288054578 >> v2:172.21.15.81:3300/0 conn(0x7f73d00b4310 0x7f73d0165ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:46.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73d4d77700 1 --2- 172.21.15.81:0/2288054578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d006e3a0 0x7f73d006c330 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009d90 tx=0x7f73c800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18dec1e92da1f90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73ceffd700 1 -- 172.21.15.81:0/2288054578 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c800e070 con 0x7f73d006e3a0 2024-05-08T23:30:46.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73ceffd700 1 -- 172.21.15.81:0/2288054578 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c800bbd0 con 0x7f73d006e3a0 2024-05-08T23:30:46.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73ceffd700 1 -- 172.21.15.81:0/2288054578 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8012430 con 0x7f73d006e3a0 2024-05-08T23:30:46.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73d67da700 1 -- 172.21.15.81:0/2288054578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d006e3a0 msgr2=0x7f73d006c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.001+0000 7f73d67da700 1 --2- 172.21.15.81:0/2288054578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d006e3a0 0x7f73d006c330 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009d90 tx=0x7f73c800b330 comp rx=0 tx=0).stop 2024-05-08T23:30:46.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d67da700 1 -- 172.21.15.81:0/2288054578 shutdown_connections 2024-05-08T23:30:46.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d67da700 1 --2- 172.21.15.81:0/2288054578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d006e3a0 0x7f73d006c330 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d67da700 1 --2- 172.21.15.81:0/2288054578 >> v2:172.21.15.81:3300/0 conn(0x7f73d00b4310 0x7f73d0165ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.009 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d67da700 1 -- 172.21.15.81:0/2288054578 >> 172.21.15.81:0/2288054578 conn(0x7f73d00aebd0 msgr2=0x7f73d00aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.009 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d67da700 1 -- 172.21.15.81:0/2288054578 shutdown_connections 2024-05-08T23:30:46.009 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d67da700 1 -- 172.21.15.81:0/2288054578 wait complete. 2024-05-08T23:30:46.009 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d67da700 1 Processor -- start 2024-05-08T23:30:46.009 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d67da700 1 -- start start 2024-05-08T23:30:46.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d67da700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d006e3a0 0x7f73d01466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d67da700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d00b4310 0x7f73d0071600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d67da700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d00cb3e0 0x7f73d0071b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d67da700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d0155400 con 0x7f73d006e3a0 2024-05-08T23:30:46.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d67da700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d0135ac0 con 0x7f73d00b4310 2024-05-08T23:30:46.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d67da700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d0146c00 con 0x7f73d00cb3e0 2024-05-08T23:30:46.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73cffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d006e3a0 0x7f73d01466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73cffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d006e3a0 0x7f73d01466c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51034/0 (socket says 172.21.15.81:51034) 2024-05-08T23:30:46.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73cffff700 1 -- 172.21.15.81:0/3951275427 learned_addr learned my addr 172.21.15.81:0/3951275427 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:46.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73cf7fe700 1 --2- 172.21.15.81:0/3951275427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d00b4310 0x7f73d0071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d4d77700 1 --2- 172.21.15.81:0/3951275427 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d00cb3e0 0x7f73d0071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73cffff700 1 -- 172.21.15.81:0/3951275427 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d00cb3e0 msgr2=0x7f73d0071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73cffff700 1 --2- 172.21.15.81:0/3951275427 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d00cb3e0 0x7f73d0071b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73cffff700 1 -- 172.21.15.81:0/3951275427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d00b4310 msgr2=0x7f73d0071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73cffff700 1 --2- 172.21.15.81:0/3951275427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d00b4310 0x7f73d0071600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73cffff700 1 -- 172.21.15.81:0/3951275427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c8009a70 con 0x7f73d006e3a0 2024-05-08T23:30:46.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.005+0000 7f73d4d77700 1 --2- 172.21.15.81:0/3951275427 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d00cb3e0 0x7f73d0071b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:46.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.009+0000 7f73cffff700 1 --2- 172.21.15.81:0/3951275427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d006e3a0 0x7f73d01466c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f73c000c550 tx=0x7f73c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.013+0000 7f73ccff9700 1 -- 172.21.15.81:0/3951275427 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c0004ca0 con 0x7f73d006e3a0 2024-05-08T23:30:46.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.013+0000 7f73d67da700 1 -- 172.21.15.81:0/3951275427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d0135d10 con 0x7f73d006e3a0 2024-05-08T23:30:46.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.013+0000 7f73d67da700 1 -- 172.21.15.81:0/3951275427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d01362d0 con 0x7f73d006e3a0 2024-05-08T23:30:46.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.013+0000 7f73ccff9700 1 -- 172.21.15.81:0/3951275427 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c0004e00 con 0x7f73d006e3a0 2024-05-08T23:30:46.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.013+0000 7f73ccff9700 1 -- 172.21.15.81:0/3951275427 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c0007310 con 0x7f73d006e3a0 2024-05-08T23:30:46.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.013+0000 7f73ccff9700 1 -- 172.21.15.81:0/3951275427 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f73c0007530 con 0x7f73d006e3a0 2024-05-08T23:30:46.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.013+0000 7f73ccff9700 1 --2- 172.21.15.81:0/3951275427 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f73bc088890 0x7f73bc08ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.013+0000 7f73cf7fe700 1 --2- 172.21.15.81:0/3951275427 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f73bc088890 0x7f73bc08ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.013+0000 7f73ccff9700 1 -- 172.21.15.81:0/3951275427 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73c0083fa0 con 0x7f73d006e3a0 2024-05-08T23:30:46.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.013+0000 7f73cf7fe700 1 --2- 172.21.15.81:0/3951275427 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f73bc088890 0x7f73bc08ad50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f73d0164fb0 tx=0x7f73c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.013+0000 7f73d67da700 1 -- 172.21.15.81:0/3951275427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b0005310 con 0x7f73d006e3a0 2024-05-08T23:30:46.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f970d0dc700 1 Processor -- start 2024-05-08T23:30:46.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f970d0dc700 1 -- start start 2024-05-08T23:30:46.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f970d0dc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97080b1f60 0x7f970806e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f970d0dc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f97080ac8e0 0x7f97080acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f970d0dc700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97080b2740 con 0x7f9708160c80 2024-05-08T23:30:46.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f970d0dc700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970806d990 con 0x7f97080ac8e0 2024-05-08T23:30:46.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f970d0dc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9708146d60 con 0x7f97080b1f60 2024-05-08T23:30:46.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f9706d9d700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f9708160c80 0x7f9708161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38672/0 (socket says 172.21.15.81:38672) 2024-05-08T23:30:46.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f9706d9d700 1 -- 172.21.15.81:0/2000137167 learned_addr learned my addr 172.21.15.81:0/2000137167 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:46.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f970759e700 1 --2- 172.21.15.81:0/2000137167 >> v2:172.21.15.117:3300/0 conn(0x7f97080ac8e0 0x7f97080acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f970759e700 1 -- 172.21.15.81:0/2000137167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97080b1f60 msgr2=0x7f970806e4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:30:46.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f970759e700 1 --2- 172.21.15.81:0/2000137167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97080b1f60 0x7f970806e4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f970759e700 1 -- 172.21.15.81:0/2000137167 >> v1:172.21.15.81:6789/0 conn(0x7f9708160c80 legacy=0x7f9708161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f970759e700 1 -- 172.21.15.81:0/2000137167 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97080b2460 con 0x7f97080ac8e0 2024-05-08T23:30:46.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f970759e700 1 --2- 172.21.15.81:0/2000137167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97080ac8e0 0x7f97080acd70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009850 tx=0x7f96f8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34a8b0d50b40d5e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7fbb9bfff700 1 Processor -- start 2024-05-08T23:30:46.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f73ccff9700 1 -- 172.21.15.81:0/3951275427 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f73c0051380 con 0x7f73d006e3a0 2024-05-08T23:30:46.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f9705d9b700 1 -- 172.21.15.81:0/2000137167 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8011830 con 0x7f97080ac8e0 2024-05-08T23:30:46.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f9705d9b700 1 -- 172.21.15.81:0/2000137167 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f8011990 con 0x7f97080ac8e0 2024-05-08T23:30:46.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7f9705d9b700 1 -- 172.21.15.81:0/2000137167 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8011d20 con 0x7f97080ac8e0 2024-05-08T23:30:46.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7f970d0dc700 1 -- 172.21.15.81:0/2000137167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97080ac8e0 msgr2=0x7f97080acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7f970d0dc700 1 --2- 172.21.15.81:0/2000137167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97080ac8e0 0x7f97080acd70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009850 tx=0x7f96f8009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:46.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7f970d0dc700 1 -- 172.21.15.81:0/2000137167 shutdown_connections 2024-05-08T23:30:46.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7f970d0dc700 1 --2- 172.21.15.81:0/2000137167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97080ac8e0 0x7f97080acd70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7f970d0dc700 1 --2- 172.21.15.81:0/2000137167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97080b1f60 0x7f970806e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7f970d0dc700 1 -- 172.21.15.81:0/2000137167 >> 172.21.15.81:0/2000137167 conn(0x7f97080aebd0 msgr2=0x7f97080aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.021+0000 7fbb9bfff700 1 -- start start 2024-05-08T23:30:46.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb9bfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb9c0b3c20 0x7fbb9c1624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb9bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbb9c071e70 0x7fbb9c0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb9bfff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb9c158960 con 0x7fbb9c165070 2024-05-08T23:30:46.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb9bfff700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb9c0b1f60 con 0x7fbb9c0b3c20 2024-05-08T23:30:46.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb9bfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb9c071120 con 0x7fbb9c071e70 2024-05-08T23:30:46.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb9affd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb9c0b3c20 0x7fbb9c1624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb9affd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb9c0b3c20 0x7fbb9c1624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51038/0 (socket says 172.21.15.81:51038) 2024-05-08T23:30:46.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb9affd700 1 -- 172.21.15.81:0/3735086633 learned_addr learned my addr 172.21.15.81:0/3735086633 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:46.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb9a7fc700 1 --2- 172.21.15.81:0/3735086633 >> v2:172.21.15.117:3300/0 conn(0x7fbb9c071e70 0x7fbb9c0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb9affd700 1 -- 172.21.15.81:0/3735086633 >> v2:172.21.15.117:3300/0 conn(0x7fbb9c071e70 msgr2=0x7fbb9c0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb9affd700 1 --2- 172.21.15.81:0/3735086633 >> v2:172.21.15.117:3300/0 conn(0x7fbb9c071e70 0x7fbb9c0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb9affd700 1 -- 172.21.15.81:0/3735086633 >> v1:172.21.15.117:6789/0 conn(0x7fbb9c165070 legacy=0x7fbb9c1654e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb9affd700 1 -- 172.21.15.81:0/3735086633 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c0b4540 con 0x7fbb9c0b3c20 2024-05-08T23:30:46.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb9a7fc700 1 --2- 172.21.15.81:0/3735086633 >> v2:172.21.15.117:3300/0 conn(0x7fbb9c071e70 0x7fbb9c0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:46.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7f970d0dc700 1 -- 172.21.15.81:0/2000137167 shutdown_connections 2024-05-08T23:30:46.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7f970d0dc700 1 -- 172.21.15.81:0/2000137167 wait complete. 2024-05-08T23:30:46.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970d0dc700 1 Processor -- start 2024-05-08T23:30:46.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb9affd700 1 --2- 172.21.15.81:0/3735086633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9c0b3c20 0x7fbb9c1624b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c06b800 tx=0x7fbb8c009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da48c60c56a6fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb99ffb700 1 -- 172.21.15.81:0/3735086633 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c00b910 con 0x7fbb9c0b3c20 2024-05-08T23:30:46.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.025+0000 7fbb99ffb700 1 -- 172.21.15.81:0/3735086633 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8c00ba70 con 0x7fbb9c0b3c20 2024-05-08T23:30:46.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7fbb9bfff700 1 -- 172.21.15.81:0/3735086633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9c0b3c20 msgr2=0x7fbb9c1624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7fbb9bfff700 1 --2- 172.21.15.81:0/3735086633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9c0b3c20 0x7fbb9c1624b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c06b800 tx=0x7fbb8c009e50 comp rx=0 tx=0).stop 2024-05-08T23:30:46.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7fbb9bfff700 1 -- 172.21.15.81:0/3735086633 shutdown_connections 2024-05-08T23:30:46.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7fbb9bfff700 1 --2- 172.21.15.81:0/3735086633 >> v2:172.21.15.117:3300/0 conn(0x7fbb9c071e70 0x7fbb9c0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7fbb9bfff700 1 --2- 172.21.15.81:0/3735086633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9c0b3c20 0x7fbb9c1624b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7fbb9bfff700 1 -- 172.21.15.81:0/3735086633 >> 172.21.15.81:0/3735086633 conn(0x7fbb9c06c330 msgr2=0x7fbb9c06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7fbb9bfff700 1 -- 172.21.15.81:0/3735086633 shutdown_connections 2024-05-08T23:30:46.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970d0dc700 1 -- start start 2024-05-08T23:30:46.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970d0dc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97080ac8e0 0x7f970815f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970d0dc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9708160c80 0x7f97081e6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970d0dc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97081e6ff0 0x7f97080aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970d0dc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97081e7550 con 0x7f9708160c80 2024-05-08T23:30:46.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970d0dc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97081e7830 con 0x7f97081e6ff0 2024-05-08T23:30:46.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970d0dc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97081e7b10 con 0x7f97080ac8e0 2024-05-08T23:30:46.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f9706d9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97080ac8e0 0x7f970815f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f9706d9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97080ac8e0 0x7f970815f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:40918/0 (socket says 172.21.15.81:40918) 2024-05-08T23:30:46.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f9706d9d700 1 -- 172.21.15.81:0/512131939 learned_addr learned my addr 172.21.15.81:0/512131939 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:46.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970759e700 1 --2- 172.21.15.81:0/512131939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97081e6ff0 0x7f97080aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7fbb9bfff700 1 -- 172.21.15.81:0/3735086633 wait complete. 2024-05-08T23:30:46.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970659c700 1 --2- 172.21.15.81:0/512131939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9708160c80 0x7f97081e6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f23b2c49700 1 Processor -- start 2024-05-08T23:30:46.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f23b2c49700 1 -- start start 2024-05-08T23:30:46.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7fbb9bfff700 1 Processor -- start 2024-05-08T23:30:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7fbb9bfff700 1 -- start start 2024-05-08T23:30:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970659c700 1 -- 172.21.15.81:0/512131939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97080ac8e0 msgr2=0x7f970815f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970659c700 1 --2- 172.21.15.81:0/512131939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97080ac8e0 0x7f970815f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970659c700 1 -- 172.21.15.81:0/512131939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97081e6ff0 msgr2=0x7f97080aaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970659c700 1 --2- 172.21.15.81:0/512131939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97081e6ff0 0x7f97080aaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970659c700 1 -- 172.21.15.81:0/512131939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f80094b0 con 0x7f9708160c80 2024-05-08T23:30:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970659c700 1 --2- 172.21.15.81:0/512131939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9708160c80 0x7f97081e6ab0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9708072900 tx=0x7f9700009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f96ef7fe700 1 -- 172.21.15.81:0/512131939 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970000cbb0 con 0x7f9708160c80 2024-05-08T23:30:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f96ef7fe700 1 -- 172.21.15.81:0/512131939 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970000cd50 con 0x7f9708160c80 2024-05-08T23:30:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f96ef7fe700 1 -- 172.21.15.81:0/512131939 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700014680 con 0x7f9708160c80 2024-05-08T23:30:46.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f23b2c49700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f23ac06d820 0x7f23ac0ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f23b2c49700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f23ac0b1f60 0x7f23ac0b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f23b2c49700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ac0b2d90 con 0x7f23ac0cdc20 2024-05-08T23:30:46.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f23b2c49700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23ac150680 con 0x7f23ac0b1f60 2024-05-08T23:30:46.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f23b2c49700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23ac06dcd0 con 0x7f23ac06d820 2024-05-08T23:30:46.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f23b1446700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f23ac06d820 0x7f23ac0ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f23b1446700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f23ac06d820 0x7f23ac0ce000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:53774/0 (socket says 172.21.15.81:53774) 2024-05-08T23:30:46.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f23b1446700 1 -- 172.21.15.81:0/3168568359 learned_addr learned my addr 172.21.15.81:0/3168568359 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:46.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7fbb9bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9c071e70 0x7fbb9c15b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7fbb9bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb9c0b3c20 0x7fbb9c0afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9bfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb9c165070 0x7fbb9c0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9bfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c15b170 con 0x7fbb9c071e70 2024-05-08T23:30:46.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9bfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c15a810 con 0x7fbb9c0b3c20 2024-05-08T23:30:46.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9bfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c15aab0 con 0x7fbb9c165070 2024-05-08T23:30:46.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970d0dc700 1 -- 172.21.15.81:0/512131939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97081e7df0 con 0x7f9708160c80 2024-05-08T23:30:46.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.029+0000 7f970d0dc700 1 -- 172.21.15.81:0/512131939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970815fea0 con 0x7f9708160c80 2024-05-08T23:30:46.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f23b1446700 1 -- 172.21.15.81:0/3168568359 >> v2:172.21.15.81:3300/0 conn(0x7f23ac0b1f60 msgr2=0x7f23ac0b23d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:46.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f23b1446700 1 --2- 172.21.15.81:0/3168568359 >> v2:172.21.15.81:3300/0 conn(0x7f23ac0b1f60 0x7f23ac0b23d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f23b1446700 1 -- 172.21.15.81:0/3168568359 >> v1:172.21.15.117:6789/0 conn(0x7f23ac0cdc20 legacy=0x7f23ac15a150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:46.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f23b1446700 1 -- 172.21.15.81:0/3168568359 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac06d390 con 0x7f23ac06d820 2024-05-08T23:30:46.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9c071e70 0x7fbb9c15b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9c071e70 0x7fbb9c15b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51050/0 (socket says 172.21.15.81:51050) 2024-05-08T23:30:46.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9affd700 1 -- 172.21.15.81:0/3146255897 learned_addr learned my addr 172.21.15.81:0/3146255897 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:46.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f23b1446700 1 --2- 172.21.15.81:0/3168568359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ac06d820 0x7f23ac0ce000 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23ac06d300 tx=0x7f23a800d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=913081e7ca81a589 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f23b0c45700 1 -- 172.21.15.81:0/3168568359 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8010070 con 0x7f23ac06d820 2024-05-08T23:30:46.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9b7fe700 1 --2- 172.21.15.81:0/3146255897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb9c165070 0x7fbb9c0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9a7fc700 1 --2- 172.21.15.81:0/3146255897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb9c0b3c20 0x7fbb9c0afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9affd700 1 -- 172.21.15.81:0/3146255897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb9c165070 msgr2=0x7fbb9c0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9affd700 1 --2- 172.21.15.81:0/3146255897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb9c165070 0x7fbb9c0b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9affd700 1 -- 172.21.15.81:0/3146255897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb9c0b3c20 msgr2=0x7fbb9c0afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9affd700 1 --2- 172.21.15.81:0/3146255897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb9c0b3c20 0x7fbb9c0afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9affd700 1 -- 172.21.15.81:0/3146255897 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c009570 con 0x7fbb9c071e70 2024-05-08T23:30:46.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9affd700 1 --2- 172.21.15.81:0/3146255897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9c071e70 0x7fbb9c15b590 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c014040 tx=0x7fbb8c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f970d0dc700 1 -- 172.21.15.81:0/512131939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97081e8310 con 0x7f9708160c80 2024-05-08T23:30:46.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f96ef7fe700 1 -- 172.21.15.81:0/512131939 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f97000147e0 con 0x7f9708160c80 2024-05-08T23:30:46.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f23b0c45700 1 -- 172.21.15.81:0/3168568359 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a800da40 con 0x7f23ac06d820 2024-05-08T23:30:46.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f23b0c45700 1 -- 172.21.15.81:0/3168568359 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a800ddd0 con 0x7f23ac06d820 2024-05-08T23:30:46.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f23b2c49700 1 -- 172.21.15.81:0/3168568359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ac06d820 msgr2=0x7f23ac0ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f23b2c49700 1 --2- 172.21.15.81:0/3168568359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ac06d820 0x7f23ac0ce000 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23ac06d300 tx=0x7f23a800d330 comp rx=0 tx=0).stop 2024-05-08T23:30:46.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb837fe700 1 -- 172.21.15.81:0/3146255897 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c026480 con 0x7fbb9c071e70 2024-05-08T23:30:46.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9bfff700 1 -- 172.21.15.81:0/3146255897 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9c15ad90 con 0x7fbb9c071e70 2024-05-08T23:30:46.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb9bfff700 1 -- 172.21.15.81:0/3146255897 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9c158560 con 0x7fbb9c071e70 2024-05-08T23:30:46.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb837fe700 1 -- 172.21.15.81:0/3146255897 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8c01f070 con 0x7fbb9c071e70 2024-05-08T23:30:46.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7fbb837fe700 1 -- 172.21.15.81:0/3146255897 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c026820 con 0x7fbb9c071e70 2024-05-08T23:30:46.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f96ef7fe700 1 --2- 172.21.15.81:0/512131939 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f96f40667b0 0x7f96f4068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f96ef7fe700 1 -- 172.21.15.81:0/512131939 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9700083bd0 con 0x7f9708160c80 2024-05-08T23:30:46.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f9706d9d700 1 --2- 172.21.15.81:0/512131939 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f96f40667b0 0x7f96f4068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.033+0000 7f9706d9d700 1 --2- 172.21.15.81:0/512131939 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f96f40667b0 0x7f96f4068c70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f96fc004ef0 tx=0x7f96fc001080 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b2c49700 1 -- 172.21.15.81:0/3168568359 shutdown_connections 2024-05-08T23:30:46.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b2c49700 1 --2- 172.21.15.81:0/3168568359 >> v2:172.21.15.81:3300/0 conn(0x7f23ac0b1f60 0x7f23ac0b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b2c49700 1 --2- 172.21.15.81:0/3168568359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ac06d820 0x7f23ac0ce000 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b2c49700 1 -- 172.21.15.81:0/3168568359 >> 172.21.15.81:0/3168568359 conn(0x7f23ac0aebd0 msgr2=0x7f23ac0aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b2c49700 1 -- 172.21.15.81:0/3168568359 shutdown_connections 2024-05-08T23:30:46.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b2c49700 1 -- 172.21.15.81:0/3168568359 wait complete. 2024-05-08T23:30:46.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7fbb9bfff700 1 -- 172.21.15.81:0/3146255897 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c005310 con 0x7fbb9c071e70 2024-05-08T23:30:46.048 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b2c49700 1 Processor -- start 2024-05-08T23:30:46.048 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b2c49700 1 -- start start 2024-05-08T23:30:46.048 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b2c49700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23ac06d820 0x7f23ac1f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.048 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b2c49700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23ac0b1f60 0x7f23ac1583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.048 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b2c49700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ac0cdc20 0x7f23ac1f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.048 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b2c49700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac0adc20 con 0x7f23ac0b1f60 2024-05-08T23:30:46.048 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b2c49700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac0a89e0 con 0x7f23ac0cdc20 2024-05-08T23:30:46.048 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b2c49700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac1f4200 con 0x7f23ac06d820 2024-05-08T23:30:46.048 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b1446700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23ac0b1f60 0x7f23ac1583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.048 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b1446700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23ac0b1f60 0x7f23ac1583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51062/0 (socket says 172.21.15.81:51062) 2024-05-08T23:30:46.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.037+0000 7f23b1446700 1 -- 172.21.15.81:0/3290732531 learned_addr learned my addr 172.21.15.81:0/3290732531 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:46.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.041+0000 7f23b2448700 1 --2- 172.21.15.81:0/3290732531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ac0cdc20 0x7f23ac1f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.041+0000 7f23b1446700 1 -- 172.21.15.81:0/3290732531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23ac06d820 msgr2=0x7f23ac1f3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.041+0000 7f23b1c47700 1 --2- 172.21.15.81:0/3290732531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23ac06d820 0x7f23ac1f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.041+0000 7f23b1446700 1 --2- 172.21.15.81:0/3290732531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23ac06d820 0x7f23ac1f3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.041+0000 7f23b1446700 1 -- 172.21.15.81:0/3290732531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ac0cdc20 msgr2=0x7f23ac1f5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.041+0000 7f23b1446700 1 --2- 172.21.15.81:0/3290732531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ac0cdc20 0x7f23ac1f5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.041+0000 7f23b1446700 1 -- 172.21.15.81:0/3290732531 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a80085f0 con 0x7f23ac0b1f60 2024-05-08T23:30:46.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.041+0000 7f23b1446700 1 --2- 172.21.15.81:0/3290732531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23ac0b1f60 0x7f23ac1583d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f23a800d730 tx=0x7f23a8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.041+0000 7f96ef7fe700 1 -- 172.21.15.81:0/512131939 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f97081e8310 con 0x7f9708160c80 2024-05-08T23:30:46.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.041+0000 7f23b1c47700 1 --2- 172.21.15.81:0/3290732531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23ac06d820 0x7f23ac1f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:46.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.041+0000 7f23a27fc700 1 -- 172.21.15.81:0/3290732531 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a80043c0 con 0x7f23ac0b1f60 2024-05-08T23:30:46.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.041+0000 7fbb837fe700 1 -- 172.21.15.81:0/3146255897 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbb8c00bbe0 con 0x7fbb9c071e70 2024-05-08T23:30:46.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.041+0000 7fbb837fe700 1 --2- 172.21.15.81:0/3146255897 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbb88066780 0x7fbb88068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.041+0000 7fbb837fe700 1 -- 172.21.15.81:0/3146255897 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb8c021070 con 0x7fbb9c071e70 2024-05-08T23:30:46.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.041+0000 7fbb9a7fc700 1 --2- 172.21.15.81:0/3146255897 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbb88066780 0x7fbb88068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.045+0000 7fbb9a7fc700 1 --2- 172.21.15.81:0/3146255897 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbb88066780 0x7fbb88068c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0cba80 tx=0x7fbb90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.049+0000 7f23b2c49700 1 -- 172.21.15.81:0/3290732531 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ac0ade70 con 0x7f23ac0b1f60 2024-05-08T23:30:46.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.049+0000 7f23b2c49700 1 -- 172.21.15.81:0/3290732531 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ac0a8670 con 0x7f23ac0b1f60 2024-05-08T23:30:46.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.049+0000 7f2397fff700 1 -- 172.21.15.81:0/3290732531 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ac0ae390 con 0x7f23ac0b1f60 2024-05-08T23:30:46.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.049+0000 7f23a27fc700 1 -- 172.21.15.81:0/3290732531 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a8004520 con 0x7f23ac0b1f60 2024-05-08T23:30:46.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.049+0000 7fbb837fe700 1 -- 172.21.15.81:0/3146255897 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbb8c085e60 con 0x7fbb9c071e70 2024-05-08T23:30:46.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.053+0000 7f23a27fc700 1 -- 172.21.15.81:0/3290732531 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a80038c0 con 0x7f23ac0b1f60 2024-05-08T23:30:46.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.053+0000 7f23a27fc700 1 -- 172.21.15.81:0/3290732531 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f23a801d070 con 0x7f23ac0b1f60 2024-05-08T23:30:46.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.053+0000 7f23a27fc700 1 --2- 172.21.15.81:0/3290732531 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2390066760 0x7f2390068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.053+0000 7f23a27fc700 1 -- 172.21.15.81:0/3290732531 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23a80854a0 con 0x7f23ac0b1f60 2024-05-08T23:30:46.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.053+0000 7f23b1c47700 1 --2- 172.21.15.81:0/3290732531 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2390066760 0x7f2390068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.057+0000 7f23b1c47700 1 --2- 172.21.15.81:0/3290732531 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2390066760 0x7f2390068c20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f239c006fd0 tx=0x7f239c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.064 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.061+0000 7f23a27fc700 1 -- 172.21.15.81:0/3290732531 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f23a80529c0 con 0x7f23ac0b1f60 2024-05-08T23:30:46.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.065+0000 7fcb1b698700 1 Processor -- start 2024-05-08T23:30:46.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.065+0000 7fcb1b698700 1 -- start start 2024-05-08T23:30:46.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fb68a579700 1 Processor -- start 2024-05-08T23:30:46.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fb68a579700 1 -- start start 2024-05-08T23:30:46.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb1b698700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb141459b0 0x7fcb14145e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb1b698700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb140ad7d0 con 0x7fcb1406d8e0 2024-05-08T23:30:46.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb1b698700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb140ac8e0 con 0x7fcb1406dcc0 2024-05-08T23:30:46.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb1b698700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb140ae370 con 0x7fcb141459b0 2024-05-08T23:30:46.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb19434700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcb1406d8e0 0x7fcb140adc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38676/0 (socket says 172.21.15.81:38676) 2024-05-08T23:30:46.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb19434700 1 -- 172.21.15.81:0/4111419289 learned_addr learned my addr 172.21.15.81:0/4111419289 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:46.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 -- 172.21.15.81:0/4111419289 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332619021 0 0) 0x7fcb140ad7d0 con 0x7fcb1406d8e0 2024-05-08T23:30:46.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb19c35700 1 --2- 172.21.15.81:0/4111419289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb141459b0 0x7fcb14145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 -- 172.21.15.81:0/4111419289 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb000035d0 con 0x7fcb1406d8e0 2024-05-08T23:30:46.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 -- 172.21.15.81:0/4111419289 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3408108178 0 0) 0x7fcb000035d0 con 0x7fcb1406d8e0 2024-05-08T23:30:46.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 -- 172.21.15.81:0/4111419289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb141459b0 msgr2=0x7fcb14145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 --2- 172.21.15.81:0/4111419289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb141459b0 0x7fcb14145e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 -- 172.21.15.81:0/4111419289 >> v1:172.21.15.117:6789/0 conn(0x7fcb1406dcc0 legacy=0x7fcb14071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 -- 172.21.15.81:0/4111419289 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb140acbc0 con 0x7fcb1406d8e0 2024-05-08T23:30:46.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 -- 172.21.15.81:0/4111419289 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7fcb10001670 con 0x7fcb1406d8e0 2024-05-08T23:30:46.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 -- 172.21.15.81:0/4111419289 >> v1:172.21.15.81:6789/0 conn(0x7fcb1406d8e0 legacy=0x7fcb140adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 --2- 172.21.15.81:0/4111419289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb00003a50 0x7fcb00003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 --2- 172.21.15.81:0/4111419289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb141459b0 0x7fcb00004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 --2- 172.21.15.81:0/4111419289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb00004db0 0x7fcb00005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 -- 172.21.15.81:0/4111419289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb140ae370 con 0x7fcb00004db0 2024-05-08T23:30:46.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 -- 172.21.15.81:0/4111419289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0000bae0 con 0x7fcb00003a50 2024-05-08T23:30:46.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 -- 172.21.15.81:0/4111419289 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0000bca0 con 0x7fcb141459b0 2024-05-08T23:30:46.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb19c35700 1 --2- 172.21.15.81:0/4111419289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb00004db0 0x7fcb00005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb19434700 1 --2- 172.21.15.81:0/4111419289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb00003a50 0x7fcb00003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb19c35700 1 -- 172.21.15.81:0/4111419289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb141459b0 msgr2=0x7fcb00004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb19c35700 1 --2- 172.21.15.81:0/4111419289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb141459b0 0x7fcb00004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb19c35700 1 -- 172.21.15.81:0/4111419289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb00003a50 msgr2=0x7fcb00003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb19c35700 1 --2- 172.21.15.81:0/4111419289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb00003a50 0x7fcb00003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb19c35700 1 -- 172.21.15.81:0/4111419289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb000035d0 con 0x7fcb00004db0 2024-05-08T23:30:46.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb19c35700 1 --2- 172.21.15.81:0/4111419289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb00004db0 0x7fcb00005230 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009f90 tx=0x7fcb000058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10eed8d11e338b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 -- 172.21.15.81:0/4111419289 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c00ec30 con 0x7fcb00004db0 2024-05-08T23:30:46.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 -- 172.21.15.81:0/4111419289 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c00f040 con 0x7fcb00004db0 2024-05-08T23:30:46.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fcb0bfff700 1 -- 172.21.15.81:0/4111419289 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c014480 con 0x7fcb00004db0 2024-05-08T23:30:46.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fb68a579700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb684072120 0x7fb68406d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fb68a579700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb68415a150 0x7fb6840b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fb689d78700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb6840b24a0 0x7fb68415ccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38682/0 (socket says 172.21.15.81:38682) 2024-05-08T23:30:46.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fb689d78700 1 -- 172.21.15.81:0/324075800 learned_addr learned my addr 172.21.15.81:0/324075800 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:46.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fb68a579700 1 -- 172.21.15.81:0/324075800 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6840b2d90 con 0x7fb6840b24a0 2024-05-08T23:30:46.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fb68a579700 1 -- 172.21.15.81:0/324075800 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb684150680 con 0x7fb684072120 2024-05-08T23:30:46.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7fb68a579700 1 -- 172.21.15.81:0/324075800 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6840b2910 con 0x7fb68415a150 2024-05-08T23:30:46.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 -- 172.21.15.81:0/324075800 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235043113 0 0) 0x7fb6840b2d90 con 0x7fb6840b24a0 2024-05-08T23:30:46.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 -- 172.21.15.81:0/324075800 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6700035d0 con 0x7fb6840b24a0 2024-05-08T23:30:46.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb689577700 1 --2- 172.21.15.81:0/324075800 >> v2:172.21.15.117:3300/0 conn(0x7fb684072120 0x7fb68406d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fcb1b698700 1 -- 172.21.15.81:0/4111419289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb00004db0 msgr2=0x7fcb00005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fcb1b698700 1 --2- 172.21.15.81:0/4111419289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb00004db0 0x7fcb00005230 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009f90 tx=0x7fcb000058a0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 -- 172.21.15.81:0/324075800 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2619369158 0 0) 0x7fb6700035d0 con 0x7fb6840b24a0 2024-05-08T23:30:46.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 -- 172.21.15.81:0/324075800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb68415a150 msgr2=0x7fb6840b1f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:30:46.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 --2- 172.21.15.81:0/324075800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb68415a150 0x7fb6840b1f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 -- 172.21.15.81:0/324075800 >> v2:172.21.15.117:3300/0 conn(0x7fb684072120 msgr2=0x7fb68406d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fcb19c35700 1 -- 172.21.15.81:0/4111419289 reap_dead start 2024-05-08T23:30:46.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 --2- 172.21.15.81:0/324075800 >> v2:172.21.15.117:3300/0 conn(0x7fb684072120 0x7fb68406d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fcb1b698700 1 -- 172.21.15.81:0/4111419289 shutdown_connections 2024-05-08T23:30:46.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fcb1b698700 1 -- 172.21.15.81:0/4111419289 >> 172.21.15.81:0/4111419289 conn(0x7fcb140b0cd0 msgr2=0x7fcb140b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 -- 172.21.15.81:0/324075800 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6840ce4b0 con 0x7fb6840b24a0 2024-05-08T23:30:46.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 -- 172.21.15.81:0/324075800 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7fb67c0019f0 con 0x7fb6840b24a0 2024-05-08T23:30:46.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 -- 172.21.15.81:0/324075800 >> v1:172.21.15.81:6789/0 conn(0x7fb6840b24a0 legacy=0x7fb68415ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 --2- 172.21.15.81:0/324075800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb670003a20 0x7fb670003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 --2- 172.21.15.81:0/324075800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb68415a150 0x7fb670004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 --2- 172.21.15.81:0/324075800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb670004db0 0x7fb670005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 -- 172.21.15.81:0/324075800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb684150680 con 0x7fb670003a20 2024-05-08T23:30:46.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 -- 172.21.15.81:0/324075800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6840b2910 con 0x7fb670004db0 2024-05-08T23:30:46.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 -- 172.21.15.81:0/324075800 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb670005c60 con 0x7fb68415a150 2024-05-08T23:30:46.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fcb1b698700 1 -- 172.21.15.81:0/4111419289 shutdown_connections 2024-05-08T23:30:46.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb688d76700 1 --2- 172.21.15.81:0/324075800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb68415a150 0x7fb670004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb689d78700 1 --2- 172.21.15.81:0/324075800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb670004db0 0x7fb670005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb689577700 1 --2- 172.21.15.81:0/324075800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb670003a20 0x7fb670003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7f42b6615700 1 Processor -- start 2024-05-08T23:30:46.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.069+0000 7f42b6615700 1 -- start start 2024-05-08T23:30:46.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b6615700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f42b00b3c40 0x7f42b006e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b6615700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b006c330 con 0x7f42b008a620 2024-05-08T23:30:46.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b6615700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b006c610 con 0x7f42b006caa0 2024-05-08T23:30:46.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b6615700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b00cc530 con 0x7f42b00b3c40 2024-05-08T23:30:46.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b4e12700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f42b008a620 0x7f42b00b4020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38684/0 (socket says 172.21.15.81:38684) 2024-05-08T23:30:46.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b4e12700 1 -- 172.21.15.81:0/4187272470 learned_addr learned my addr 172.21.15.81:0/4187272470 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:46.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b5613700 1 --2- 172.21.15.81:0/4187272470 >> v2:172.21.15.117:3300/0 conn(0x7f42b00b3c40 0x7f42b006e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 -- 172.21.15.81:0/4187272470 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 998544242 0 0) 0x7f42b006c330 con 0x7f42b008a620 2024-05-08T23:30:46.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 -- 172.21.15.81:0/4187272470 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f429c0035d0 con 0x7f42b008a620 2024-05-08T23:30:46.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 -- 172.21.15.81:0/4187272470 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 546107234 0 0) 0x7f429c0035d0 con 0x7f42b008a620 2024-05-08T23:30:46.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 -- 172.21.15.81:0/4187272470 >> v2:172.21.15.117:3300/0 conn(0x7f42b00b3c40 msgr2=0x7f42b006e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 --2- 172.21.15.81:0/4187272470 >> v2:172.21.15.117:3300/0 conn(0x7f42b00b3c40 0x7f42b006e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 -- 172.21.15.81:0/4187272470 >> v1:172.21.15.117:6789/0 conn(0x7f42b006caa0 legacy=0x7f42b006cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 -- 172.21.15.81:0/4187272470 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b00b2a10 con 0x7f42b008a620 2024-05-08T23:30:46.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 -- 172.21.15.81:0/4187272470 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f42a80019f0 con 0x7f42b008a620 2024-05-08T23:30:46.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 -- 172.21.15.81:0/4187272470 >> v1:172.21.15.81:6789/0 conn(0x7f42b008a620 legacy=0x7f42b00b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 --2- 172.21.15.81:0/4187272470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f429c003a70 0x7f429c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 --2- 172.21.15.81:0/4187272470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f429c004870 0x7f429c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 --2- 172.21.15.81:0/4187272470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f429c005230 0x7f429c00dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 -- 172.21.15.81:0/4187272470 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b00cc530 con 0x7f429c003a70 2024-05-08T23:30:46.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 -- 172.21.15.81:0/4187272470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c00e340 con 0x7f429c004870 2024-05-08T23:30:46.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 -- 172.21.15.81:0/4187272470 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429c00e500 con 0x7f429c005230 2024-05-08T23:30:46.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b4e12700 1 --2- 172.21.15.81:0/4187272470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f429c004870 0x7f429c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b5613700 1 --2- 172.21.15.81:0/4187272470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f429c003a70 0x7f429c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b5e14700 1 --2- 172.21.15.81:0/4187272470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f429c005230 0x7f429c00dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb689577700 1 -- 172.21.15.81:0/324075800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb68415a150 msgr2=0x7fb670004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb689577700 1 --2- 172.21.15.81:0/324075800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb68415a150 0x7fb670004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb689577700 1 -- 172.21.15.81:0/324075800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb670004db0 msgr2=0x7fb670005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb689577700 1 --2- 172.21.15.81:0/324075800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb670004db0 0x7fb670005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb689577700 1 -- 172.21.15.81:0/324075800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6700035d0 con 0x7fb670003a20 2024-05-08T23:30:46.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb689d78700 1 --2- 172.21.15.81:0/324075800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb670004db0 0x7fb670005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:46.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb689577700 1 --2- 172.21.15.81:0/324075800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb670003a20 0x7fb670003e40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb6800098b0 tx=0x7fb6700058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a03c4b3d2f8f6e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 -- 172.21.15.81:0/324075800 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68000c070 con 0x7fb670003a20 2024-05-08T23:30:46.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 -- 172.21.15.81:0/324075800 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb680013630 con 0x7fb670003a20 2024-05-08T23:30:46.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb67bfff700 1 -- 172.21.15.81:0/324075800 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6800139e0 con 0x7fb670003a20 2024-05-08T23:30:46.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fcb1b698700 1 -- 172.21.15.81:0/4111419289 wait complete. 2024-05-08T23:30:46.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fcb1b698700 1 Processor -- start 2024-05-08T23:30:46.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b5613700 1 -- 172.21.15.81:0/4187272470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f429c005230 msgr2=0x7f429c00dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b5613700 1 --2- 172.21.15.81:0/4187272470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f429c005230 0x7f429c00dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b5613700 1 -- 172.21.15.81:0/4187272470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f429c004870 msgr2=0x7f429c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b5613700 1 --2- 172.21.15.81:0/4187272470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f429c004870 0x7f429c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b5613700 1 -- 172.21.15.81:0/4187272470 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c0035d0 con 0x7f429c003a70 2024-05-08T23:30:46.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b5e14700 1 -- 172.21.15.81:0/4187272470 reap_dead start 2024-05-08T23:30:46.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42b5613700 1 --2- 172.21.15.81:0/4187272470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f429c003a70 0x7f429c003e90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009b80 tx=0x7f429c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e581a39db18b8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 -- 172.21.15.81:0/4187272470 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0092d0 con 0x7f429c003a70 2024-05-08T23:30:46.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb68a579700 1 -- 172.21.15.81:0/324075800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb670003a20 msgr2=0x7fb670003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb68a579700 1 --2- 172.21.15.81:0/324075800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb670003a20 0x7fb670003e40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb6800098b0 tx=0x7fb6700058a0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fb689d78700 1 -- 172.21.15.81:0/324075800 reap_dead start 2024-05-08T23:30:46.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 -- 172.21.15.81:0/4187272470 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac010040 con 0x7f429c003a70 2024-05-08T23:30:46.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7f42a7fff700 1 -- 172.21.15.81:0/4187272470 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00a730 con 0x7f429c003a70 2024-05-08T23:30:46.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b6615700 1 -- 172.21.15.81:0/4187272470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f429c003a70 msgr2=0x7f429c003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b6615700 1 --2- 172.21.15.81:0/4187272470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f429c003a70 0x7f429c003e90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009b80 tx=0x7f429c0058a0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.073+0000 7fcb1b698700 1 -- start start 2024-05-08T23:30:46.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb1b698700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb00004db0 0x7fcb14139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb1b698700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb1413bbc0 0x7fcb1413c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb1b698700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb1415f860 0x7fcb1415fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb1b698700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb14142ce0 con 0x7fcb1415f860 2024-05-08T23:30:46.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb1b698700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1413c580 con 0x7fcb1413bbc0 2024-05-08T23:30:46.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb1b698700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb14160690 con 0x7fcb00004db0 2024-05-08T23:30:46.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b6615700 1 -- 172.21.15.81:0/4187272470 shutdown_connections 2024-05-08T23:30:46.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b6615700 1 --2- 172.21.15.81:0/4187272470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f429c003a70 0x7f429c003e90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b6615700 1 -- 172.21.15.81:0/4187272470 >> 172.21.15.81:0/4187272470 conn(0x7f42b00aebd0 msgr2=0x7f42b00aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b6615700 1 -- 172.21.15.81:0/4187272470 shutdown_connections 2024-05-08T23:30:46.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb19c35700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb1415f860 0x7fcb1415fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b6615700 1 -- 172.21.15.81:0/4187272470 wait complete. 2024-05-08T23:30:46.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb19c35700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb1415f860 0x7fcb1415fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51096/0 (socket says 172.21.15.81:51096) 2024-05-08T23:30:46.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb19c35700 1 -- 172.21.15.81:0/594645661 learned_addr learned my addr 172.21.15.81:0/594645661 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:46.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb18c33700 1 --2- 172.21.15.81:0/594645661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb1413bbc0 0x7fcb1413c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb19434700 1 --2- 172.21.15.81:0/594645661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb00004db0 0x7fcb14139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb19c35700 1 -- 172.21.15.81:0/594645661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb00004db0 msgr2=0x7fcb14139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb19c35700 1 --2- 172.21.15.81:0/594645661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb00004db0 0x7fcb14139940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb19c35700 1 -- 172.21.15.81:0/594645661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb1413bbc0 msgr2=0x7fcb1413c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb19c35700 1 --2- 172.21.15.81:0/594645661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb1413bbc0 0x7fcb1413c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb19c35700 1 -- 172.21.15.81:0/594645661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb14160970 con 0x7fcb1415f860 2024-05-08T23:30:46.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb18c33700 1 --2- 172.21.15.81:0/594645661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb1413bbc0 0x7fcb1413c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:46.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fb68a579700 1 -- 172.21.15.81:0/324075800 shutdown_connections 2024-05-08T23:30:46.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fb68a579700 1 -- 172.21.15.81:0/324075800 >> 172.21.15.81:0/324075800 conn(0x7fb6840aebd0 msgr2=0x7fb6840aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb19c35700 1 --2- 172.21.15.81:0/594645661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb1415f860 0x7fcb1415fcf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcb000047d0 tx=0x7fcb0c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb09ffb700 1 -- 172.21.15.81:0/594645661 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c014aa0 con 0x7fcb1415f860 2024-05-08T23:30:46.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb09ffb700 1 -- 172.21.15.81:0/594645661 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c014c20 con 0x7fcb1415f860 2024-05-08T23:30:46.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb09ffb700 1 -- 172.21.15.81:0/594645661 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c026420 con 0x7fcb1415f860 2024-05-08T23:30:46.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb1b698700 1 -- 172.21.15.81:0/594645661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb14160c50 con 0x7fcb1415f860 2024-05-08T23:30:46.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fcb1b698700 1 -- 172.21.15.81:0/594645661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb14139e80 con 0x7fcb1415f860 2024-05-08T23:30:46.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b6615700 1 Processor -- start 2024-05-08T23:30:46.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fb68a579700 1 -- 172.21.15.81:0/324075800 shutdown_connections 2024-05-08T23:30:46.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7fb68a579700 1 -- 172.21.15.81:0/324075800 wait complete. 2024-05-08T23:30:46.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b6615700 1 -- start start 2024-05-08T23:30:46.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b6615700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f429c003a70 0x7f42b01f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b6615700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42b00a84d0 0x7f42b00a8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b6615700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42b00aa3e0 0x7f42b00aa860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b6615700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b00a8e90 con 0x7f42b00aa3e0 2024-05-08T23:30:46.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b6615700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b01fa3c0 con 0x7f42b00a84d0 2024-05-08T23:30:46.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b6615700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b016cf50 con 0x7f429c003a70 2024-05-08T23:30:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.077+0000 7f42b5e14700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42b00aa3e0 0x7f42b00aa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7f42b5e14700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42b00aa3e0 0x7f42b00aa860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51110/0 (socket says 172.21.15.81:51110) 2024-05-08T23:30:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7f42b5e14700 1 -- 172.21.15.81:0/1697202109 learned_addr learned my addr 172.21.15.81:0/1697202109 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7f42b5613700 1 --2- 172.21.15.81:0/1697202109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f429c003a70 0x7f42b01f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7f42b4e12700 1 --2- 172.21.15.81:0/1697202109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42b00a84d0 0x7f42b00a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7f42b5613700 1 -- 172.21.15.81:0/1697202109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42b00a84d0 msgr2=0x7f42b00a8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7f42b5613700 1 --2- 172.21.15.81:0/1697202109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42b00a84d0 0x7f42b00a8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7f42b5613700 1 -- 172.21.15.81:0/1697202109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42b00aa3e0 msgr2=0x7f42b00aa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7f42b5613700 1 --2- 172.21.15.81:0/1697202109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42b00aa3e0 0x7f42b00aa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7f42b5613700 1 -- 172.21.15.81:0/1697202109 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b0166980 con 0x7f429c003a70 2024-05-08T23:30:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7f42b5613700 1 --2- 172.21.15.81:0/1697202109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f429c003a70 0x7f42b01f9e80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f42ac016000 tx=0x7f42ac007250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb68a579700 1 Processor -- start 2024-05-08T23:30:46.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb68a579700 1 -- start start 2024-05-08T23:30:46.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fcb09ffb700 1 -- 172.21.15.81:0/594645661 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcb0c023020 con 0x7fcb1415f860 2024-05-08T23:30:46.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fcb09ffb700 1 --2- 172.21.15.81:0/594645661 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcaf8066730 0x7fcaf8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fcb09ffb700 1 -- 172.21.15.81:0/594645661 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb0c085570 con 0x7fcb1415f860 2024-05-08T23:30:46.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fcb19434700 1 --2- 172.21.15.81:0/594645661 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcaf8066730 0x7fcaf8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fcb19434700 1 --2- 172.21.15.81:0/594645661 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcaf8066730 0x7fcaf8068bf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcb10005ae0 tx=0x7fcb10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb68a579700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb670003a20 0x7fb68414db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb68a579700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb684156b80 0x7fb684157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb68a579700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb684154a00 0x7fb684154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb68a579700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb684152990 con 0x7fb670003a20 2024-05-08T23:30:46.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb68a579700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6841553c0 con 0x7fb684156b80 2024-05-08T23:30:46.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb68a579700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68414e050 con 0x7fb684154a00 2024-05-08T23:30:46.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb689577700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb670003a20 0x7fb68414db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb689577700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb670003a20 0x7fb68414db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51122/0 (socket says 172.21.15.81:51122) 2024-05-08T23:30:46.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb689577700 1 -- 172.21.15.81:0/3117596422 learned_addr learned my addr 172.21.15.81:0/3117596422 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:46.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb688d76700 1 --2- 172.21.15.81:0/3117596422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb684156b80 0x7fb684157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb689d78700 1 --2- 172.21.15.81:0/3117596422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb684154a00 0x7fb684154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb689577700 1 -- 172.21.15.81:0/3117596422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb684154a00 msgr2=0x7fb684154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb689577700 1 --2- 172.21.15.81:0/3117596422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb684154a00 0x7fb684154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb689577700 1 -- 172.21.15.81:0/3117596422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb684156b80 msgr2=0x7fb684157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb689577700 1 --2- 172.21.15.81:0/3117596422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb684156b80 0x7fb684157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb689577700 1 -- 172.21.15.81:0/3117596422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb684151890 con 0x7fb670003a20 2024-05-08T23:30:46.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.081+0000 7fb689577700 1 --2- 172.21.15.81:0/3117596422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb670003a20 0x7fb68414db10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb680015000 tx=0x7fb6800047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.085+0000 7fcb1b698700 1 -- 172.21.15.81:0/594645661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf4005310 con 0x7fcb1415f860 2024-05-08T23:30:46.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.085+0000 7fb679ffb700 1 -- 172.21.15.81:0/3117596422 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680013bb0 con 0x7fb670003a20 2024-05-08T23:30:46.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.085+0000 7f42a5ffb700 1 -- 172.21.15.81:0/1697202109 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac007000 con 0x7f429c003a70 2024-05-08T23:30:46.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.085+0000 7fb679ffb700 1 -- 172.21.15.81:0/3117596422 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb680013d10 con 0x7fb670003a20 2024-05-08T23:30:46.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.085+0000 7fb68a579700 1 -- 172.21.15.81:0/3117596422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb684151ff0 con 0x7fb670003a20 2024-05-08T23:30:46.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.085+0000 7fb679ffb700 1 -- 172.21.15.81:0/3117596422 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6800038c0 con 0x7fb670003a20 2024-05-08T23:30:46.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.085+0000 7fb68a579700 1 -- 172.21.15.81:0/3117596422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6841525b0 con 0x7fb670003a20 2024-05-08T23:30:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.085+0000 7f42b6615700 1 -- 172.21.15.81:0/1697202109 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b0166c60 con 0x7f429c003a70 2024-05-08T23:30:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.085+0000 7f42b6615700 1 -- 172.21.15.81:0/1697202109 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b0167220 con 0x7f429c003a70 2024-05-08T23:30:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.085+0000 7f42a5ffb700 1 -- 172.21.15.81:0/1697202109 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac005e90 con 0x7f429c003a70 2024-05-08T23:30:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.085+0000 7f42a5ffb700 1 -- 172.21.15.81:0/1697202109 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0036c0 con 0x7f429c003a70 2024-05-08T23:30:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.089+0000 7f42a5ffb700 1 -- 172.21.15.81:0/1697202109 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f42ac01b070 con 0x7f429c003a70 2024-05-08T23:30:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.089+0000 7f42a5ffb700 1 --2- 172.21.15.81:0/1697202109 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42940667b0 0x7f4294068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.089+0000 7f42a5ffb700 1 -- 172.21.15.81:0/1697202109 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42ac084bd0 con 0x7f429c003a70 2024-05-08T23:30:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.089+0000 7f42b4e12700 1 --2- 172.21.15.81:0/1697202109 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42940667b0 0x7f4294068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.089+0000 7fb68a579700 1 -- 172.21.15.81:0/3117596422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6840684b0 con 0x7fb670003a20 2024-05-08T23:30:46.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.089+0000 7f42b4e12700 1 --2- 172.21.15.81:0/1697202109 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42940667b0 0x7f4294068c70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f42b01fa6a0 tx=0x7f42a8006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.089+0000 7fcb09ffb700 1 -- 172.21.15.81:0/594645661 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcb0c0529f0 con 0x7fcb1415f860 2024-05-08T23:30:46.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.089+0000 7f42b6615700 1 -- 172.21.15.81:0/1697202109 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4290005310 con 0x7f429c003a70 2024-05-08T23:30:46.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fb679ffb700 1 -- 172.21.15.81:0/3117596422 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb680003a20 con 0x7fb670003a20 2024-05-08T23:30:46.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a359e700 1 Processor -- start 2024-05-08T23:30:46.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fb679ffb700 1 --2- 172.21.15.81:0/3117596422 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6680667b0 0x7fb668068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fb679ffb700 1 -- 172.21.15.81:0/3117596422 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6800851f0 con 0x7fb670003a20 2024-05-08T23:30:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a359e700 1 -- start start 2024-05-08T23:30:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a359e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa3a406d820 0x7fa3a40ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a359e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3a40b2d90 con 0x7fa3a40cdc20 2024-05-08T23:30:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a359e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3a406d390 con 0x7fa3a40b1f60 2024-05-08T23:30:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a359e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3a4150680 con 0x7fa3a406d820 2024-05-08T23:30:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a259c700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa3a40cdc20 0x7fa3a415a150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38686/0 (socket says 172.21.15.81:38686) 2024-05-08T23:30:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a259c700 1 -- 172.21.15.81:0/1244081145 learned_addr learned my addr 172.21.15.81:0/1244081145 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:46.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fb688d76700 1 --2- 172.21.15.81:0/3117596422 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6680667b0 0x7fb668068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fb688d76700 1 --2- 172.21.15.81:0/3117596422 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6680667b0 0x7fb668068c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb674006fd0 tx=0x7fb674008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a159a700 1 -- 172.21.15.81:0/1244081145 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65501767 0 0) 0x7fa3a406d390 con 0x7fa3a40b1f60 2024-05-08T23:30:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a159a700 1 -- 172.21.15.81:0/1244081145 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa38c0035d0 con 0x7fa3a40b1f60 2024-05-08T23:30:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a159a700 1 -- 172.21.15.81:0/1244081145 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1368280429 0 0) 0x7fa38c0035d0 con 0x7fa3a40b1f60 2024-05-08T23:30:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a159a700 1 -- 172.21.15.81:0/1244081145 >> v2:172.21.15.117:3300/0 conn(0x7fa3a406d820 msgr2=0x7fa3a40ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a159a700 1 --2- 172.21.15.81:0/1244081145 >> v2:172.21.15.117:3300/0 conn(0x7fa3a406d820 0x7fa3a40ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a159a700 1 -- 172.21.15.81:0/1244081145 >> v1:172.21.15.81:6789/0 conn(0x7fa3a40cdc20 legacy=0x7fa3a415a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a159a700 1 -- 172.21.15.81:0/1244081145 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a406c780 con 0x7fa3a40b1f60 2024-05-08T23:30:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.093+0000 7fa3a159a700 1 -- 172.21.15.81:0/1244081145 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7fa39c001920 con 0x7fa3a40b1f60 2024-05-08T23:30:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a159a700 1 -- 172.21.15.81:0/1244081145 >> v1:172.21.15.117:6789/0 conn(0x7fa3a40b1f60 legacy=0x7fa3a40b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a159a700 1 --2- 172.21.15.81:0/1244081145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c003a70 0x7fa38c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a159a700 1 --2- 172.21.15.81:0/1244081145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa38c004870 0x7fa38c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a159a700 1 --2- 172.21.15.81:0/1244081145 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa38c005430 0x7fa38c00cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a159a700 1 -- 172.21.15.81:0/1244081145 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a4150680 con 0x7fa38c004870 2024-05-08T23:30:46.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a159a700 1 -- 172.21.15.81:0/1244081145 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c00d3b0 con 0x7fa38c003a70 2024-05-08T23:30:46.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a159a700 1 -- 172.21.15.81:0/1244081145 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c00d570 con 0x7fa38c005430 2024-05-08T23:30:46.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a259c700 1 --2- 172.21.15.81:0/1244081145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c003a70 0x7fa38c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a2d9d700 1 --2- 172.21.15.81:0/1244081145 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa38c005430 0x7fa38c00cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a2d9d700 1 -- 172.21.15.81:0/1244081145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c003a70 msgr2=0x7fa38c003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a2d9d700 1 --2- 172.21.15.81:0/1244081145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c003a70 0x7fa38c003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a2d9d700 1 -- 172.21.15.81:0/1244081145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa38c004870 msgr2=0x7fa38c004cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:30:46.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a2d9d700 1 --2- 172.21.15.81:0/1244081145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa38c004870 0x7fa38c004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a2d9d700 1 -- 172.21.15.81:0/1244081145 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38c0035d0 con 0x7fa38c005430 2024-05-08T23:30:46.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a259c700 1 --2- 172.21.15.81:0/1244081145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c003a70 0x7fa38c003e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:46.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a2d9d700 1 -- 172.21.15.81:0/1244081145 reap_dead start 2024-05-08T23:30:46.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fa3a2d9d700 1 --2- 172.21.15.81:0/1244081145 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa38c005430 0x7fa38c00cb40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00a790 tx=0x7fa38c0058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=638a862aa8b43622 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.097+0000 7fb679ffb700 1 -- 172.21.15.81:0/3117596422 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb6800526d0 con 0x7fb670003a20 2024-05-08T23:30:46.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.101+0000 7fa3a159a700 1 -- 172.21.15.81:0/1244081145 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c00a250 con 0x7fa38c005430 2024-05-08T23:30:46.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.101+0000 7fa3a159a700 1 -- 172.21.15.81:0/1244081145 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c012660 con 0x7fa38c005430 2024-05-08T23:30:46.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.101+0000 7fa3a359e700 1 -- 172.21.15.81:0/1244081145 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa38c005430 msgr2=0x7fa38c00cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.101+0000 7fa3a359e700 1 --2- 172.21.15.81:0/1244081145 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa38c005430 0x7fa38c00cb40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00a790 tx=0x7fa38c0058a0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.101+0000 7f42a5ffb700 1 -- 172.21.15.81:0/1697202109 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f42ac0520b0 con 0x7f429c003a70 2024-05-08T23:30:46.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.109+0000 7fa3a359e700 1 -- 172.21.15.81:0/1244081145 shutdown_connections 2024-05-08T23:30:46.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.109+0000 7fa3a359e700 1 --2- 172.21.15.81:0/1244081145 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa38c005430 0x7fa38c00cb40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.109+0000 7fa3a359e700 1 -- 172.21.15.81:0/1244081145 >> 172.21.15.81:0/1244081145 conn(0x7fa3a40aebd0 msgr2=0x7fa3a40aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.109+0000 7fa3a359e700 1 -- 172.21.15.81:0/1244081145 shutdown_connections 2024-05-08T23:30:46.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.109+0000 7fa3a359e700 1 -- 172.21.15.81:0/1244081145 wait complete. 2024-05-08T23:30:46.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.109+0000 7fa3a359e700 1 Processor -- start 2024-05-08T23:30:46.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.109+0000 7fa3a359e700 1 -- start start 2024-05-08T23:30:46.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a359e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c005430 0x7fa3a414fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a359e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3a4200c50 0x7fa3a414ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a359e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3a42010d0 0x7fa3a4203080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a359e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a4152050 con 0x7fa3a4200c50 2024-05-08T23:30:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a359e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a41516b0 con 0x7fa38c005430 2024-05-08T23:30:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a359e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a4151990 con 0x7fa3a42010d0 2024-05-08T23:30:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a1d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3a4200c50 0x7fa3a414ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a1d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3a4200c50 0x7fa3a414ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51134/0 (socket says 172.21.15.81:51134) 2024-05-08T23:30:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a1d9b700 1 -- 172.21.15.81:0/3196986154 learned_addr learned my addr 172.21.15.81:0/3196986154 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a259c700 1 --2- 172.21.15.81:0/3196986154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c005430 0x7fa3a414fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a1d9b700 1 -- 172.21.15.81:0/3196986154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3a42010d0 msgr2=0x7fa3a4203080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a1d9b700 1 --2- 172.21.15.81:0/3196986154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3a42010d0 0x7fa3a4203080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a1d9b700 1 -- 172.21.15.81:0/3196986154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c005430 msgr2=0x7fa3a414fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a1d9b700 1 --2- 172.21.15.81:0/3196986154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c005430 0x7fa3a414fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a1d9b700 1 -- 172.21.15.81:0/3196986154 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a4151c70 con 0x7fa3a4200c50 2024-05-08T23:30:46.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a1d9b700 1 --2- 172.21.15.81:0/3196986154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3a4200c50 0x7fa3a414ec60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa39800b500 tx=0x7fa39800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa392ffd700 1 -- 172.21.15.81:0/3196986154 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa398011830 con 0x7fa3a4200c50 2024-05-08T23:30:46.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a359e700 1 -- 172.21.15.81:0/3196986154 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a41513a0 con 0x7fa3a4200c50 2024-05-08T23:30:46.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a359e700 1 -- 172.21.15.81:0/3196986154 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a414f1a0 con 0x7fa3a4200c50 2024-05-08T23:30:46.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa392ffd700 1 -- 172.21.15.81:0/3196986154 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa398011990 con 0x7fa3a4200c50 2024-05-08T23:30:46.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa392ffd700 1 -- 172.21.15.81:0/3196986154 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa398011d40 con 0x7fa3a4200c50 2024-05-08T23:30:46.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.113+0000 7fa3a359e700 1 -- 172.21.15.81:0/3196986154 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa388005310 con 0x7fa3a4200c50 2024-05-08T23:30:46.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.117+0000 7fa392ffd700 1 -- 172.21.15.81:0/3196986154 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa39801a400 con 0x7fa3a4200c50 2024-05-08T23:30:46.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.117+0000 7fa392ffd700 1 --2- 172.21.15.81:0/3196986154 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa384066730 0x7fa384068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.117+0000 7fa392ffd700 1 -- 172.21.15.81:0/3196986154 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3980840b0 con 0x7fa3a4200c50 2024-05-08T23:30:46.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.117+0000 7fa3a259c700 1 --2- 172.21.15.81:0/3196986154 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa384066730 0x7fa384068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.117+0000 7fa3a259c700 1 --2- 172.21.15.81:0/3196986154 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa384066730 0x7fa384068bf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3940048f0 tx=0x7fa394012040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.125+0000 7fa392ffd700 1 -- 172.21.15.81:0/3196986154 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa398051420 con 0x7fa3a4200c50 2024-05-08T23:30:46.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.337+0000 7f73d67da700 1 -- 172.21.15.81:0/3951275427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f73b00055c0 con 0x7f73d006e3a0 2024-05-08T23:30:46.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.337+0000 7f73ccff9700 1 -- 172.21.15.81:0/3951275427 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f73c00549a0 con 0x7f73d006e3a0 2024-05-08T23:30:46.344 INFO:teuthology.orchestra.run.smithi081.stdout:64424509446 2024-05-08T23:30:46.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.345+0000 7f73b67fc700 1 -- 172.21.15.81:0/3951275427 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f73bc088890 msgr2=0x7f73bc08ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.345+0000 7f73b67fc700 1 --2- 172.21.15.81:0/3951275427 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f73bc088890 0x7f73bc08ad50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f73d0164fb0 tx=0x7f73c4008040 comp rx=0 tx=0).stop 2024-05-08T23:30:46.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.345+0000 7f73b67fc700 1 -- 172.21.15.81:0/3951275427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d006e3a0 msgr2=0x7f73d01466c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.345+0000 7f73b67fc700 1 --2- 172.21.15.81:0/3951275427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d006e3a0 0x7f73d01466c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f73c000c550 tx=0x7f73c000c910 comp rx=0 tx=0).stop 2024-05-08T23:30:46.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.345+0000 7f73b67fc700 1 -- 172.21.15.81:0/3951275427 shutdown_connections 2024-05-08T23:30:46.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.345+0000 7f73b67fc700 1 --2- 172.21.15.81:0/3951275427 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f73bc088890 0x7f73bc08ad50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.345+0000 7f73b67fc700 1 --2- 172.21.15.81:0/3951275427 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d00cb3e0 0x7f73d0071b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.345+0000 7f73b67fc700 1 --2- 172.21.15.81:0/3951275427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d00b4310 0x7f73d0071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.345+0000 7f73b67fc700 1 --2- 172.21.15.81:0/3951275427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d006e3a0 0x7f73d01466c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.345+0000 7f73b67fc700 1 -- 172.21.15.81:0/3951275427 >> 172.21.15.81:0/3951275427 conn(0x7f73d00aebd0 msgr2=0x7f73d0139fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.345+0000 7f73b67fc700 1 -- 172.21.15.81:0/3951275427 shutdown_connections 2024-05-08T23:30:46.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.345+0000 7f73b67fc700 1 -- 172.21.15.81:0/3951275427 wait complete. 2024-05-08T23:30:46.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.349+0000 7f970d0dc700 1 -- 172.21.15.81:0/512131939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f97081e8310 con 0x7f9708160c80 2024-05-08T23:30:46.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.349+0000 7f96ef7fe700 1 -- 172.21.15.81:0/512131939 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9700019020 con 0x7f9708160c80 2024-05-08T23:30:46.364 INFO:teuthology.orchestra.run.smithi081.stdout:60129542151 2024-05-08T23:30:46.364 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.4 2024-05-08T23:30:46.364 DEBUG:teuthology.parallel:result is None 2024-05-08T23:30:46.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.361+0000 7f970d0dc700 1 -- 172.21.15.81:0/512131939 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f96f40667b0 msgr2=0x7f96f4068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.361+0000 7f970d0dc700 1 --2- 172.21.15.81:0/512131939 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f96f40667b0 0x7f96f4068c70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f96fc004ef0 tx=0x7f96fc001080 comp rx=0 tx=0).stop 2024-05-08T23:30:46.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.361+0000 7f970d0dc700 1 -- 172.21.15.81:0/512131939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9708160c80 msgr2=0x7f97081e6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.361+0000 7f970d0dc700 1 --2- 172.21.15.81:0/512131939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9708160c80 0x7f97081e6ab0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9708072900 tx=0x7f9700009f40 comp rx=0 tx=0).stop 2024-05-08T23:30:46.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.361+0000 7f970d0dc700 1 -- 172.21.15.81:0/512131939 shutdown_connections 2024-05-08T23:30:46.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.361+0000 7f970d0dc700 1 --2- 172.21.15.81:0/512131939 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f96f40667b0 0x7f96f4068c70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.361+0000 7f970d0dc700 1 --2- 172.21.15.81:0/512131939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97081e6ff0 0x7f97080aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.361+0000 7f970d0dc700 1 --2- 172.21.15.81:0/512131939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9708160c80 0x7f97081e6ab0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.361+0000 7f970d0dc700 1 --2- 172.21.15.81:0/512131939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97080ac8e0 0x7f970815f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.361+0000 7f970d0dc700 1 -- 172.21.15.81:0/512131939 >> 172.21.15.81:0/512131939 conn(0x7f97080aebd0 msgr2=0x7f9708071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.361+0000 7f970d0dc700 1 -- 172.21.15.81:0/512131939 shutdown_connections 2024-05-08T23:30:46.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.361+0000 7f970d0dc700 1 -- 172.21.15.81:0/512131939 wait complete. 2024-05-08T23:30:46.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.365+0000 7fbb9bfff700 1 -- 172.21.15.81:0/3146255897 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbb7c005890 con 0x7fbb9c071e70 2024-05-08T23:30:46.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.365+0000 7fbb837fe700 1 -- 172.21.15.81:0/3146255897 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb8c0532f0 con 0x7fbb9c071e70 2024-05-08T23:30:46.370 INFO:teuthology.orchestra.run.smithi081.stdout:64424509446 2024-05-08T23:30:46.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.369+0000 7fbb9bfff700 1 -- 172.21.15.81:0/3146255897 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbb88066780 msgr2=0x7fbb88068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.369+0000 7fbb9bfff700 1 --2- 172.21.15.81:0/3146255897 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbb88066780 0x7fbb88068c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0cba80 tx=0x7fbb90008040 comp rx=0 tx=0).stop 2024-05-08T23:30:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.369+0000 7fbb9bfff700 1 -- 172.21.15.81:0/3146255897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9c071e70 msgr2=0x7fbb9c15b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.369+0000 7fbb9bfff700 1 --2- 172.21.15.81:0/3146255897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9c071e70 0x7fbb9c15b590 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c014040 tx=0x7fbb8c004770 comp rx=0 tx=0).stop 2024-05-08T23:30:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.369+0000 7fbb9bfff700 1 -- 172.21.15.81:0/3146255897 shutdown_connections 2024-05-08T23:30:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.369+0000 7fbb9bfff700 1 --2- 172.21.15.81:0/3146255897 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbb88066780 0x7fbb88068c40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.369+0000 7fbb9bfff700 1 --2- 172.21.15.81:0/3146255897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb9c165070 0x7fbb9c0b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.369+0000 7fbb9bfff700 1 --2- 172.21.15.81:0/3146255897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb9c0b3c20 0x7fbb9c0afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.369+0000 7fbb9bfff700 1 --2- 172.21.15.81:0/3146255897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb9c071e70 0x7fbb9c15b590 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.369+0000 7fbb9bfff700 1 -- 172.21.15.81:0/3146255897 >> 172.21.15.81:0/3146255897 conn(0x7fbb9c06c330 msgr2=0x7fbb9c0b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.369+0000 7fbb9bfff700 1 -- 172.21.15.81:0/3146255897 shutdown_connections 2024-05-08T23:30:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.369+0000 7fbb9bfff700 1 -- 172.21.15.81:0/3146255897 wait complete. 2024-05-08T23:30:46.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.0 2024-05-08T23:30:46.380 DEBUG:teuthology.parallel:result is None 2024-05-08T23:30:46.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.2 2024-05-08T23:30:46.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.377+0000 7f2397fff700 1 -- 172.21.15.81:0/3290732531 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f23ac0a84d0 con 0x7f23ac0b1f60 2024-05-08T23:30:46.383 DEBUG:teuthology.parallel:result is None 2024-05-08T23:30:46.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.377+0000 7f23a27fc700 1 -- 172.21.15.81:0/3290732531 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f23a8023020 con 0x7f23ac0b1f60 2024-05-08T23:30:46.384 INFO:teuthology.orchestra.run.smithi081.stdout:64424509448 2024-05-08T23:30:46.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.381+0000 7f2397fff700 1 -- 172.21.15.81:0/3290732531 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2390066760 msgr2=0x7f2390068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.381+0000 7f2397fff700 1 --2- 172.21.15.81:0/3290732531 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2390066760 0x7f2390068c20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f239c006fd0 tx=0x7f239c008040 comp rx=0 tx=0).stop 2024-05-08T23:30:46.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.381+0000 7f2397fff700 1 -- 172.21.15.81:0/3290732531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23ac0b1f60 msgr2=0x7f23ac1583d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.381+0000 7f2397fff700 1 --2- 172.21.15.81:0/3290732531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23ac0b1f60 0x7f23ac1583d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f23a800d730 tx=0x7f23a8004770 comp rx=0 tx=0).stop 2024-05-08T23:30:46.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.381+0000 7f2397fff700 1 -- 172.21.15.81:0/3290732531 shutdown_connections 2024-05-08T23:30:46.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.381+0000 7f2397fff700 1 --2- 172.21.15.81:0/3290732531 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2390066760 0x7f2390068c20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.381+0000 7f2397fff700 1 --2- 172.21.15.81:0/3290732531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ac0cdc20 0x7f23ac1f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.381+0000 7f2397fff700 1 --2- 172.21.15.81:0/3290732531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23ac0b1f60 0x7f23ac1583d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.381+0000 7f2397fff700 1 --2- 172.21.15.81:0/3290732531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23ac06d820 0x7f23ac1f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.381+0000 7f2397fff700 1 -- 172.21.15.81:0/3290732531 >> 172.21.15.81:0/3290732531 conn(0x7f23ac0aebd0 msgr2=0x7f23ac0b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.381+0000 7f2397fff700 1 -- 172.21.15.81:0/3290732531 shutdown_connections 2024-05-08T23:30:46.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.381+0000 7f2397fff700 1 -- 172.21.15.81:0/3290732531 wait complete. 2024-05-08T23:30:46.396 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.5 2024-05-08T23:30:46.396 DEBUG:teuthology.parallel:result is None 2024-05-08T23:30:46.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.397+0000 7fb68a579700 1 -- 172.21.15.81:0/3117596422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb6840affc0 con 0x7fb670003a20 2024-05-08T23:30:46.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.397+0000 7fb679ffb700 1 -- 172.21.15.81:0/3117596422 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb680017020 con 0x7fb670003a20 2024-05-08T23:30:46.402 INFO:teuthology.orchestra.run.smithi081.stdout:64424509446 2024-05-08T23:30:46.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.401+0000 7fb68a579700 1 -- 172.21.15.81:0/3117596422 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6680667b0 msgr2=0x7fb668068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.401+0000 7fb68a579700 1 --2- 172.21.15.81:0/3117596422 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6680667b0 0x7fb668068c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb674006fd0 tx=0x7fb674008040 comp rx=0 tx=0).stop 2024-05-08T23:30:46.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.401+0000 7fb68a579700 1 -- 172.21.15.81:0/3117596422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb670003a20 msgr2=0x7fb68414db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.401+0000 7fb68a579700 1 --2- 172.21.15.81:0/3117596422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb670003a20 0x7fb68414db10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb680015000 tx=0x7fb6800047e0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.401+0000 7fb68a579700 1 -- 172.21.15.81:0/3117596422 shutdown_connections 2024-05-08T23:30:46.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.401+0000 7fb68a579700 1 --2- 172.21.15.81:0/3117596422 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6680667b0 0x7fb668068c70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.401+0000 7fb68a579700 1 --2- 172.21.15.81:0/3117596422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb684154a00 0x7fb684154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.401+0000 7fb68a579700 1 --2- 172.21.15.81:0/3117596422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb684156b80 0x7fb684157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.401+0000 7fb68a579700 1 --2- 172.21.15.81:0/3117596422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb670003a20 0x7fb68414db10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.401+0000 7fb68a579700 1 -- 172.21.15.81:0/3117596422 >> 172.21.15.81:0/3117596422 conn(0x7fb6840aebd0 msgr2=0x7fb684070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.401+0000 7fb68a579700 1 -- 172.21.15.81:0/3117596422 shutdown_connections 2024-05-08T23:30:46.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.401+0000 7fb68a579700 1 -- 172.21.15.81:0/3117596422 wait complete. 2024-05-08T23:30:46.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.409+0000 7fa3a359e700 1 -- 172.21.15.81:0/3196986154 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa388005890 con 0x7fa3a4200c50 2024-05-08T23:30:46.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.409+0000 7fa392ffd700 1 -- 172.21.15.81:0/3196986154 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa398009300 con 0x7fa3a4200c50 2024-05-08T23:30:46.412 INFO:teuthology.orchestra.run.smithi081.stdout:64424509448 2024-05-08T23:30:46.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.1 2024-05-08T23:30:46.414 DEBUG:teuthology.parallel:result is None 2024-05-08T23:30:46.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.409+0000 7fa3a359e700 1 -- 172.21.15.81:0/3196986154 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa384066730 msgr2=0x7fa384068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.409+0000 7fa3a359e700 1 --2- 172.21.15.81:0/3196986154 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa384066730 0x7fa384068bf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3940048f0 tx=0x7fa394012040 comp rx=0 tx=0).stop 2024-05-08T23:30:46.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.409+0000 7fa3a359e700 1 -- 172.21.15.81:0/3196986154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3a4200c50 msgr2=0x7fa3a414ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.409+0000 7fa3a359e700 1 --2- 172.21.15.81:0/3196986154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3a4200c50 0x7fa3a414ec60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa39800b500 tx=0x7fa39800b8c0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.409+0000 7fa3a359e700 1 -- 172.21.15.81:0/3196986154 shutdown_connections 2024-05-08T23:30:46.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.409+0000 7fa3a359e700 1 --2- 172.21.15.81:0/3196986154 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa384066730 0x7fa384068bf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.409+0000 7fa3a359e700 1 --2- 172.21.15.81:0/3196986154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3a42010d0 0x7fa3a4203080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.409+0000 7fa3a359e700 1 --2- 172.21.15.81:0/3196986154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3a4200c50 0x7fa3a414ec60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.409+0000 7fa3a359e700 1 --2- 172.21.15.81:0/3196986154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c005430 0x7fa3a414fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.409+0000 7fa3a359e700 1 -- 172.21.15.81:0/3196986154 >> 172.21.15.81:0/3196986154 conn(0x7fa3a40aebd0 msgr2=0x7fa3a40aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.409+0000 7fa3a359e700 1 -- 172.21.15.81:0/3196986154 shutdown_connections 2024-05-08T23:30:46.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.409+0000 7fa3a359e700 1 -- 172.21.15.81:0/3196986154 wait complete. 2024-05-08T23:30:46.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.413+0000 7fcb1b698700 1 -- 172.21.15.81:0/594645661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fcaf40055c0 con 0x7fcb1415f860 2024-05-08T23:30:46.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.413+0000 7fcb09ffb700 1 -- 172.21.15.81:0/594645661 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcb0c02b020 con 0x7fcb1415f860 2024-05-08T23:30:46.417 INFO:teuthology.orchestra.run.smithi081.stdout:60129542151 2024-05-08T23:30:46.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7f42b6615700 1 -- 172.21.15.81:0/1697202109 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4290005890 con 0x7f429c003a70 2024-05-08T23:30:46.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7fcb1b698700 1 -- 172.21.15.81:0/594645661 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcaf8066730 msgr2=0x7fcaf8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7fcb1b698700 1 --2- 172.21.15.81:0/594645661 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcaf8066730 0x7fcaf8068bf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcb10005ae0 tx=0x7fcb10009040 comp rx=0 tx=0).stop 2024-05-08T23:30:46.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7fcb1b698700 1 -- 172.21.15.81:0/594645661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb1415f860 msgr2=0x7fcb1415fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7fcb1b698700 1 --2- 172.21.15.81:0/594645661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb1415f860 0x7fcb1415fcf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcb000047d0 tx=0x7fcb0c009750 comp rx=0 tx=0).stop 2024-05-08T23:30:46.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7fcb1b698700 1 -- 172.21.15.81:0/594645661 shutdown_connections 2024-05-08T23:30:46.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7fcb1b698700 1 --2- 172.21.15.81:0/594645661 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcaf8066730 0x7fcaf8068bf0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7fcb1b698700 1 --2- 172.21.15.81:0/594645661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb1415f860 0x7fcb1415fcf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7fcb1b698700 1 --2- 172.21.15.81:0/594645661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb1413bbc0 0x7fcb1413c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7fcb1b698700 1 --2- 172.21.15.81:0/594645661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb00004db0 0x7fcb14139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7fcb1b698700 1 -- 172.21.15.81:0/594645661 >> 172.21.15.81:0/594645661 conn(0x7fcb140b0cd0 msgr2=0x7fcb140b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.422 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7fcb1b698700 1 -- 172.21.15.81:0/594645661 shutdown_connections 2024-05-08T23:30:46.422 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7fcb1b698700 1 -- 172.21.15.81:0/594645661 wait complete. 2024-05-08T23:30:46.422 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7f42a5ffb700 1 -- 172.21.15.81:0/1697202109 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f42ac019070 con 0x7f429c003a70 2024-05-08T23:30:46.422 INFO:teuthology.orchestra.run.smithi081.stdout:64424509446 2024-05-08T23:30:46.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7f42b6615700 1 -- 172.21.15.81:0/1697202109 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42940667b0 msgr2=0x7f4294068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7f42b6615700 1 --2- 172.21.15.81:0/1697202109 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42940667b0 0x7f4294068c70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f42b01fa6a0 tx=0x7f42a8006040 comp rx=0 tx=0).stop 2024-05-08T23:30:46.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7f42b6615700 1 -- 172.21.15.81:0/1697202109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f429c003a70 msgr2=0x7f42b01f9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7f42b6615700 1 --2- 172.21.15.81:0/1697202109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f429c003a70 0x7f42b01f9e80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f42ac016000 tx=0x7f42ac007250 comp rx=0 tx=0).stop 2024-05-08T23:30:46.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7f42b6615700 1 -- 172.21.15.81:0/1697202109 shutdown_connections 2024-05-08T23:30:46.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7f42b6615700 1 --2- 172.21.15.81:0/1697202109 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42940667b0 0x7f4294068c70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.424 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7f42b6615700 1 --2- 172.21.15.81:0/1697202109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42b00aa3e0 0x7f42b00aa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.424 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7f42b6615700 1 --2- 172.21.15.81:0/1697202109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42b00a84d0 0x7f42b00a8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.424 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7f42b6615700 1 --2- 172.21.15.81:0/1697202109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f429c003a70 0x7f42b01f9e80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.424 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7f42b6615700 1 -- 172.21.15.81:0/1697202109 >> 172.21.15.81:0/1697202109 conn(0x7f42b00aebd0 msgr2=0x7f42b00aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.424 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7f42b6615700 1 -- 172.21.15.81:0/1697202109 shutdown_connections 2024-05-08T23:30:46.424 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.417+0000 7f42b6615700 1 -- 172.21.15.81:0/1697202109 wait complete. 2024-05-08T23:30:46.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.3 2024-05-08T23:30:46.434 DEBUG:teuthology.parallel:result is None 2024-05-08T23:30:46.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.6 2024-05-08T23:30:46.434 DEBUG:teuthology.parallel:result is None 2024-05-08T23:30:46.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.7 2024-05-08T23:30:46.436 DEBUG:teuthology.parallel:result is None 2024-05-08T23:30:46.436 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T23:30:46.436 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:30:46.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.533+0000 7f543a6b2700 1 Processor -- start 2024-05-08T23:30:46.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.533+0000 7f543a6b2700 1 -- start start 2024-05-08T23:30:46.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.533+0000 7f543a6b2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5434070110 0x7f54340b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.533+0000 7f543a6b2700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f54340704f0 0x7f5434159db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.533+0000 7f543a6b2700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54340b3590 con 0x7f54340b4e50 2024-05-08T23:30:46.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.533+0000 7f543a6b2700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5434071ab0 con 0x7f54340704f0 2024-05-08T23:30:46.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.533+0000 7f543a6b2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54340b24f0 con 0x7f5434070110 2024-05-08T23:30:46.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.533+0000 7f5439eb1700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f54340704f0 0x7f5434159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.533+0000 7f54396b0700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5434070110 0x7f54340b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.533+0000 7f5439eb1700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f54340704f0 0x7f5434159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:53900/0 (socket says 172.21.15.81:53900) 2024-05-08T23:30:46.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.533+0000 7f5438eaf700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f54340b4e50 0x7f54341596a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:47740/0 (socket says 172.21.15.81:47740) 2024-05-08T23:30:46.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.533+0000 7f5439eb1700 1 -- 172.21.15.81:0/1180924735 learned_addr learned my addr 172.21.15.81:0/1180924735 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:46.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f5439eb1700 1 -- 172.21.15.81:0/1180924735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5434070110 msgr2=0x7f54340b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f5439eb1700 1 --2- 172.21.15.81:0/1180924735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5434070110 0x7f54340b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f5439eb1700 1 -- 172.21.15.81:0/1180924735 >> v1:172.21.15.117:6789/0 conn(0x7f54340b4e50 legacy=0x7f54341596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f542bfff700 1 -- 172.21.15.81:0/1180924735 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693196529 0 0) 0x7f54340b3590 con 0x7f54340b4e50 2024-05-08T23:30:46.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f5439eb1700 1 -- 172.21.15.81:0/1180924735 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54340b27d0 con 0x7f54340704f0 2024-05-08T23:30:46.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f54396b0700 1 --2- 172.21.15.81:0/1180924735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5434070110 0x7f54340b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:46.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f5439eb1700 1 --2- 172.21.15.81:0/1180924735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54340704f0 0x7f5434159db0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f542c009e50 tx=0x7f542c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac08665e86d2b078 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f542bfff700 1 -- 172.21.15.81:0/1180924735 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c00e070 con 0x7f54340704f0 2024-05-08T23:30:46.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f542bfff700 1 -- 172.21.15.81:0/1180924735 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c00b910 con 0x7f54340704f0 2024-05-08T23:30:46.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f542bfff700 1 -- 172.21.15.81:0/1180924735 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c00bcc0 con 0x7f54340704f0 2024-05-08T23:30:46.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f543a6b2700 1 -- 172.21.15.81:0/1180924735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54340704f0 msgr2=0x7f5434159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f543a6b2700 1 --2- 172.21.15.81:0/1180924735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54340704f0 0x7f5434159db0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f542c009e50 tx=0x7f542c00b330 comp rx=0 tx=0).stop 2024-05-08T23:30:46.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f543a6b2700 1 -- 172.21.15.81:0/1180924735 shutdown_connections 2024-05-08T23:30:46.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f543a6b2700 1 --2- 172.21.15.81:0/1180924735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54340704f0 0x7f5434159db0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f543a6b2700 1 --2- 172.21.15.81:0/1180924735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5434070110 0x7f54340b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f543a6b2700 1 -- 172.21.15.81:0/1180924735 >> 172.21.15.81:0/1180924735 conn(0x7f543406c330 msgr2=0x7f543406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f543a6b2700 1 -- 172.21.15.81:0/1180924735 shutdown_connections 2024-05-08T23:30:46.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.537+0000 7f543a6b2700 1 -- 172.21.15.81:0/1180924735 wait complete. 2024-05-08T23:30:46.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f543a6b2700 1 Processor -- start 2024-05-08T23:30:46.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f543a6b2700 1 -- start start 2024-05-08T23:30:46.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f543a6b2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5434070110 0x7f54341481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f543a6b2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54340704f0 0x7f54341537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f543a6b2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54340b4e50 0x7f5434153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f543a6b2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54341e2f80 con 0x7f5434070110 2024-05-08T23:30:46.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f543a6b2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54341486f0 con 0x7f54340704f0 2024-05-08T23:30:46.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f543a6b2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5434152b40 con 0x7f54340b4e50 2024-05-08T23:30:46.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f54396b0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5434070110 0x7f54341481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f54396b0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5434070110 0x7f54341481b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51144/0 (socket says 172.21.15.81:51144) 2024-05-08T23:30:46.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f54396b0700 1 -- 172.21.15.81:0/2934640756 learned_addr learned my addr 172.21.15.81:0/2934640756 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:46.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f5438eaf700 1 --2- 172.21.15.81:0/2934640756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54340704f0 0x7f54341537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f5439eb1700 1 --2- 172.21.15.81:0/2934640756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54340b4e50 0x7f5434153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f54396b0700 1 -- 172.21.15.81:0/2934640756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54340b4e50 msgr2=0x7f5434153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f54396b0700 1 --2- 172.21.15.81:0/2934640756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54340b4e50 0x7f5434153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f54396b0700 1 -- 172.21.15.81:0/2934640756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54340704f0 msgr2=0x7f54341537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f54396b0700 1 --2- 172.21.15.81:0/2934640756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54340704f0 0x7f54341537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f54396b0700 1 -- 172.21.15.81:0/2934640756 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54300094b0 con 0x7f5434070110 2024-05-08T23:30:46.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f54396b0700 1 --2- 172.21.15.81:0/2934640756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5434070110 0x7f54341481b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f543000ebe0 tx=0x7f543000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f5429ffb700 1 -- 172.21.15.81:0/2934640756 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430009770 con 0x7f5434070110 2024-05-08T23:30:46.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f5429ffb700 1 -- 172.21.15.81:0/2934640756 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543000c850 con 0x7f5434070110 2024-05-08T23:30:46.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f543a6b2700 1 -- 172.21.15.81:0/2934640756 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542c009b30 con 0x7f5434070110 2024-05-08T23:30:46.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.541+0000 7f5429ffb700 1 -- 172.21.15.81:0/2934640756 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543000cbe0 con 0x7f5434070110 2024-05-08T23:30:46.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.545+0000 7f543a6b2700 1 -- 172.21.15.81:0/2934640756 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5434153100 con 0x7f5434070110 2024-05-08T23:30:46.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.545+0000 7f543a6b2700 1 -- 172.21.15.81:0/2934640756 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54340684b0 con 0x7f5434070110 2024-05-08T23:30:46.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.545+0000 7f5429ffb700 1 -- 172.21.15.81:0/2934640756 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f543000cd40 con 0x7f5434070110 2024-05-08T23:30:46.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.545+0000 7f5429ffb700 1 --2- 172.21.15.81:0/2934640756 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5418066730 0x7f5418068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.545+0000 7f5429ffb700 1 -- 172.21.15.81:0/2934640756 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54300841b0 con 0x7f5434070110 2024-05-08T23:30:46.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.545+0000 7f5438eaf700 1 --2- 172.21.15.81:0/2934640756 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5418066730 0x7f5418068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.545+0000 7f5438eaf700 1 --2- 172.21.15.81:0/2934640756 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5418066730 0x7f5418068bf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5434151990 tx=0x7f5424007e90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.549+0000 7f5429ffb700 1 -- 172.21.15.81:0/2934640756 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5430051520 con 0x7f5434070110 2024-05-08T23:30:46.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.709+0000 7f543a6b2700 1 -- 172.21.15.81:0/2934640756 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54340b3070 con 0x7f5418066730 2024-05-08T23:30:46.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.721+0000 7f5429ffb700 1 -- 172.21.15.81:0/2934640756 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264587 (secure 0 0 0) 0x7f54240012b0 con 0x7f5418066730 2024-05-08T23:30:46.730 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:46.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.725+0000 7f543a6b2700 1 -- 172.21.15.81:0/2934640756 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5418066730 msgr2=0x7f5418068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.725+0000 7f543a6b2700 1 --2- 172.21.15.81:0/2934640756 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5418066730 0x7f5418068bf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5434151990 tx=0x7f5424007e90 comp rx=0 tx=0).stop 2024-05-08T23:30:46.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.725+0000 7f543a6b2700 1 -- 172.21.15.81:0/2934640756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5434070110 msgr2=0x7f54341481b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.725+0000 7f543a6b2700 1 --2- 172.21.15.81:0/2934640756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5434070110 0x7f54341481b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f543000ebe0 tx=0x7f543000eef0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.725+0000 7f543a6b2700 1 -- 172.21.15.81:0/2934640756 shutdown_connections 2024-05-08T23:30:46.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.725+0000 7f543a6b2700 1 --2- 172.21.15.81:0/2934640756 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5418066730 0x7f5418068bf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.725+0000 7f543a6b2700 1 --2- 172.21.15.81:0/2934640756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54340b4e50 0x7f5434153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.725+0000 7f543a6b2700 1 --2- 172.21.15.81:0/2934640756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54340704f0 0x7f54341537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.725+0000 7f543a6b2700 1 --2- 172.21.15.81:0/2934640756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5434070110 0x7f54341481b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.725+0000 7f543a6b2700 1 -- 172.21.15.81:0/2934640756 >> 172.21.15.81:0/2934640756 conn(0x7f543406c330 msgr2=0x7f543406cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.725+0000 7f543a6b2700 1 -- 172.21.15.81:0/2934640756 shutdown_connections 2024-05-08T23:30:46.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.725+0000 7f543a6b2700 1 -- 172.21.15.81:0/2934640756 wait complete. 2024-05-08T23:30:46.733 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:30:46.745 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-05-08T23:30:45.326555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219152,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755568,"statfs":{"total":773094113280,"available":772869701632,"internally_reserved":0,"allocated":3022848,"data_stored":1799972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.032511"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388868+0000","last_change":"2024-05-08T23:30:38.707899+0000","last_active":"2024-05-08T23:30:39.388868+0000","last_peered":"2024-05-08T23:30:39.388868+0000","last_clean":"2024-05-08T23:30:39.388868+0000","last_became_active":"2024-05-08T23:30:38.706532+0000","last_became_peered":"2024-05-08T23:30:38.706532+0000","last_unstale":"2024-05-08T23:30:39.388868+0000","last_undegraded":"2024-05-08T23:30:39.388868+0000","last_fullsized":"2024-05-08T23:30:39.388868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:21.109032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389098+0000","last_change":"2024-05-08T23:30:38.684226+0000","last_active":"2024-05-08T23:30:40.389098+0000","last_peered":"2024-05-08T23:30:40.389098+0000","last_clean":"2024-05-08T23:30:40.389098+0000","last_became_active":"2024-05-08T23:30:38.683958+0000","last_became_peered":"2024-05-08T23:30:38.683958+0000","last_unstale":"2024-05-08T23:30:40.389098+0000","last_undegraded":"2024-05-08T23:30:40.389098+0000","last_fullsized":"2024-05-08T23:30:40.389098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:07.222797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392385+0000","last_change":"2024-05-08T23:30:38.686289+0000","last_active":"2024-05-08T23:30:39.392385+0000","last_peered":"2024-05-08T23:30:39.392385+0000","last_clean":"2024-05-08T23:30:39.392385+0000","last_became_active":"2024-05-08T23:30:38.685661+0000","last_became_peered":"2024-05-08T23:30:38.685661+0000","last_unstale":"2024-05-08T23:30:39.392385+0000","last_undegraded":"2024-05-08T23:30:39.392385+0000","last_fullsized":"2024-05-08T23:30:39.392385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:34.727934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243711+0000","last_change":"2024-05-08T23:30:38.694133+0000","last_active":"2024-05-08T23:30:40.243711+0000","last_peered":"2024-05-08T23:30:40.243711+0000","last_clean":"2024-05-08T23:30:40.243711+0000","last_became_active":"2024-05-08T23:30:38.693846+0000","last_became_peered":"2024-05-08T23:30:38.693846+0000","last_unstale":"2024-05-08T23:30:40.243711+0000","last_undegraded":"2024-05-08T23:30:40.243711+0000","last_fullsized":"2024-05-08T23:30:40.243711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:44.481684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388606+0000","last_change":"2024-05-08T23:30:38.705630+0000","last_active":"2024-05-08T23:30:39.388606+0000","last_peered":"2024-05-08T23:30:39.388606+0000","last_clean":"2024-05-08T23:30:39.388606+0000","last_became_active":"2024-05-08T23:30:38.705310+0000","last_became_peered":"2024-05-08T23:30:38.705310+0000","last_unstale":"2024-05-08T23:30:39.388606+0000","last_undegraded":"2024-05-08T23:30:39.388606+0000","last_fullsized":"2024-05-08T23:30:39.388606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:41.949561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243496+0000","last_change":"2024-05-08T23:30:38.691423+0000","last_active":"2024-05-08T23:30:40.243496+0000","last_peered":"2024-05-08T23:30:40.243496+0000","last_clean":"2024-05-08T23:30:40.243496+0000","last_became_active":"2024-05-08T23:30:38.690973+0000","last_became_peered":"2024-05-08T23:30:38.690973+0000","last_unstale":"2024-05-08T23:30:40.243496+0000","last_undegraded":"2024-05-08T23:30:40.243496+0000","last_fullsized":"2024-05-08T23:30:40.243496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:42.503600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388516+0000","last_change":"2024-05-08T23:30:38.702702+0000","last_active":"2024-05-08T23:30:39.388516+0000","last_peered":"2024-05-08T23:30:39.388516+0000","last_clean":"2024-05-08T23:30:39.388516+0000","last_became_active":"2024-05-08T23:30:38.702341+0000","last_became_peered":"2024-05-08T23:30:38.702341+0000","last_unstale":"2024-05-08T23:30:39.388516+0000","last_undegraded":"2024-05-08T23:30:39.388516+0000","last_fullsized":"2024-05-08T23:30:39.388516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:05.546774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675055+0000","last_change":"2024-05-08T23:30:38.692708+0000","last_active":"2024-05-08T23:30:39.675055+0000","last_peered":"2024-05-08T23:30:39.675055+0000","last_clean":"2024-05-08T23:30:39.675055+0000","last_became_active":"2024-05-08T23:30:38.692424+0000","last_became_peered":"2024-05-08T23:30:38.692424+0000","last_unstale":"2024-05-08T23:30:39.675055+0000","last_undegraded":"2024-05-08T23:30:39.675055+0000","last_fullsized":"2024-05-08T23:30:39.675055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:58.747769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388895+0000","last_change":"2024-05-08T23:30:38.708610+0000","last_active":"2024-05-08T23:30:39.388895+0000","last_peered":"2024-05-08T23:30:39.388895+0000","last_clean":"2024-05-08T23:30:39.388895+0000","last_became_active":"2024-05-08T23:30:38.708389+0000","last_became_peered":"2024-05-08T23:30:38.708389+0000","last_unstale":"2024-05-08T23:30:39.388895+0000","last_undegraded":"2024-05-08T23:30:39.388895+0000","last_fullsized":"2024-05-08T23:30:39.388895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:41.257114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392823+0000","last_change":"2024-05-08T23:30:38.690662+0000","last_active":"2024-05-08T23:30:40.392823+0000","last_peered":"2024-05-08T23:30:40.392823+0000","last_clean":"2024-05-08T23:30:40.392823+0000","last_became_active":"2024-05-08T23:30:38.690378+0000","last_became_peered":"2024-05-08T23:30:38.690378+0000","last_unstale":"2024-05-08T23:30:40.392823+0000","last_undegraded":"2024-05-08T23:30:40.392823+0000","last_fullsized":"2024-05-08T23:30:40.392823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:33.598895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392801+0000","last_change":"2024-05-08T23:30:38.692171+0000","last_active":"2024-05-08T23:30:40.392801+0000","last_peered":"2024-05-08T23:30:40.392801+0000","last_clean":"2024-05-08T23:30:40.392801+0000","last_became_active":"2024-05-08T23:30:38.691611+0000","last_became_peered":"2024-05-08T23:30:38.691611+0000","last_unstale":"2024-05-08T23:30:40.392801+0000","last_undegraded":"2024-05-08T23:30:40.392801+0000","last_fullsized":"2024-05-08T23:30:40.392801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959822+0000","last_change":"2024-05-08T23:30:38.694336+0000","last_active":"2024-05-08T23:30:39.959822+0000","last_peered":"2024-05-08T23:30:39.959822+0000","last_clean":"2024-05-08T23:30:39.959822+0000","last_became_active":"2024-05-08T23:30:38.694038+0000","last_became_peered":"2024-05-08T23:30:38.694038+0000","last_unstale":"2024-05-08T23:30:39.959822+0000","last_undegraded":"2024-05-08T23:30:39.959822+0000","last_fullsized":"2024-05-08T23:30:39.959822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:07.443142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676123+0000","last_change":"2024-05-08T23:30:38.691325+0000","last_active":"2024-05-08T23:30:39.676123+0000","last_peered":"2024-05-08T23:30:39.676123+0000","last_clean":"2024-05-08T23:30:39.676123+0000","last_became_active":"2024-05-08T23:30:38.691074+0000","last_became_peered":"2024-05-08T23:30:38.691074+0000","last_unstale":"2024-05-08T23:30:39.676123+0000","last_undegraded":"2024-05-08T23:30:39.676123+0000","last_fullsized":"2024-05-08T23:30:39.676123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:37.994049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392094+0000","last_change":"2024-05-08T23:30:38.679460+0000","last_active":"2024-05-08T23:30:39.392094+0000","last_peered":"2024-05-08T23:30:39.392094+0000","last_clean":"2024-05-08T23:30:39.392094+0000","last_became_active":"2024-05-08T23:30:38.678871+0000","last_became_peered":"2024-05-08T23:30:38.678871+0000","last_unstale":"2024-05-08T23:30:39.392094+0000","last_undegraded":"2024-05-08T23:30:39.392094+0000","last_fullsized":"2024-05-08T23:30:39.392094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:37.465394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388721+0000","last_change":"2024-05-08T23:30:38.692896+0000","last_active":"2024-05-08T23:30:40.388721+0000","last_peered":"2024-05-08T23:30:40.388721+0000","last_clean":"2024-05-08T23:30:40.388721+0000","last_became_active":"2024-05-08T23:30:38.692347+0000","last_became_peered":"2024-05-08T23:30:38.692347+0000","last_unstale":"2024-05-08T23:30:40.388721+0000","last_undegraded":"2024-05-08T23:30:40.388721+0000","last_fullsized":"2024-05-08T23:30:40.388721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:01.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243296+0000","last_change":"2024-05-08T23:30:38.689233+0000","last_active":"2024-05-08T23:30:40.243296+0000","last_peered":"2024-05-08T23:30:40.243296+0000","last_clean":"2024-05-08T23:30:40.243296+0000","last_became_active":"2024-05-08T23:30:38.688984+0000","last_became_peered":"2024-05-08T23:30:38.688984+0000","last_unstale":"2024-05-08T23:30:40.243296+0000","last_undegraded":"2024-05-08T23:30:40.243296+0000","last_fullsized":"2024-05-08T23:30:40.243296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:07.682170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244541+0000","last_change":"2024-05-08T23:30:38.706139+0000","last_active":"2024-05-08T23:30:40.244541+0000","last_peered":"2024-05-08T23:30:40.244541+0000","last_clean":"2024-05-08T23:30:40.244541+0000","last_became_active":"2024-05-08T23:30:38.705878+0000","last_became_peered":"2024-05-08T23:30:38.705878+0000","last_unstale":"2024-05-08T23:30:40.244541+0000","last_undegraded":"2024-05-08T23:30:40.244541+0000","last_fullsized":"2024-05-08T23:30:40.244541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:39.788098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389205+0000","last_change":"2024-05-08T23:30:38.685704+0000","last_active":"2024-05-08T23:30:40.389205+0000","last_peered":"2024-05-08T23:30:40.389205+0000","last_clean":"2024-05-08T23:30:40.389205+0000","last_became_active":"2024-05-08T23:30:38.685435+0000","last_became_peered":"2024-05-08T23:30:38.685435+0000","last_unstale":"2024-05-08T23:30:40.389205+0000","last_undegraded":"2024-05-08T23:30:40.389205+0000","last_fullsized":"2024-05-08T23:30:40.389205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:07.443142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388798+0000","last_change":"2024-05-08T23:30:38.710968+0000","last_active":"2024-05-08T23:30:39.388798+0000","last_peered":"2024-05-08T23:30:39.388798+0000","last_clean":"2024-05-08T23:30:39.388798+0000","last_became_active":"2024-05-08T23:30:38.710706+0000","last_became_peered":"2024-05-08T23:30:38.710706+0000","last_unstale":"2024-05-08T23:30:39.388798+0000","last_undegraded":"2024-05-08T23:30:39.388798+0000","last_fullsized":"2024-05-08T23:30:39.388798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:32.727647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392743+0000","last_change":"2024-05-08T23:30:38.701983+0000","last_active":"2024-05-08T23:30:40.392743+0000","last_peered":"2024-05-08T23:30:40.392743+0000","last_clean":"2024-05-08T23:30:40.392743+0000","last_became_active":"2024-05-08T23:30:38.701386+0000","last_became_peered":"2024-05-08T23:30:38.701386+0000","last_unstale":"2024-05-08T23:30:40.392743+0000","last_undegraded":"2024-05-08T23:30:40.392743+0000","last_fullsized":"2024-05-08T23:30:40.392743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:12.789913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392189+0000","last_change":"2024-05-08T23:30:38.705814+0000","last_active":"2024-05-08T23:30:39.392189+0000","last_peered":"2024-05-08T23:30:39.392189+0000","last_clean":"2024-05-08T23:30:39.392189+0000","last_became_active":"2024-05-08T23:30:38.705393+0000","last_became_peered":"2024-05-08T23:30:38.705393+0000","last_unstale":"2024-05-08T23:30:39.392189+0000","last_undegraded":"2024-05-08T23:30:39.392189+0000","last_fullsized":"2024-05-08T23:30:39.392189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:35.129372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243643+0000","last_change":"2024-05-08T23:30:38.703498+0000","last_active":"2024-05-08T23:30:40.243643+0000","last_peered":"2024-05-08T23:30:40.243643+0000","last_clean":"2024-05-08T23:30:40.243643+0000","last_became_active":"2024-05-08T23:30:38.703206+0000","last_became_peered":"2024-05-08T23:30:38.703206+0000","last_unstale":"2024-05-08T23:30:40.243643+0000","last_undegraded":"2024-05-08T23:30:40.243643+0000","last_fullsized":"2024-05-08T23:30:40.243643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:58.747769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675186+0000","last_change":"2024-05-08T23:30:38.697689+0000","last_active":"2024-05-08T23:30:39.675186+0000","last_peered":"2024-05-08T23:30:39.675186+0000","last_clean":"2024-05-08T23:30:39.675186+0000","last_became_active":"2024-05-08T23:30:38.697373+0000","last_became_peered":"2024-05-08T23:30:38.697373+0000","last_unstale":"2024-05-08T23:30:39.675186+0000","last_undegraded":"2024-05-08T23:30:39.675186+0000","last_fullsized":"2024-05-08T23:30:39.675186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:42.503600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243343+0000","last_change":"2024-05-08T23:30:38.693170+0000","last_active":"2024-05-08T23:30:40.243343+0000","last_peered":"2024-05-08T23:30:40.243343+0000","last_clean":"2024-05-08T23:30:40.243343+0000","last_became_active":"2024-05-08T23:30:38.692809+0000","last_became_peered":"2024-05-08T23:30:38.692809+0000","last_unstale":"2024-05-08T23:30:40.243343+0000","last_undegraded":"2024-05-08T23:30:40.243343+0000","last_fullsized":"2024-05-08T23:30:40.243343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:51.717280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675706+0000","last_change":"2024-05-08T23:30:38.696080+0000","last_active":"2024-05-08T23:30:39.675706+0000","last_peered":"2024-05-08T23:30:39.675706+0000","last_clean":"2024-05-08T23:30:39.675706+0000","last_became_active":"2024-05-08T23:30:38.695821+0000","last_became_peered":"2024-05-08T23:30:38.695821+0000","last_unstale":"2024-05-08T23:30:39.675706+0000","last_undegraded":"2024-05-08T23:30:39.675706+0000","last_fullsized":"2024-05-08T23:30:39.675706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:33.598895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392843+0000","last_change":"2024-05-08T23:30:38.704917+0000","last_active":"2024-05-08T23:30:40.392843+0000","last_peered":"2024-05-08T23:30:40.392843+0000","last_clean":"2024-05-08T23:30:40.392843+0000","last_became_active":"2024-05-08T23:30:38.704663+0000","last_became_peered":"2024-05-08T23:30:38.704663+0000","last_unstale":"2024-05-08T23:30:40.392843+0000","last_undegraded":"2024-05-08T23:30:40.392843+0000","last_fullsized":"2024-05-08T23:30:40.392843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:41.133153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388927+0000","last_change":"2024-05-08T23:30:38.693867+0000","last_active":"2024-05-08T23:30:40.388927+0000","last_peered":"2024-05-08T23:30:40.388927+0000","last_clean":"2024-05-08T23:30:40.388927+0000","last_became_active":"2024-05-08T23:30:38.693637+0000","last_became_peered":"2024-05-08T23:30:38.693637+0000","last_unstale":"2024-05-08T23:30:40.388927+0000","last_undegraded":"2024-05-08T23:30:40.388927+0000","last_fullsized":"2024-05-08T23:30:40.388927+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:33.598895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676136+0000","last_change":"2024-05-08T23:30:38.697753+0000","last_active":"2024-05-08T23:30:39.676136+0000","last_peered":"2024-05-08T23:30:39.676136+0000","last_clean":"2024-05-08T23:30:39.676136+0000","last_became_active":"2024-05-08T23:30:38.697337+0000","last_became_peered":"2024-05-08T23:30:38.697337+0000","last_unstale":"2024-05-08T23:30:39.676136+0000","last_undegraded":"2024-05-08T23:30:39.676136+0000","last_fullsized":"2024-05-08T23:30:39.676136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392238+0000","last_change":"2024-05-08T23:30:38.688908+0000","last_active":"2024-05-08T23:30:39.392238+0000","last_peered":"2024-05-08T23:30:39.392238+0000","last_clean":"2024-05-08T23:30:39.392238+0000","last_became_active":"2024-05-08T23:30:38.688636+0000","last_became_peered":"2024-05-08T23:30:38.688636+0000","last_unstale":"2024-05-08T23:30:39.392238+0000","last_undegraded":"2024-05-08T23:30:39.392238+0000","last_fullsized":"2024-05-08T23:30:39.392238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:41.059680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243784+0000","last_change":"2024-05-08T23:30:38.688336+0000","last_active":"2024-05-08T23:30:40.243784+0000","last_peered":"2024-05-08T23:30:40.243784+0000","last_clean":"2024-05-08T23:30:40.243784+0000","last_became_active":"2024-05-08T23:30:38.688041+0000","last_became_peered":"2024-05-08T23:30:38.688041+0000","last_unstale":"2024-05-08T23:30:40.243784+0000","last_undegraded":"2024-05-08T23:30:40.243784+0000","last_fullsized":"2024-05-08T23:30:40.243784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:17.135855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389063+0000","last_change":"2024-05-08T23:30:38.692840+0000","last_active":"2024-05-08T23:30:40.389063+0000","last_peered":"2024-05-08T23:30:40.389063+0000","last_clean":"2024-05-08T23:30:40.389063+0000","last_became_active":"2024-05-08T23:30:38.690283+0000","last_became_peered":"2024-05-08T23:30:38.690283+0000","last_unstale":"2024-05-08T23:30:40.389063+0000","last_undegraded":"2024-05-08T23:30:40.389063+0000","last_fullsized":"2024-05-08T23:30:40.389063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:11.511515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392596+0000","last_change":"2024-05-08T23:30:38.693989+0000","last_active":"2024-05-08T23:30:40.392596+0000","last_peered":"2024-05-08T23:30:40.392596+0000","last_clean":"2024-05-08T23:30:40.392596+0000","last_became_active":"2024-05-08T23:30:38.693491+0000","last_became_peered":"2024-05-08T23:30:38.693491+0000","last_unstale":"2024-05-08T23:30:40.392596+0000","last_undegraded":"2024-05-08T23:30:40.392596+0000","last_fullsized":"2024-05-08T23:30:40.392596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:07.222797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959371+0000","last_change":"2024-05-08T23:30:38.681154+0000","last_active":"2024-05-08T23:30:39.959371+0000","last_peered":"2024-05-08T23:30:39.959371+0000","last_clean":"2024-05-08T23:30:39.959371+0000","last_became_active":"2024-05-08T23:30:38.680826+0000","last_became_peered":"2024-05-08T23:30:38.680826+0000","last_unstale":"2024-05-08T23:30:39.959371+0000","last_undegraded":"2024-05-08T23:30:39.959371+0000","last_fullsized":"2024-05-08T23:30:39.959371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:11.511515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675917+0000","last_change":"2024-05-08T23:30:38.317473+0000","last_active":"2024-05-08T23:30:39.675917+0000","last_peered":"2024-05-08T23:30:39.675917+0000","last_clean":"2024-05-08T23:30:39.675917+0000","last_became_active":"2024-05-08T23:30:38.317235+0000","last_became_peered":"2024-05-08T23:30:38.317235+0000","last_unstale":"2024-05-08T23:30:39.675917+0000","last_undegraded":"2024-05-08T23:30:39.675917+0000","last_fullsized":"2024-05-08T23:30:39.675917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:31.924157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392285+0000","last_change":"2024-05-08T23:30:38.680780+0000","last_active":"2024-05-08T23:30:39.392285+0000","last_peered":"2024-05-08T23:30:39.392285+0000","last_clean":"2024-05-08T23:30:39.392285+0000","last_became_active":"2024-05-08T23:30:38.680508+0000","last_became_peered":"2024-05-08T23:30:38.680508+0000","last_unstale":"2024-05-08T23:30:39.392285+0000","last_undegraded":"2024-05-08T23:30:39.392285+0000","last_fullsized":"2024-05-08T23:30:39.392285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:24.058714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392711+0000","last_change":"2024-05-08T23:30:38.309456+0000","last_active":"2024-05-08T23:30:40.392711+0000","last_peered":"2024-05-08T23:30:40.392711+0000","last_clean":"2024-05-08T23:30:40.392711+0000","last_became_active":"2024-05-08T23:30:38.308975+0000","last_became_peered":"2024-05-08T23:30:38.308975+0000","last_unstale":"2024-05-08T23:30:40.392711+0000","last_undegraded":"2024-05-08T23:30:40.392711+0000","last_fullsized":"2024-05-08T23:30:40.392711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:00.309450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675599+0000","last_change":"2024-05-08T23:30:38.317811+0000","last_active":"2024-05-08T23:30:39.675599+0000","last_peered":"2024-05-08T23:30:39.675599+0000","last_clean":"2024-05-08T23:30:39.675599+0000","last_became_active":"2024-05-08T23:30:38.317505+0000","last_became_peered":"2024-05-08T23:30:38.317505+0000","last_unstale":"2024-05-08T23:30:39.675599+0000","last_undegraded":"2024-05-08T23:30:39.675599+0000","last_fullsized":"2024-05-08T23:30:39.675599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:14.380730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675152+0000","last_change":"2024-05-08T23:30:38.695850+0000","last_active":"2024-05-08T23:30:39.675152+0000","last_peered":"2024-05-08T23:30:39.675152+0000","last_clean":"2024-05-08T23:30:39.675152+0000","last_became_active":"2024-05-08T23:30:38.693296+0000","last_became_peered":"2024-05-08T23:30:38.693296+0000","last_unstale":"2024-05-08T23:30:39.675152+0000","last_undegraded":"2024-05-08T23:30:39.675152+0000","last_fullsized":"2024-05-08T23:30:39.675152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:33.805704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243572+0000","last_change":"2024-05-08T23:30:38.314634+0000","last_active":"2024-05-08T23:30:40.243572+0000","last_peered":"2024-05-08T23:30:40.243572+0000","last_clean":"2024-05-08T23:30:40.243572+0000","last_became_active":"2024-05-08T23:30:38.314368+0000","last_became_peered":"2024-05-08T23:30:38.314368+0000","last_unstale":"2024-05-08T23:30:40.243572+0000","last_undegraded":"2024-05-08T23:30:40.243572+0000","last_fullsized":"2024-05-08T23:30:40.243572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:12:55.004696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388682+0000","last_change":"2024-05-08T23:30:38.688337+0000","last_active":"2024-05-08T23:30:40.388682+0000","last_peered":"2024-05-08T23:30:40.388682+0000","last_clean":"2024-05-08T23:30:40.388682+0000","last_became_active":"2024-05-08T23:30:38.687594+0000","last_became_peered":"2024-05-08T23:30:38.687594+0000","last_unstale":"2024-05-08T23:30:40.388682+0000","last_undegraded":"2024-05-08T23:30:40.388682+0000","last_fullsized":"2024-05-08T23:30:40.388682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392864+0000","last_change":"2024-05-08T23:30:38.315204+0000","last_active":"2024-05-08T23:30:40.392864+0000","last_peered":"2024-05-08T23:30:40.392864+0000","last_clean":"2024-05-08T23:30:40.392864+0000","last_became_active":"2024-05-08T23:30:38.314885+0000","last_became_peered":"2024-05-08T23:30:38.314885+0000","last_unstale":"2024-05-08T23:30:40.392864+0000","last_undegraded":"2024-05-08T23:30:40.392864+0000","last_fullsized":"2024-05-08T23:30:40.392864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:03.938472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244273+0000","last_change":"2024-05-08T23:30:38.703534+0000","last_active":"2024-05-08T23:30:40.244273+0000","last_peered":"2024-05-08T23:30:40.244273+0000","last_clean":"2024-05-08T23:30:40.244273+0000","last_became_active":"2024-05-08T23:30:38.703221+0000","last_became_peered":"2024-05-08T23:30:38.703221+0000","last_unstale":"2024-05-08T23:30:40.244273+0000","last_undegraded":"2024-05-08T23:30:40.244273+0000","last_fullsized":"2024-05-08T23:30:40.244273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:58.807960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244382+0000","last_change":"2024-05-08T23:30:38.318632+0000","last_active":"2024-05-08T23:30:40.244382+0000","last_peered":"2024-05-08T23:30:40.244382+0000","last_clean":"2024-05-08T23:30:40.244382+0000","last_became_active":"2024-05-08T23:30:38.318283+0000","last_became_peered":"2024-05-08T23:30:38.318283+0000","last_unstale":"2024-05-08T23:30:40.244382+0000","last_undegraded":"2024-05-08T23:30:40.244382+0000","last_fullsized":"2024-05-08T23:30:40.244382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:00.309450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392543+0000","last_change":"2024-05-08T23:30:38.691103+0000","last_active":"2024-05-08T23:30:39.392543+0000","last_peered":"2024-05-08T23:30:39.392543+0000","last_clean":"2024-05-08T23:30:39.392543+0000","last_became_active":"2024-05-08T23:30:38.690525+0000","last_became_peered":"2024-05-08T23:30:38.690525+0000","last_unstale":"2024-05-08T23:30:39.392543+0000","last_undegraded":"2024-05-08T23:30:39.392543+0000","last_fullsized":"2024-05-08T23:30:39.392543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:21.833335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388976+0000","last_change":"2024-05-08T23:30:38.317224+0000","last_active":"2024-05-08T23:30:40.388976+0000","last_peered":"2024-05-08T23:30:40.388976+0000","last_clean":"2024-05-08T23:30:40.388976+0000","last_became_active":"2024-05-08T23:30:38.316980+0000","last_became_peered":"2024-05-08T23:30:38.316980+0000","last_unstale":"2024-05-08T23:30:40.388976+0000","last_undegraded":"2024-05-08T23:30:40.388976+0000","last_fullsized":"2024-05-08T23:30:40.388976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:10.501763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959964+0000","last_change":"2024-05-08T23:30:38.703383+0000","last_active":"2024-05-08T23:30:39.959964+0000","last_peered":"2024-05-08T23:30:39.959964+0000","last_clean":"2024-05-08T23:30:39.959964+0000","last_became_active":"2024-05-08T23:30:38.703119+0000","last_became_peered":"2024-05-08T23:30:38.703119+0000","last_unstale":"2024-05-08T23:30:39.959964+0000","last_undegraded":"2024-05-08T23:30:39.959964+0000","last_fullsized":"2024-05-08T23:30:39.959964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:01.319202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392765+0000","last_change":"2024-05-08T23:30:38.315586+0000","last_active":"2024-05-08T23:30:40.392765+0000","last_peered":"2024-05-08T23:30:40.392765+0000","last_clean":"2024-05-08T23:30:40.392765+0000","last_became_active":"2024-05-08T23:30:38.315274+0000","last_became_peered":"2024-05-08T23:30:38.315274+0000","last_unstale":"2024-05-08T23:30:40.392765+0000","last_undegraded":"2024-05-08T23:30:40.392765+0000","last_fullsized":"2024-05-08T23:30:40.392765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:31.924157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388710+0000","last_change":"2024-05-08T23:30:38.705407+0000","last_active":"2024-05-08T23:30:39.388710+0000","last_peered":"2024-05-08T23:30:39.388710+0000","last_clean":"2024-05-08T23:30:39.388710+0000","last_became_active":"2024-05-08T23:30:38.705035+0000","last_became_peered":"2024-05-08T23:30:38.705035+0000","last_unstale":"2024-05-08T23:30:39.388710+0000","last_undegraded":"2024-05-08T23:30:39.388710+0000","last_fullsized":"2024-05-08T23:30:39.388710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:49.850123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392334+0000","last_change":"2024-05-08T23:30:38.308541+0000","last_active":"2024-05-08T23:30:39.392334+0000","last_peered":"2024-05-08T23:30:39.392334+0000","last_clean":"2024-05-08T23:30:39.392334+0000","last_became_active":"2024-05-08T23:30:38.308285+0000","last_became_peered":"2024-05-08T23:30:38.308285+0000","last_unstale":"2024-05-08T23:30:39.392334+0000","last_undegraded":"2024-05-08T23:30:39.392334+0000","last_fullsized":"2024-05-08T23:30:39.392334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:12.981854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392148+0000","last_change":"2024-05-08T23:30:38.702752+0000","last_active":"2024-05-08T23:30:39.392148+0000","last_peered":"2024-05-08T23:30:39.392148+0000","last_clean":"2024-05-08T23:30:39.392148+0000","last_became_active":"2024-05-08T23:30:38.702534+0000","last_became_peered":"2024-05-08T23:30:38.702534+0000","last_unstale":"2024-05-08T23:30:39.392148+0000","last_undegraded":"2024-05-08T23:30:39.392148+0000","last_fullsized":"2024-05-08T23:30:39.392148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:40.230149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243826+0000","last_change":"2024-05-08T23:30:38.318065+0000","last_active":"2024-05-08T23:30:40.243826+0000","last_peered":"2024-05-08T23:30:40.243826+0000","last_clean":"2024-05-08T23:30:40.243826+0000","last_became_active":"2024-05-08T23:30:38.317810+0000","last_became_peered":"2024-05-08T23:30:38.317810+0000","last_unstale":"2024-05-08T23:30:40.243826+0000","last_undegraded":"2024-05-08T23:30:40.243826+0000","last_fullsized":"2024-05-08T23:30:40.243826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:39.460999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388629+0000","last_change":"2024-05-08T23:30:38.711628+0000","last_active":"2024-05-08T23:30:39.388629+0000","last_peered":"2024-05-08T23:30:39.388629+0000","last_clean":"2024-05-08T23:30:39.388629+0000","last_became_active":"2024-05-08T23:30:38.711306+0000","last_became_peered":"2024-05-08T23:30:38.711306+0000","last_unstale":"2024-05-08T23:30:39.388629+0000","last_undegraded":"2024-05-08T23:30:39.388629+0000","last_fullsized":"2024-05-08T23:30:39.388629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:34.727934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392656+0000","last_change":"2024-05-08T23:30:38.314564+0000","last_active":"2024-05-08T23:30:40.392656+0000","last_peered":"2024-05-08T23:30:40.392656+0000","last_clean":"2024-05-08T23:30:40.392656+0000","last_became_active":"2024-05-08T23:30:38.314293+0000","last_became_peered":"2024-05-08T23:30:38.314293+0000","last_unstale":"2024-05-08T23:30:40.392656+0000","last_undegraded":"2024-05-08T23:30:40.392656+0000","last_fullsized":"2024-05-08T23:30:40.392656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392627+0000","last_change":"2024-05-08T23:30:38.694428+0000","last_active":"2024-05-08T23:30:40.392627+0000","last_peered":"2024-05-08T23:30:40.392627+0000","last_clean":"2024-05-08T23:30:40.392627+0000","last_became_active":"2024-05-08T23:30:38.693827+0000","last_became_peered":"2024-05-08T23:30:38.693827+0000","last_unstale":"2024-05-08T23:30:40.392627+0000","last_undegraded":"2024-05-08T23:30:40.392627+0000","last_fullsized":"2024-05-08T23:30:40.392627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:01.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389000+0000","last_change":"2024-05-08T23:30:38.309066+0000","last_active":"2024-05-08T23:30:40.389000+0000","last_peered":"2024-05-08T23:30:40.389000+0000","last_clean":"2024-05-08T23:30:40.389000+0000","last_became_active":"2024-05-08T23:30:38.308581+0000","last_became_peered":"2024-05-08T23:30:38.308581+0000","last_unstale":"2024-05-08T23:30:40.389000+0000","last_undegraded":"2024-05-08T23:30:40.389000+0000","last_fullsized":"2024-05-08T23:30:40.389000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388543+0000","last_change":"2024-05-08T23:30:38.704429+0000","last_active":"2024-05-08T23:30:39.388543+0000","last_peered":"2024-05-08T23:30:39.388543+0000","last_clean":"2024-05-08T23:30:39.388543+0000","last_became_active":"2024-05-08T23:30:38.703989+0000","last_became_peered":"2024-05-08T23:30:38.703989+0000","last_unstale":"2024-05-08T23:30:39.388543+0000","last_undegraded":"2024-05-08T23:30:39.388543+0000","last_fullsized":"2024-05-08T23:30:39.388543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:56.584934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.960069+0000","last_change":"2024-05-08T23:30:38.309970+0000","last_active":"2024-05-08T23:30:39.960069+0000","last_peered":"2024-05-08T23:30:39.960069+0000","last_clean":"2024-05-08T23:30:39.960069+0000","last_became_active":"2024-05-08T23:30:38.308960+0000","last_became_peered":"2024-05-08T23:30:38.308960+0000","last_unstale":"2024-05-08T23:30:39.960069+0000","last_undegraded":"2024-05-08T23:30:39.960069+0000","last_fullsized":"2024-05-08T23:30:39.960069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:41.142396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388847+0000","last_change":"2024-05-08T23:30:38.705438+0000","last_active":"2024-05-08T23:30:39.388847+0000","last_peered":"2024-05-08T23:30:39.388847+0000","last_clean":"2024-05-08T23:30:39.388847+0000","last_became_active":"2024-05-08T23:30:38.705141+0000","last_became_peered":"2024-05-08T23:30:38.705141+0000","last_unstale":"2024-05-08T23:30:39.388847+0000","last_undegraded":"2024-05-08T23:30:39.388847+0000","last_fullsized":"2024-05-08T23:30:39.388847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:37.465394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389256+0000","last_change":"2024-05-08T23:30:38.312114+0000","last_active":"2024-05-08T23:30:40.389256+0000","last_peered":"2024-05-08T23:30:40.389256+0000","last_clean":"2024-05-08T23:30:40.389256+0000","last_became_active":"2024-05-08T23:30:38.311826+0000","last_became_peered":"2024-05-08T23:30:38.311826+0000","last_unstale":"2024-05-08T23:30:40.389256+0000","last_undegraded":"2024-05-08T23:30:40.389256+0000","last_fullsized":"2024-05-08T23:30:40.389256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:12:55.004696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388919+0000","last_change":"2024-05-08T23:30:38.708759+0000","last_active":"2024-05-08T23:30:39.388919+0000","last_peered":"2024-05-08T23:30:39.388919+0000","last_clean":"2024-05-08T23:30:39.388919+0000","last_became_active":"2024-05-08T23:30:38.708506+0000","last_became_peered":"2024-05-08T23:30:38.708506+0000","last_unstale":"2024-05-08T23:30:39.388919+0000","last_undegraded":"2024-05-08T23:30:39.388919+0000","last_fullsized":"2024-05-08T23:30:39.388919+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:15.958328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389048+0000","last_change":"2024-05-08T23:30:38.314282+0000","last_active":"2024-05-08T23:30:40.389048+0000","last_peered":"2024-05-08T23:30:40.389048+0000","last_clean":"2024-05-08T23:30:40.389048+0000","last_became_active":"2024-05-08T23:30:38.314040+0000","last_became_peered":"2024-05-08T23:30:38.314040+0000","last_unstale":"2024-05-08T23:30:40.389048+0000","last_undegraded":"2024-05-08T23:30:40.389048+0000","last_fullsized":"2024-05-08T23:30:40.389048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:39.460999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243979+0000","last_change":"2024-05-08T23:30:38.688690+0000","last_active":"2024-05-08T23:30:40.243979+0000","last_peered":"2024-05-08T23:30:40.243979+0000","last_clean":"2024-05-08T23:30:40.243979+0000","last_became_active":"2024-05-08T23:30:38.688342+0000","last_became_peered":"2024-05-08T23:30:38.688342+0000","last_unstale":"2024-05-08T23:30:40.243979+0000","last_undegraded":"2024-05-08T23:30:40.243979+0000","last_fullsized":"2024-05-08T23:30:40.243979+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:13.779218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389159+0000","last_change":"2024-05-08T23:30:38.314058+0000","last_active":"2024-05-08T23:30:40.389159+0000","last_peered":"2024-05-08T23:30:40.389159+0000","last_clean":"2024-05-08T23:30:40.389159+0000","last_became_active":"2024-05-08T23:30:38.313813+0000","last_became_peered":"2024-05-08T23:30:38.313813+0000","last_unstale":"2024-05-08T23:30:40.389159+0000","last_undegraded":"2024-05-08T23:30:40.389159+0000","last_fullsized":"2024-05-08T23:30:40.389159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:14.380730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388747+0000","last_change":"2024-05-08T23:30:38.707800+0000","last_active":"2024-05-08T23:30:39.388747+0000","last_peered":"2024-05-08T23:30:39.388747+0000","last_clean":"2024-05-08T23:30:39.388747+0000","last_became_active":"2024-05-08T23:30:38.706575+0000","last_became_peered":"2024-05-08T23:30:38.706575+0000","last_unstale":"2024-05-08T23:30:39.388747+0000","last_undegraded":"2024-05-08T23:30:39.388747+0000","last_fullsized":"2024-05-08T23:30:39.388747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:16.601918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.695881+0000","last_change":"2024-05-08T23:30:27.226484+0000","last_active":"2024-05-08T23:30:38.695881+0000","last_peered":"2024-05-08T23:30:38.695881+0000","last_clean":"2024-05-08T23:30:38.695881+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:30:38.695881+0000","last_undegraded":"2024-05-08T23:30:38.695881+0000","last_fullsized":"2024-05-08T23:30:38.695881+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_clean_scrub_stamp":"2024-05-08T23:30:26.204118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:14.267287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243573+0000","last_change":"2024-05-08T23:30:38.317146+0000","last_active":"2024-05-08T23:30:40.243573+0000","last_peered":"2024-05-08T23:30:40.243573+0000","last_clean":"2024-05-08T23:30:40.243573+0000","last_became_active":"2024-05-08T23:30:38.316813+0000","last_became_peered":"2024-05-08T23:30:38.316813+0000","last_unstale":"2024-05-08T23:30:40.243573+0000","last_undegraded":"2024-05-08T23:30:40.243573+0000","last_fullsized":"2024-05-08T23:30:40.243573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:03.938472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675574+0000","last_change":"2024-05-08T23:30:38.690153+0000","last_active":"2024-05-08T23:30:39.675574+0000","last_peered":"2024-05-08T23:30:39.675574+0000","last_clean":"2024-05-08T23:30:39.675574+0000","last_became_active":"2024-05-08T23:30:38.689929+0000","last_became_peered":"2024-05-08T23:30:38.689929+0000","last_unstale":"2024-05-08T23:30:39.675574+0000","last_undegraded":"2024-05-08T23:30:39.675574+0000","last_fullsized":"2024-05-08T23:30:39.675574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:07.682170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388454+0000","last_change":"2024-05-08T23:30:38.319918+0000","last_active":"2024-05-08T23:30:39.388454+0000","last_peered":"2024-05-08T23:30:39.388454+0000","last_clean":"2024-05-08T23:30:39.388454+0000","last_became_active":"2024-05-08T23:30:38.319606+0000","last_became_peered":"2024-05-08T23:30:38.319606+0000","last_unstale":"2024-05-08T23:30:39.388454+0000","last_undegraded":"2024-05-08T23:30:39.388454+0000","last_fullsized":"2024-05-08T23:30:39.388454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:24.905425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392035+0000","last_change":"2024-05-08T23:30:38.681448+0000","last_active":"2024-05-08T23:30:39.392035+0000","last_peered":"2024-05-08T23:30:39.392035+0000","last_clean":"2024-05-08T23:30:39.392035+0000","last_became_active":"2024-05-08T23:30:38.681002+0000","last_became_peered":"2024-05-08T23:30:38.681002+0000","last_unstale":"2024-05-08T23:30:39.392035+0000","last_undegraded":"2024-05-08T23:30:39.392035+0000","last_fullsized":"2024-05-08T23:30:39.392035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:40.104243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389112+0000","last_change":"2024-05-08T23:30:38.312104+0000","last_active":"2024-05-08T23:30:40.389112+0000","last_peered":"2024-05-08T23:30:40.389112+0000","last_clean":"2024-05-08T23:30:40.389112+0000","last_became_active":"2024-05-08T23:30:38.311833+0000","last_became_peered":"2024-05-08T23:30:38.311833+0000","last_unstale":"2024-05-08T23:30:40.389112+0000","last_undegraded":"2024-05-08T23:30:40.389112+0000","last_fullsized":"2024-05-08T23:30:40.389112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:31.924157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388624+0000","last_change":"2024-05-08T23:30:38.686436+0000","last_active":"2024-05-08T23:30:40.388624+0000","last_peered":"2024-05-08T23:30:40.388624+0000","last_clean":"2024-05-08T23:30:40.388624+0000","last_became_active":"2024-05-08T23:30:38.686176+0000","last_became_peered":"2024-05-08T23:30:38.686176+0000","last_unstale":"2024-05-08T23:30:40.388624+0000","last_undegraded":"2024-05-08T23:30:40.388624+0000","last_fullsized":"2024-05-08T23:30:40.388624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:41.133153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244021+0000","last_change":"2024-05-08T23:30:38.317558+0000","last_active":"2024-05-08T23:30:40.244021+0000","last_peered":"2024-05-08T23:30:40.244021+0000","last_clean":"2024-05-08T23:30:40.244021+0000","last_became_active":"2024-05-08T23:30:38.317296+0000","last_became_peered":"2024-05-08T23:30:38.317296+0000","last_unstale":"2024-05-08T23:30:40.244021+0000","last_undegraded":"2024-05-08T23:30:40.244021+0000","last_fullsized":"2024-05-08T23:30:40.244021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959765+0000","last_change":"2024-05-08T23:30:38.682924+0000","last_active":"2024-05-08T23:30:39.959765+0000","last_peered":"2024-05-08T23:30:39.959765+0000","last_clean":"2024-05-08T23:30:39.959765+0000","last_became_active":"2024-05-08T23:30:38.682112+0000","last_became_peered":"2024-05-08T23:30:38.682112+0000","last_unstale":"2024-05-08T23:30:39.959765+0000","last_undegraded":"2024-05-08T23:30:39.959765+0000","last_fullsized":"2024-05-08T23:30:39.959765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:04.948224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388873+0000","last_change":"2024-05-08T23:30:38.317398+0000","last_active":"2024-05-08T23:30:40.388873+0000","last_peered":"2024-05-08T23:30:40.388873+0000","last_clean":"2024-05-08T23:30:40.388873+0000","last_became_active":"2024-05-08T23:30:38.317167+0000","last_became_peered":"2024-05-08T23:30:38.317167+0000","last_unstale":"2024-05-08T23:30:40.388873+0000","last_undegraded":"2024-05-08T23:30:40.388873+0000","last_fullsized":"2024-05-08T23:30:40.388873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244386+0000","last_change":"2024-05-08T23:30:38.693098+0000","last_active":"2024-05-08T23:30:40.244386+0000","last_peered":"2024-05-08T23:30:40.244386+0000","last_clean":"2024-05-08T23:30:40.244386+0000","last_became_active":"2024-05-08T23:30:38.692852+0000","last_became_peered":"2024-05-08T23:30:38.692852+0000","last_unstale":"2024-05-08T23:30:40.244386+0000","last_undegraded":"2024-05-08T23:30:40.244386+0000","last_fullsized":"2024-05-08T23:30:40.244386+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:02.678862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388824+0000","last_change":"2024-05-08T23:30:38.320883+0000","last_active":"2024-05-08T23:30:39.388824+0000","last_peered":"2024-05-08T23:30:39.388824+0000","last_clean":"2024-05-08T23:30:39.388824+0000","last_became_active":"2024-05-08T23:30:38.320610+0000","last_became_peered":"2024-05-08T23:30:38.320610+0000","last_unstale":"2024-05-08T23:30:39.388824+0000","last_undegraded":"2024-05-08T23:30:39.388824+0000","last_fullsized":"2024-05-08T23:30:39.388824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:02.623735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388835+0000","last_change":"2024-05-08T23:30:38.687478+0000","last_active":"2024-05-08T23:30:40.388835+0000","last_peered":"2024-05-08T23:30:40.388835+0000","last_clean":"2024-05-08T23:30:40.388835+0000","last_became_active":"2024-05-08T23:30:38.686229+0000","last_became_peered":"2024-05-08T23:30:38.686229+0000","last_unstale":"2024-05-08T23:30:40.388835+0000","last_undegraded":"2024-05-08T23:30:40.388835+0000","last_fullsized":"2024-05-08T23:30:40.388835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:37.994049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.460801+0000","last_change":"2024-05-08T23:30:38.318046+0000","last_active":"2024-05-08T23:30:40.460801+0000","last_peered":"2024-05-08T23:30:40.460801+0000","last_clean":"2024-05-08T23:30:40.460801+0000","last_became_active":"2024-05-08T23:30:38.317757+0000","last_became_peered":"2024-05-08T23:30:38.317757+0000","last_unstale":"2024-05-08T23:30:40.460801+0000","last_undegraded":"2024-05-08T23:30:40.460801+0000","last_fullsized":"2024-05-08T23:30:40.460801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:00.309450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676059+0000","last_change":"2024-05-08T23:30:38.706013+0000","last_active":"2024-05-08T23:30:39.676059+0000","last_peered":"2024-05-08T23:30:39.676059+0000","last_clean":"2024-05-08T23:30:39.676059+0000","last_became_active":"2024-05-08T23:30:38.705709+0000","last_became_peered":"2024-05-08T23:30:38.705709+0000","last_unstale":"2024-05-08T23:30:39.676059+0000","last_undegraded":"2024-05-08T23:30:39.676059+0000","last_fullsized":"2024-05-08T23:30:39.676059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:01.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392885+0000","last_change":"2024-05-08T23:30:38.317850+0000","last_active":"2024-05-08T23:30:40.392885+0000","last_peered":"2024-05-08T23:30:40.392885+0000","last_clean":"2024-05-08T23:30:40.392885+0000","last_became_active":"2024-05-08T23:30:38.317598+0000","last_became_peered":"2024-05-08T23:30:38.317598+0000","last_unstale":"2024-05-08T23:30:40.392885+0000","last_undegraded":"2024-05-08T23:30:40.392885+0000","last_fullsized":"2024-05-08T23:30:40.392885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:39.460999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392452+0000","last_change":"2024-05-08T23:30:38.679213+0000","last_active":"2024-05-08T23:30:39.392452+0000","last_peered":"2024-05-08T23:30:39.392452+0000","last_clean":"2024-05-08T23:30:39.392452+0000","last_became_active":"2024-05-08T23:30:38.678935+0000","last_became_peered":"2024-05-08T23:30:38.678935+0000","last_unstale":"2024-05-08T23:30:39.392452+0000","last_undegraded":"2024-05-08T23:30:39.392452+0000","last_fullsized":"2024-05-08T23:30:39.392452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:39.006348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388422+0000","last_change":"2024-05-08T23:30:38.320666+0000","last_active":"2024-05-08T23:30:39.388422+0000","last_peered":"2024-05-08T23:30:39.388422+0000","last_clean":"2024-05-08T23:30:39.388422+0000","last_became_active":"2024-05-08T23:30:38.320387+0000","last_became_peered":"2024-05-08T23:30:38.320387+0000","last_unstale":"2024-05-08T23:30:39.388422+0000","last_undegraded":"2024-05-08T23:30:39.388422+0000","last_fullsized":"2024-05-08T23:30:39.388422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:35.811032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959476+0000","last_change":"2024-05-08T23:30:38.692172+0000","last_active":"2024-05-08T23:30:39.959476+0000","last_peered":"2024-05-08T23:30:39.959476+0000","last_clean":"2024-05-08T23:30:39.959476+0000","last_became_active":"2024-05-08T23:30:38.688675+0000","last_became_peered":"2024-05-08T23:30:38.688675+0000","last_unstale":"2024-05-08T23:30:39.959476+0000","last_undegraded":"2024-05-08T23:30:39.959476+0000","last_fullsized":"2024-05-08T23:30:39.959476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:40.470751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388758+0000","last_change":"2024-05-08T23:30:38.319809+0000","last_active":"2024-05-08T23:30:40.388758+0000","last_peered":"2024-05-08T23:30:40.388758+0000","last_clean":"2024-05-08T23:30:40.388758+0000","last_became_active":"2024-05-08T23:30:38.319500+0000","last_became_peered":"2024-05-08T23:30:38.319500+0000","last_unstale":"2024-05-08T23:30:40.388758+0000","last_undegraded":"2024-05-08T23:30:40.388758+0000","last_fullsized":"2024-05-08T23:30:40.388758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:06.433390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675251+0000","last_change":"2024-05-08T23:30:38.694080+0000","last_active":"2024-05-08T23:30:39.675251+0000","last_peered":"2024-05-08T23:30:39.675251+0000","last_clean":"2024-05-08T23:30:39.675251+0000","last_became_active":"2024-05-08T23:30:38.693469+0000","last_became_peered":"2024-05-08T23:30:38.693469+0000","last_unstale":"2024-05-08T23:30:39.675251+0000","last_undegraded":"2024-05-08T23:30:39.675251+0000","last_fullsized":"2024-05-08T23:30:39.675251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:07.222797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388797+0000","last_change":"2024-05-08T23:30:38.316969+0000","last_active":"2024-05-08T23:30:40.388797+0000","last_peered":"2024-05-08T23:30:40.388797+0000","last_clean":"2024-05-08T23:30:40.388797+0000","last_became_active":"2024-05-08T23:30:38.316648+0000","last_became_peered":"2024-05-08T23:30:38.316648+0000","last_unstale":"2024-05-08T23:30:40.388797+0000","last_undegraded":"2024-05-08T23:30:40.388797+0000","last_fullsized":"2024-05-08T23:30:40.388797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:03.938472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675646+0000","last_change":"2024-05-08T23:30:38.696097+0000","last_active":"2024-05-08T23:30:39.675646+0000","last_peered":"2024-05-08T23:30:39.675646+0000","last_clean":"2024-05-08T23:30:39.675646+0000","last_became_active":"2024-05-08T23:30:38.695832+0000","last_became_peered":"2024-05-08T23:30:38.695832+0000","last_unstale":"2024-05-08T23:30:39.675646+0000","last_undegraded":"2024-05-08T23:30:39.675646+0000","last_fullsized":"2024-05-08T23:30:39.675646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:12.789913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388651+0000","last_change":"2024-05-08T23:30:38.320025+0000","last_active":"2024-05-08T23:30:39.388651+0000","last_peered":"2024-05-08T23:30:39.388651+0000","last_clean":"2024-05-08T23:30:39.388651+0000","last_became_active":"2024-05-08T23:30:38.319609+0000","last_became_peered":"2024-05-08T23:30:38.319609+0000","last_unstale":"2024-05-08T23:30:39.388651+0000","last_undegraded":"2024-05-08T23:30:39.388651+0000","last_fullsized":"2024-05-08T23:30:39.388651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:42.475179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392678+0000","last_change":"2024-05-08T23:30:38.699456+0000","last_active":"2024-05-08T23:30:40.392678+0000","last_peered":"2024-05-08T23:30:40.392678+0000","last_clean":"2024-05-08T23:30:40.392678+0000","last_became_active":"2024-05-08T23:30:38.699013+0000","last_became_peered":"2024-05-08T23:30:38.699013+0000","last_unstale":"2024-05-08T23:30:40.392678+0000","last_undegraded":"2024-05-08T23:30:40.392678+0000","last_fullsized":"2024-05-08T23:30:40.392678+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:37.994049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244260+0000","last_change":"2024-05-08T23:30:38.695946+0000","last_active":"2024-05-08T23:30:40.244260+0000","last_peered":"2024-05-08T23:30:40.244260+0000","last_clean":"2024-05-08T23:30:40.244260+0000","last_became_active":"2024-05-08T23:30:38.695704+0000","last_became_peered":"2024-05-08T23:30:38.695704+0000","last_unstale":"2024-05-08T23:30:40.244260+0000","last_undegraded":"2024-05-08T23:30:40.244260+0000","last_fullsized":"2024-05-08T23:30:40.244260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388680+0000","last_change":"2024-05-08T23:30:38.319876+0000","last_active":"2024-05-08T23:30:39.388680+0000","last_peered":"2024-05-08T23:30:39.388680+0000","last_clean":"2024-05-08T23:30:39.388680+0000","last_became_active":"2024-05-08T23:30:38.319589+0000","last_became_peered":"2024-05-08T23:30:38.319589+0000","last_unstale":"2024-05-08T23:30:39.388680+0000","last_undegraded":"2024-05-08T23:30:39.388680+0000","last_fullsized":"2024-05-08T23:30:39.388680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:12.479867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243895+0000","last_change":"2024-05-08T23:30:38.697783+0000","last_active":"2024-05-08T23:30:40.243895+0000","last_peered":"2024-05-08T23:30:40.243895+0000","last_clean":"2024-05-08T23:30:40.243895+0000","last_became_active":"2024-05-08T23:30:38.697295+0000","last_became_peered":"2024-05-08T23:30:38.697295+0000","last_unstale":"2024-05-08T23:30:40.243895+0000","last_undegraded":"2024-05-08T23:30:40.243895+0000","last_fullsized":"2024-05-08T23:30:40.243895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:12.789913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388775+0000","last_change":"2024-05-08T23:30:38.320543+0000","last_active":"2024-05-08T23:30:39.388775+0000","last_peered":"2024-05-08T23:30:39.388775+0000","last_clean":"2024-05-08T23:30:39.388775+0000","last_became_active":"2024-05-08T23:30:38.320231+0000","last_became_peered":"2024-05-08T23:30:38.320231+0000","last_unstale":"2024-05-08T23:30:39.388775+0000","last_undegraded":"2024-05-08T23:30:39.388775+0000","last_fullsized":"2024-05-08T23:30:39.388775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:08:09.987948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392499+0000","last_change":"2024-05-08T23:30:38.691260+0000","last_active":"2024-05-08T23:30:39.392499+0000","last_peered":"2024-05-08T23:30:39.392499+0000","last_clean":"2024-05-08T23:30:39.392499+0000","last_became_active":"2024-05-08T23:30:38.690856+0000","last_became_peered":"2024-05-08T23:30:38.690856+0000","last_unstale":"2024-05-08T23:30:39.392499+0000","last_undegraded":"2024-05-08T23:30:39.392499+0000","last_fullsized":"2024-05-08T23:30:39.392499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:58.926788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676224+0000","last_change":"2024-05-08T23:30:38.311855+0000","last_active":"2024-05-08T23:30:39.676224+0000","last_peered":"2024-05-08T23:30:39.676224+0000","last_clean":"2024-05-08T23:30:39.676224+0000","last_became_active":"2024-05-08T23:30:38.311542+0000","last_became_peered":"2024-05-08T23:30:38.311542+0000","last_unstale":"2024-05-08T23:30:39.676224+0000","last_undegraded":"2024-05-08T23:30:39.676224+0000","last_fullsized":"2024-05-08T23:30:39.676224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:30.035897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676005+0000","last_change":"2024-05-08T23:30:38.688998+0000","last_active":"2024-05-08T23:30:39.676005+0000","last_peered":"2024-05-08T23:30:39.676005+0000","last_clean":"2024-05-08T23:30:39.676005+0000","last_became_active":"2024-05-08T23:30:38.688723+0000","last_became_peered":"2024-05-08T23:30:38.688723+0000","last_unstale":"2024-05-08T23:30:39.676005+0000","last_undegraded":"2024-05-08T23:30:39.676005+0000","last_fullsized":"2024-05-08T23:30:39.676005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:41.133153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392591+0000","last_change":"2024-05-08T23:30:38.315584+0000","last_active":"2024-05-08T23:30:39.392591+0000","last_peered":"2024-05-08T23:30:39.392591+0000","last_clean":"2024-05-08T23:30:39.392591+0000","last_became_active":"2024-05-08T23:30:38.315234+0000","last_became_peered":"2024-05-08T23:30:38.315234+0000","last_unstale":"2024-05-08T23:30:39.392591+0000","last_undegraded":"2024-05-08T23:30:39.392591+0000","last_fullsized":"2024-05-08T23:30:39.392591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:02.623735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":78378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":76904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":76940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":78378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":76974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":819200,"data_stored":667764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":667820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:30:46.749 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:30:46.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.845+0000 7f59b4cb7700 1 Processor -- start 2024-05-08T23:30:46.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.845+0000 7f59b4cb7700 1 -- start start 2024-05-08T23:30:46.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59b4cb7700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f59a8055dc0 0x7f59a804faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59b4cb7700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f59a803a150 0x7f59a803a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59b4cb7700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a803aaa0 con 0x7f59a80df8b0 2024-05-08T23:30:46.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59b4cb7700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a8056230 con 0x7f59a803a150 2024-05-08T23:30:46.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59b4cb7700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a8006ce0 con 0x7f59a8055dc0 2024-05-08T23:30:46.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59af7fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f59a80df8b0 0x7f59a8036750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38700/0 (socket says 172.21.15.81:38700) 2024-05-08T23:30:46.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59af7fe700 1 -- 172.21.15.81:0/602240894 learned_addr learned my addr 172.21.15.81:0/602240894 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:46.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59affff700 1 --2- 172.21.15.81:0/602240894 >> v2:172.21.15.81:3300/0 conn(0x7f59a803a150 0x7f59a803a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 -- 172.21.15.81:0/602240894 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352000590 0 0) 0x7f59a803aaa0 con 0x7f59a80df8b0 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 -- 172.21.15.81:0/602240894 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59a00034f0 con 0x7f59a80df8b0 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59aeffd700 1 --2- 172.21.15.81:0/602240894 >> v2:172.21.15.117:3300/0 conn(0x7f59a8055dc0 0x7f59a804faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 -- 172.21.15.81:0/602240894 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1233772552 0 0) 0x7f59a00034f0 con 0x7f59a80df8b0 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 -- 172.21.15.81:0/602240894 >> v2:172.21.15.117:3300/0 conn(0x7f59a8055dc0 msgr2=0x7f59a804faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 --2- 172.21.15.81:0/602240894 >> v2:172.21.15.117:3300/0 conn(0x7f59a8055dc0 0x7f59a804faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 -- 172.21.15.81:0/602240894 >> v2:172.21.15.81:3300/0 conn(0x7f59a803a150 msgr2=0x7f59a803a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 --2- 172.21.15.81:0/602240894 >> v2:172.21.15.81:3300/0 conn(0x7f59a803a150 0x7f59a803a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 -- 172.21.15.81:0/602240894 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a8006fc0 con 0x7f59a80df8b0 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 -- 172.21.15.81:0/602240894 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f59a4001670 con 0x7f59a80df8b0 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 -- 172.21.15.81:0/602240894 >> v1:172.21.15.81:6789/0 conn(0x7f59a80df8b0 legacy=0x7f59a8036750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 --2- 172.21.15.81:0/602240894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a0003b10 0x7f59a0003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 --2- 172.21.15.81:0/602240894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a0008950 0x7f59a000ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 --2- 172.21.15.81:0/602240894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59a000b500 0x7f59a000d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 -- 172.21.15.81:0/602240894 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a8056230 con 0x7f59a000b500 2024-05-08T23:30:46.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 -- 172.21.15.81:0/602240894 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a8006ce0 con 0x7f59a0008950 2024-05-08T23:30:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 -- 172.21.15.81:0/602240894 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a000e310 con 0x7f59a0003b10 2024-05-08T23:30:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59affff700 1 --2- 172.21.15.81:0/602240894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59a000b500 0x7f59a000d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59af7fe700 1 --2- 172.21.15.81:0/602240894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a0003b10 0x7f59a0003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59aeffd700 1 --2- 172.21.15.81:0/602240894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a0008950 0x7f59a000ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59affff700 1 -- 172.21.15.81:0/602240894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a0003b10 msgr2=0x7f59a0003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59affff700 1 --2- 172.21.15.81:0/602240894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a0003b10 0x7f59a0003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59affff700 1 -- 172.21.15.81:0/602240894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a0008950 msgr2=0x7f59a000ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59affff700 1 --2- 172.21.15.81:0/602240894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a0008950 0x7f59a000ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59affff700 1 -- 172.21.15.81:0/602240894 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a00034f0 con 0x7f59a000b500 2024-05-08T23:30:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59affff700 1 -- 172.21.15.81:0/602240894 reap_dead start 2024-05-08T23:30:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59affff700 1 --2- 172.21.15.81:0/602240894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59a000b500 0x7f59a000d9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f59b004ff70 tx=0x7f59a00037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6284c38e6aa4d94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 -- 172.21.15.81:0/602240894 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0067bb0 con 0x7f59a000b500 2024-05-08T23:30:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 -- 172.21.15.81:0/602240894 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b0067d30 con 0x7f59a000b500 2024-05-08T23:30:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.849+0000 7f59ae7fc700 1 -- 172.21.15.81:0/602240894 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0072380 con 0x7f59a000b500 2024-05-08T23:30:46.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.853+0000 7f59b4cb7700 1 -- 172.21.15.81:0/602240894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59a000b500 msgr2=0x7f59a000d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.853+0000 7f59b4cb7700 1 --2- 172.21.15.81:0/602240894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59a000b500 0x7f59a000d9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f59b004ff70 tx=0x7f59a00037a0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.853+0000 7f59b4cb7700 1 -- 172.21.15.81:0/602240894 shutdown_connections 2024-05-08T23:30:46.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.853+0000 7f59b4cb7700 1 --2- 172.21.15.81:0/602240894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59a000b500 0x7f59a000d9c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.853+0000 7f59b4cb7700 1 -- 172.21.15.81:0/602240894 >> 172.21.15.81:0/602240894 conn(0x7f59a801e770 msgr2=0x7f59a80554a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:46.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.853+0000 7f59b4cb7700 1 -- 172.21.15.81:0/602240894 shutdown_connections 2024-05-08T23:30:46.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.853+0000 7f59b4cb7700 1 -- 172.21.15.81:0/602240894 wait complete. 2024-05-08T23:30:46.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.853+0000 7f59b4cb7700 1 Processor -- start 2024-05-08T23:30:46.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.853+0000 7f59b4cb7700 1 -- start start 2024-05-08T23:30:46.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59b4cb7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59a000b500 0x7f59a80d82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59b4cb7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a804ac20 0x7f59a804edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59b4cb7700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a8048fd0 0x7f59a800b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59b4cb7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a804a5c0 con 0x7f59a000b500 2024-05-08T23:30:46.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59b4cb7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a804b120 con 0x7f59a804ac20 2024-05-08T23:30:46.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59b4cb7700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a804b400 con 0x7f59a8048fd0 2024-05-08T23:30:46.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59af7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59a000b500 0x7f59a80d82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59af7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59a000b500 0x7f59a80d82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51186/0 (socket says 172.21.15.81:51186) 2024-05-08T23:30:46.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59af7fe700 1 -- 172.21.15.81:0/2038921813 learned_addr learned my addr 172.21.15.81:0/2038921813 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:46.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59aeffd700 1 --2- 172.21.15.81:0/2038921813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a804ac20 0x7f59a804edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59affff700 1 --2- 172.21.15.81:0/2038921813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a8048fd0 0x7f59a800b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59af7fe700 1 -- 172.21.15.81:0/2038921813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a8048fd0 msgr2=0x7f59a800b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59af7fe700 1 --2- 172.21.15.81:0/2038921813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a8048fd0 0x7f59a800b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59af7fe700 1 -- 172.21.15.81:0/2038921813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a804ac20 msgr2=0x7f59a804edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:46.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59af7fe700 1 --2- 172.21.15.81:0/2038921813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a804ac20 0x7f59a804edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:46.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59af7fe700 1 -- 172.21.15.81:0/2038921813 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a804b6e0 con 0x7f59a000b500 2024-05-08T23:30:46.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59af7fe700 1 --2- 172.21.15.81:0/2038921813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59a000b500 0x7f59a80d82f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f59a4007340 tx=0x7f59a4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f599bfff700 1 -- 172.21.15.81:0/2038921813 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4007e60 con 0x7f59a000b500 2024-05-08T23:30:46.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f599bfff700 1 -- 172.21.15.81:0/2038921813 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a400f8d0 con 0x7f59a000b500 2024-05-08T23:30:46.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59b4cb7700 1 -- 172.21.15.81:0/2038921813 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a804d640 con 0x7f59a000b500 2024-05-08T23:30:46.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f599bfff700 1 -- 172.21.15.81:0/2038921813 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a400fc60 con 0x7f59a000b500 2024-05-08T23:30:46.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59b4cb7700 1 -- 172.21.15.81:0/2038921813 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a804d260 con 0x7f59a000b500 2024-05-08T23:30:46.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.857+0000 7f59b4cb7700 1 -- 172.21.15.81:0/2038921813 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a8005890 con 0x7f59a000b500 2024-05-08T23:30:46.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.861+0000 7f599bfff700 1 -- 172.21.15.81:0/2038921813 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59a401d460 con 0x7f59a000b500 2024-05-08T23:30:46.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.861+0000 7f599bfff700 1 --2- 172.21.15.81:0/2038921813 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5994066730 0x7f5994068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:46.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.861+0000 7f599bfff700 1 -- 172.21.15.81:0/2038921813 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59a40842e0 con 0x7f59a000b500 2024-05-08T23:30:46.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.861+0000 7f59aeffd700 1 --2- 172.21.15.81:0/2038921813 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5994066730 0x7f5994068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:46.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.861+0000 7f59aeffd700 1 --2- 172.21.15.81:0/2038921813 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5994066730 0x7f5994068bf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f59a804ce60 tx=0x7f599c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:46.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:46.865+0000 7f599bfff700 1 -- 172.21.15.81:0/2038921813 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f59a4051540 con 0x7f59a000b500 2024-05-08T23:30:47.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.021+0000 7f59b4cb7700 1 -- 172.21.15.81:0/2038921813 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f59a80dfc90 con 0x7f5994066730 2024-05-08T23:30:47.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.025+0000 7f599bfff700 1 -- 172.21.15.81:0/2038921813 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264587 (secure 0 0 0) 0x7f599c009de0 con 0x7f5994066730 2024-05-08T23:30:47.035 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:47.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.029+0000 7f59b4cb7700 1 -- 172.21.15.81:0/2038921813 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5994066730 msgr2=0x7f5994068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.029+0000 7f59b4cb7700 1 --2- 172.21.15.81:0/2038921813 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5994066730 0x7f5994068bf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f59a804ce60 tx=0x7f599c008040 comp rx=0 tx=0).stop 2024-05-08T23:30:47.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.029+0000 7f59b4cb7700 1 -- 172.21.15.81:0/2038921813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59a000b500 msgr2=0x7f59a80d82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.029+0000 7f59b4cb7700 1 --2- 172.21.15.81:0/2038921813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59a000b500 0x7f59a80d82f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f59a4007340 tx=0x7f59a4007700 comp rx=0 tx=0).stop 2024-05-08T23:30:47.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.029+0000 7f59b4cb7700 1 -- 172.21.15.81:0/2038921813 shutdown_connections 2024-05-08T23:30:47.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.029+0000 7f59b4cb7700 1 --2- 172.21.15.81:0/2038921813 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5994066730 0x7f5994068bf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.029+0000 7f59b4cb7700 1 --2- 172.21.15.81:0/2038921813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a8048fd0 0x7f59a800b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.029+0000 7f59b4cb7700 1 --2- 172.21.15.81:0/2038921813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a804ac20 0x7f59a804edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.029+0000 7f59b4cb7700 1 --2- 172.21.15.81:0/2038921813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59a000b500 0x7f59a80d82f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.029+0000 7f59b4cb7700 1 -- 172.21.15.81:0/2038921813 >> 172.21.15.81:0/2038921813 conn(0x7f59a801e770 msgr2=0x7f59a80379c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:47.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.029+0000 7f59b4cb7700 1 -- 172.21.15.81:0/2038921813 shutdown_connections 2024-05-08T23:30:47.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.029+0000 7f59b4cb7700 1 -- 172.21.15.81:0/2038921813 wait complete. 2024-05-08T23:30:47.038 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:30:47.047 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-05-08T23:30:45.326555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219152,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755568,"statfs":{"total":773094113280,"available":772869701632,"internally_reserved":0,"allocated":3022848,"data_stored":1799972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.032511"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388868+0000","last_change":"2024-05-08T23:30:38.707899+0000","last_active":"2024-05-08T23:30:39.388868+0000","last_peered":"2024-05-08T23:30:39.388868+0000","last_clean":"2024-05-08T23:30:39.388868+0000","last_became_active":"2024-05-08T23:30:38.706532+0000","last_became_peered":"2024-05-08T23:30:38.706532+0000","last_unstale":"2024-05-08T23:30:39.388868+0000","last_undegraded":"2024-05-08T23:30:39.388868+0000","last_fullsized":"2024-05-08T23:30:39.388868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:21.109032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389098+0000","last_change":"2024-05-08T23:30:38.684226+0000","last_active":"2024-05-08T23:30:40.389098+0000","last_peered":"2024-05-08T23:30:40.389098+0000","last_clean":"2024-05-08T23:30:40.389098+0000","last_became_active":"2024-05-08T23:30:38.683958+0000","last_became_peered":"2024-05-08T23:30:38.683958+0000","last_unstale":"2024-05-08T23:30:40.389098+0000","last_undegraded":"2024-05-08T23:30:40.389098+0000","last_fullsized":"2024-05-08T23:30:40.389098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:07.222797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392385+0000","last_change":"2024-05-08T23:30:38.686289+0000","last_active":"2024-05-08T23:30:39.392385+0000","last_peered":"2024-05-08T23:30:39.392385+0000","last_clean":"2024-05-08T23:30:39.392385+0000","last_became_active":"2024-05-08T23:30:38.685661+0000","last_became_peered":"2024-05-08T23:30:38.685661+0000","last_unstale":"2024-05-08T23:30:39.392385+0000","last_undegraded":"2024-05-08T23:30:39.392385+0000","last_fullsized":"2024-05-08T23:30:39.392385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:34.727934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243711+0000","last_change":"2024-05-08T23:30:38.694133+0000","last_active":"2024-05-08T23:30:40.243711+0000","last_peered":"2024-05-08T23:30:40.243711+0000","last_clean":"2024-05-08T23:30:40.243711+0000","last_became_active":"2024-05-08T23:30:38.693846+0000","last_became_peered":"2024-05-08T23:30:38.693846+0000","last_unstale":"2024-05-08T23:30:40.243711+0000","last_undegraded":"2024-05-08T23:30:40.243711+0000","last_fullsized":"2024-05-08T23:30:40.243711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:20:44.481684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388606+0000","last_change":"2024-05-08T23:30:38.705630+0000","last_active":"2024-05-08T23:30:39.388606+0000","last_peered":"2024-05-08T23:30:39.388606+0000","last_clean":"2024-05-08T23:30:39.388606+0000","last_became_active":"2024-05-08T23:30:38.705310+0000","last_became_peered":"2024-05-08T23:30:38.705310+0000","last_unstale":"2024-05-08T23:30:39.388606+0000","last_undegraded":"2024-05-08T23:30:39.388606+0000","last_fullsized":"2024-05-08T23:30:39.388606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:41.949561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243496+0000","last_change":"2024-05-08T23:30:38.691423+0000","last_active":"2024-05-08T23:30:40.243496+0000","last_peered":"2024-05-08T23:30:40.243496+0000","last_clean":"2024-05-08T23:30:40.243496+0000","last_became_active":"2024-05-08T23:30:38.690973+0000","last_became_peered":"2024-05-08T23:30:38.690973+0000","last_unstale":"2024-05-08T23:30:40.243496+0000","last_undegraded":"2024-05-08T23:30:40.243496+0000","last_fullsized":"2024-05-08T23:30:40.243496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:42.503600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388516+0000","last_change":"2024-05-08T23:30:38.702702+0000","last_active":"2024-05-08T23:30:39.388516+0000","last_peered":"2024-05-08T23:30:39.388516+0000","last_clean":"2024-05-08T23:30:39.388516+0000","last_became_active":"2024-05-08T23:30:38.702341+0000","last_became_peered":"2024-05-08T23:30:38.702341+0000","last_unstale":"2024-05-08T23:30:39.388516+0000","last_undegraded":"2024-05-08T23:30:39.388516+0000","last_fullsized":"2024-05-08T23:30:39.388516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:05.546774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675055+0000","last_change":"2024-05-08T23:30:38.692708+0000","last_active":"2024-05-08T23:30:39.675055+0000","last_peered":"2024-05-08T23:30:39.675055+0000","last_clean":"2024-05-08T23:30:39.675055+0000","last_became_active":"2024-05-08T23:30:38.692424+0000","last_became_peered":"2024-05-08T23:30:38.692424+0000","last_unstale":"2024-05-08T23:30:39.675055+0000","last_undegraded":"2024-05-08T23:30:39.675055+0000","last_fullsized":"2024-05-08T23:30:39.675055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:58.747769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388895+0000","last_change":"2024-05-08T23:30:38.708610+0000","last_active":"2024-05-08T23:30:39.388895+0000","last_peered":"2024-05-08T23:30:39.388895+0000","last_clean":"2024-05-08T23:30:39.388895+0000","last_became_active":"2024-05-08T23:30:38.708389+0000","last_became_peered":"2024-05-08T23:30:38.708389+0000","last_unstale":"2024-05-08T23:30:39.388895+0000","last_undegraded":"2024-05-08T23:30:39.388895+0000","last_fullsized":"2024-05-08T23:30:39.388895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:41.257114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392823+0000","last_change":"2024-05-08T23:30:38.690662+0000","last_active":"2024-05-08T23:30:40.392823+0000","last_peered":"2024-05-08T23:30:40.392823+0000","last_clean":"2024-05-08T23:30:40.392823+0000","last_became_active":"2024-05-08T23:30:38.690378+0000","last_became_peered":"2024-05-08T23:30:38.690378+0000","last_unstale":"2024-05-08T23:30:40.392823+0000","last_undegraded":"2024-05-08T23:30:40.392823+0000","last_fullsized":"2024-05-08T23:30:40.392823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:33.598895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392801+0000","last_change":"2024-05-08T23:30:38.692171+0000","last_active":"2024-05-08T23:30:40.392801+0000","last_peered":"2024-05-08T23:30:40.392801+0000","last_clean":"2024-05-08T23:30:40.392801+0000","last_became_active":"2024-05-08T23:30:38.691611+0000","last_became_peered":"2024-05-08T23:30:38.691611+0000","last_unstale":"2024-05-08T23:30:40.392801+0000","last_undegraded":"2024-05-08T23:30:40.392801+0000","last_fullsized":"2024-05-08T23:30:40.392801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959822+0000","last_change":"2024-05-08T23:30:38.694336+0000","last_active":"2024-05-08T23:30:39.959822+0000","last_peered":"2024-05-08T23:30:39.959822+0000","last_clean":"2024-05-08T23:30:39.959822+0000","last_became_active":"2024-05-08T23:30:38.694038+0000","last_became_peered":"2024-05-08T23:30:38.694038+0000","last_unstale":"2024-05-08T23:30:39.959822+0000","last_undegraded":"2024-05-08T23:30:39.959822+0000","last_fullsized":"2024-05-08T23:30:39.959822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:07.443142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676123+0000","last_change":"2024-05-08T23:30:38.691325+0000","last_active":"2024-05-08T23:30:39.676123+0000","last_peered":"2024-05-08T23:30:39.676123+0000","last_clean":"2024-05-08T23:30:39.676123+0000","last_became_active":"2024-05-08T23:30:38.691074+0000","last_became_peered":"2024-05-08T23:30:38.691074+0000","last_unstale":"2024-05-08T23:30:39.676123+0000","last_undegraded":"2024-05-08T23:30:39.676123+0000","last_fullsized":"2024-05-08T23:30:39.676123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:37.994049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392094+0000","last_change":"2024-05-08T23:30:38.679460+0000","last_active":"2024-05-08T23:30:39.392094+0000","last_peered":"2024-05-08T23:30:39.392094+0000","last_clean":"2024-05-08T23:30:39.392094+0000","last_became_active":"2024-05-08T23:30:38.678871+0000","last_became_peered":"2024-05-08T23:30:38.678871+0000","last_unstale":"2024-05-08T23:30:39.392094+0000","last_undegraded":"2024-05-08T23:30:39.392094+0000","last_fullsized":"2024-05-08T23:30:39.392094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:37.465394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388721+0000","last_change":"2024-05-08T23:30:38.692896+0000","last_active":"2024-05-08T23:30:40.388721+0000","last_peered":"2024-05-08T23:30:40.388721+0000","last_clean":"2024-05-08T23:30:40.388721+0000","last_became_active":"2024-05-08T23:30:38.692347+0000","last_became_peered":"2024-05-08T23:30:38.692347+0000","last_unstale":"2024-05-08T23:30:40.388721+0000","last_undegraded":"2024-05-08T23:30:40.388721+0000","last_fullsized":"2024-05-08T23:30:40.388721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:01.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243296+0000","last_change":"2024-05-08T23:30:38.689233+0000","last_active":"2024-05-08T23:30:40.243296+0000","last_peered":"2024-05-08T23:30:40.243296+0000","last_clean":"2024-05-08T23:30:40.243296+0000","last_became_active":"2024-05-08T23:30:38.688984+0000","last_became_peered":"2024-05-08T23:30:38.688984+0000","last_unstale":"2024-05-08T23:30:40.243296+0000","last_undegraded":"2024-05-08T23:30:40.243296+0000","last_fullsized":"2024-05-08T23:30:40.243296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:07.682170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244541+0000","last_change":"2024-05-08T23:30:38.706139+0000","last_active":"2024-05-08T23:30:40.244541+0000","last_peered":"2024-05-08T23:30:40.244541+0000","last_clean":"2024-05-08T23:30:40.244541+0000","last_became_active":"2024-05-08T23:30:38.705878+0000","last_became_peered":"2024-05-08T23:30:38.705878+0000","last_unstale":"2024-05-08T23:30:40.244541+0000","last_undegraded":"2024-05-08T23:30:40.244541+0000","last_fullsized":"2024-05-08T23:30:40.244541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:39.788098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389205+0000","last_change":"2024-05-08T23:30:38.685704+0000","last_active":"2024-05-08T23:30:40.389205+0000","last_peered":"2024-05-08T23:30:40.389205+0000","last_clean":"2024-05-08T23:30:40.389205+0000","last_became_active":"2024-05-08T23:30:38.685435+0000","last_became_peered":"2024-05-08T23:30:38.685435+0000","last_unstale":"2024-05-08T23:30:40.389205+0000","last_undegraded":"2024-05-08T23:30:40.389205+0000","last_fullsized":"2024-05-08T23:30:40.389205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:07.443142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388798+0000","last_change":"2024-05-08T23:30:38.710968+0000","last_active":"2024-05-08T23:30:39.388798+0000","last_peered":"2024-05-08T23:30:39.388798+0000","last_clean":"2024-05-08T23:30:39.388798+0000","last_became_active":"2024-05-08T23:30:38.710706+0000","last_became_peered":"2024-05-08T23:30:38.710706+0000","last_unstale":"2024-05-08T23:30:39.388798+0000","last_undegraded":"2024-05-08T23:30:39.388798+0000","last_fullsized":"2024-05-08T23:30:39.388798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:32.727647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392743+0000","last_change":"2024-05-08T23:30:38.701983+0000","last_active":"2024-05-08T23:30:40.392743+0000","last_peered":"2024-05-08T23:30:40.392743+0000","last_clean":"2024-05-08T23:30:40.392743+0000","last_became_active":"2024-05-08T23:30:38.701386+0000","last_became_peered":"2024-05-08T23:30:38.701386+0000","last_unstale":"2024-05-08T23:30:40.392743+0000","last_undegraded":"2024-05-08T23:30:40.392743+0000","last_fullsized":"2024-05-08T23:30:40.392743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:12.789913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392189+0000","last_change":"2024-05-08T23:30:38.705814+0000","last_active":"2024-05-08T23:30:39.392189+0000","last_peered":"2024-05-08T23:30:39.392189+0000","last_clean":"2024-05-08T23:30:39.392189+0000","last_became_active":"2024-05-08T23:30:38.705393+0000","last_became_peered":"2024-05-08T23:30:38.705393+0000","last_unstale":"2024-05-08T23:30:39.392189+0000","last_undegraded":"2024-05-08T23:30:39.392189+0000","last_fullsized":"2024-05-08T23:30:39.392189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:35.129372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243643+0000","last_change":"2024-05-08T23:30:38.703498+0000","last_active":"2024-05-08T23:30:40.243643+0000","last_peered":"2024-05-08T23:30:40.243643+0000","last_clean":"2024-05-08T23:30:40.243643+0000","last_became_active":"2024-05-08T23:30:38.703206+0000","last_became_peered":"2024-05-08T23:30:38.703206+0000","last_unstale":"2024-05-08T23:30:40.243643+0000","last_undegraded":"2024-05-08T23:30:40.243643+0000","last_fullsized":"2024-05-08T23:30:40.243643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:58.747769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675186+0000","last_change":"2024-05-08T23:30:38.697689+0000","last_active":"2024-05-08T23:30:39.675186+0000","last_peered":"2024-05-08T23:30:39.675186+0000","last_clean":"2024-05-08T23:30:39.675186+0000","last_became_active":"2024-05-08T23:30:38.697373+0000","last_became_peered":"2024-05-08T23:30:38.697373+0000","last_unstale":"2024-05-08T23:30:39.675186+0000","last_undegraded":"2024-05-08T23:30:39.675186+0000","last_fullsized":"2024-05-08T23:30:39.675186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:42.503600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243343+0000","last_change":"2024-05-08T23:30:38.693170+0000","last_active":"2024-05-08T23:30:40.243343+0000","last_peered":"2024-05-08T23:30:40.243343+0000","last_clean":"2024-05-08T23:30:40.243343+0000","last_became_active":"2024-05-08T23:30:38.692809+0000","last_became_peered":"2024-05-08T23:30:38.692809+0000","last_unstale":"2024-05-08T23:30:40.243343+0000","last_undegraded":"2024-05-08T23:30:40.243343+0000","last_fullsized":"2024-05-08T23:30:40.243343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:51.717280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675706+0000","last_change":"2024-05-08T23:30:38.696080+0000","last_active":"2024-05-08T23:30:39.675706+0000","last_peered":"2024-05-08T23:30:39.675706+0000","last_clean":"2024-05-08T23:30:39.675706+0000","last_became_active":"2024-05-08T23:30:38.695821+0000","last_became_peered":"2024-05-08T23:30:38.695821+0000","last_unstale":"2024-05-08T23:30:39.675706+0000","last_undegraded":"2024-05-08T23:30:39.675706+0000","last_fullsized":"2024-05-08T23:30:39.675706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:33.598895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392843+0000","last_change":"2024-05-08T23:30:38.704917+0000","last_active":"2024-05-08T23:30:40.392843+0000","last_peered":"2024-05-08T23:30:40.392843+0000","last_clean":"2024-05-08T23:30:40.392843+0000","last_became_active":"2024-05-08T23:30:38.704663+0000","last_became_peered":"2024-05-08T23:30:38.704663+0000","last_unstale":"2024-05-08T23:30:40.392843+0000","last_undegraded":"2024-05-08T23:30:40.392843+0000","last_fullsized":"2024-05-08T23:30:40.392843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:41.133153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388927+0000","last_change":"2024-05-08T23:30:38.693867+0000","last_active":"2024-05-08T23:30:40.388927+0000","last_peered":"2024-05-08T23:30:40.388927+0000","last_clean":"2024-05-08T23:30:40.388927+0000","last_became_active":"2024-05-08T23:30:38.693637+0000","last_became_peered":"2024-05-08T23:30:38.693637+0000","last_unstale":"2024-05-08T23:30:40.388927+0000","last_undegraded":"2024-05-08T23:30:40.388927+0000","last_fullsized":"2024-05-08T23:30:40.388927+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:33.598895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676136+0000","last_change":"2024-05-08T23:30:38.697753+0000","last_active":"2024-05-08T23:30:39.676136+0000","last_peered":"2024-05-08T23:30:39.676136+0000","last_clean":"2024-05-08T23:30:39.676136+0000","last_became_active":"2024-05-08T23:30:38.697337+0000","last_became_peered":"2024-05-08T23:30:38.697337+0000","last_unstale":"2024-05-08T23:30:39.676136+0000","last_undegraded":"2024-05-08T23:30:39.676136+0000","last_fullsized":"2024-05-08T23:30:39.676136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392238+0000","last_change":"2024-05-08T23:30:38.688908+0000","last_active":"2024-05-08T23:30:39.392238+0000","last_peered":"2024-05-08T23:30:39.392238+0000","last_clean":"2024-05-08T23:30:39.392238+0000","last_became_active":"2024-05-08T23:30:38.688636+0000","last_became_peered":"2024-05-08T23:30:38.688636+0000","last_unstale":"2024-05-08T23:30:39.392238+0000","last_undegraded":"2024-05-08T23:30:39.392238+0000","last_fullsized":"2024-05-08T23:30:39.392238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:41.059680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243784+0000","last_change":"2024-05-08T23:30:38.688336+0000","last_active":"2024-05-08T23:30:40.243784+0000","last_peered":"2024-05-08T23:30:40.243784+0000","last_clean":"2024-05-08T23:30:40.243784+0000","last_became_active":"2024-05-08T23:30:38.688041+0000","last_became_peered":"2024-05-08T23:30:38.688041+0000","last_unstale":"2024-05-08T23:30:40.243784+0000","last_undegraded":"2024-05-08T23:30:40.243784+0000","last_fullsized":"2024-05-08T23:30:40.243784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:47:17.135855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389063+0000","last_change":"2024-05-08T23:30:38.692840+0000","last_active":"2024-05-08T23:30:40.389063+0000","last_peered":"2024-05-08T23:30:40.389063+0000","last_clean":"2024-05-08T23:30:40.389063+0000","last_became_active":"2024-05-08T23:30:38.690283+0000","last_became_peered":"2024-05-08T23:30:38.690283+0000","last_unstale":"2024-05-08T23:30:40.389063+0000","last_undegraded":"2024-05-08T23:30:40.389063+0000","last_fullsized":"2024-05-08T23:30:40.389063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:11.511515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392596+0000","last_change":"2024-05-08T23:30:38.693989+0000","last_active":"2024-05-08T23:30:40.392596+0000","last_peered":"2024-05-08T23:30:40.392596+0000","last_clean":"2024-05-08T23:30:40.392596+0000","last_became_active":"2024-05-08T23:30:38.693491+0000","last_became_peered":"2024-05-08T23:30:38.693491+0000","last_unstale":"2024-05-08T23:30:40.392596+0000","last_undegraded":"2024-05-08T23:30:40.392596+0000","last_fullsized":"2024-05-08T23:30:40.392596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:07.222797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959371+0000","last_change":"2024-05-08T23:30:38.681154+0000","last_active":"2024-05-08T23:30:39.959371+0000","last_peered":"2024-05-08T23:30:39.959371+0000","last_clean":"2024-05-08T23:30:39.959371+0000","last_became_active":"2024-05-08T23:30:38.680826+0000","last_became_peered":"2024-05-08T23:30:38.680826+0000","last_unstale":"2024-05-08T23:30:39.959371+0000","last_undegraded":"2024-05-08T23:30:39.959371+0000","last_fullsized":"2024-05-08T23:30:39.959371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:11.511515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675917+0000","last_change":"2024-05-08T23:30:38.317473+0000","last_active":"2024-05-08T23:30:39.675917+0000","last_peered":"2024-05-08T23:30:39.675917+0000","last_clean":"2024-05-08T23:30:39.675917+0000","last_became_active":"2024-05-08T23:30:38.317235+0000","last_became_peered":"2024-05-08T23:30:38.317235+0000","last_unstale":"2024-05-08T23:30:39.675917+0000","last_undegraded":"2024-05-08T23:30:39.675917+0000","last_fullsized":"2024-05-08T23:30:39.675917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:31.924157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392285+0000","last_change":"2024-05-08T23:30:38.680780+0000","last_active":"2024-05-08T23:30:39.392285+0000","last_peered":"2024-05-08T23:30:39.392285+0000","last_clean":"2024-05-08T23:30:39.392285+0000","last_became_active":"2024-05-08T23:30:38.680508+0000","last_became_peered":"2024-05-08T23:30:38.680508+0000","last_unstale":"2024-05-08T23:30:39.392285+0000","last_undegraded":"2024-05-08T23:30:39.392285+0000","last_fullsized":"2024-05-08T23:30:39.392285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:24.058714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392711+0000","last_change":"2024-05-08T23:30:38.309456+0000","last_active":"2024-05-08T23:30:40.392711+0000","last_peered":"2024-05-08T23:30:40.392711+0000","last_clean":"2024-05-08T23:30:40.392711+0000","last_became_active":"2024-05-08T23:30:38.308975+0000","last_became_peered":"2024-05-08T23:30:38.308975+0000","last_unstale":"2024-05-08T23:30:40.392711+0000","last_undegraded":"2024-05-08T23:30:40.392711+0000","last_fullsized":"2024-05-08T23:30:40.392711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:00.309450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675599+0000","last_change":"2024-05-08T23:30:38.317811+0000","last_active":"2024-05-08T23:30:39.675599+0000","last_peered":"2024-05-08T23:30:39.675599+0000","last_clean":"2024-05-08T23:30:39.675599+0000","last_became_active":"2024-05-08T23:30:38.317505+0000","last_became_peered":"2024-05-08T23:30:38.317505+0000","last_unstale":"2024-05-08T23:30:39.675599+0000","last_undegraded":"2024-05-08T23:30:39.675599+0000","last_fullsized":"2024-05-08T23:30:39.675599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:14.380730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675152+0000","last_change":"2024-05-08T23:30:38.695850+0000","last_active":"2024-05-08T23:30:39.675152+0000","last_peered":"2024-05-08T23:30:39.675152+0000","last_clean":"2024-05-08T23:30:39.675152+0000","last_became_active":"2024-05-08T23:30:38.693296+0000","last_became_peered":"2024-05-08T23:30:38.693296+0000","last_unstale":"2024-05-08T23:30:39.675152+0000","last_undegraded":"2024-05-08T23:30:39.675152+0000","last_fullsized":"2024-05-08T23:30:39.675152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:35:33.805704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243572+0000","last_change":"2024-05-08T23:30:38.314634+0000","last_active":"2024-05-08T23:30:40.243572+0000","last_peered":"2024-05-08T23:30:40.243572+0000","last_clean":"2024-05-08T23:30:40.243572+0000","last_became_active":"2024-05-08T23:30:38.314368+0000","last_became_peered":"2024-05-08T23:30:38.314368+0000","last_unstale":"2024-05-08T23:30:40.243572+0000","last_undegraded":"2024-05-08T23:30:40.243572+0000","last_fullsized":"2024-05-08T23:30:40.243572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:12:55.004696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388682+0000","last_change":"2024-05-08T23:30:38.688337+0000","last_active":"2024-05-08T23:30:40.388682+0000","last_peered":"2024-05-08T23:30:40.388682+0000","last_clean":"2024-05-08T23:30:40.388682+0000","last_became_active":"2024-05-08T23:30:38.687594+0000","last_became_peered":"2024-05-08T23:30:38.687594+0000","last_unstale":"2024-05-08T23:30:40.388682+0000","last_undegraded":"2024-05-08T23:30:40.388682+0000","last_fullsized":"2024-05-08T23:30:40.388682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392864+0000","last_change":"2024-05-08T23:30:38.315204+0000","last_active":"2024-05-08T23:30:40.392864+0000","last_peered":"2024-05-08T23:30:40.392864+0000","last_clean":"2024-05-08T23:30:40.392864+0000","last_became_active":"2024-05-08T23:30:38.314885+0000","last_became_peered":"2024-05-08T23:30:38.314885+0000","last_unstale":"2024-05-08T23:30:40.392864+0000","last_undegraded":"2024-05-08T23:30:40.392864+0000","last_fullsized":"2024-05-08T23:30:40.392864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:03.938472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244273+0000","last_change":"2024-05-08T23:30:38.703534+0000","last_active":"2024-05-08T23:30:40.244273+0000","last_peered":"2024-05-08T23:30:40.244273+0000","last_clean":"2024-05-08T23:30:40.244273+0000","last_became_active":"2024-05-08T23:30:38.703221+0000","last_became_peered":"2024-05-08T23:30:38.703221+0000","last_unstale":"2024-05-08T23:30:40.244273+0000","last_undegraded":"2024-05-08T23:30:40.244273+0000","last_fullsized":"2024-05-08T23:30:40.244273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:58.807960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244382+0000","last_change":"2024-05-08T23:30:38.318632+0000","last_active":"2024-05-08T23:30:40.244382+0000","last_peered":"2024-05-08T23:30:40.244382+0000","last_clean":"2024-05-08T23:30:40.244382+0000","last_became_active":"2024-05-08T23:30:38.318283+0000","last_became_peered":"2024-05-08T23:30:38.318283+0000","last_unstale":"2024-05-08T23:30:40.244382+0000","last_undegraded":"2024-05-08T23:30:40.244382+0000","last_fullsized":"2024-05-08T23:30:40.244382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:00.309450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392543+0000","last_change":"2024-05-08T23:30:38.691103+0000","last_active":"2024-05-08T23:30:39.392543+0000","last_peered":"2024-05-08T23:30:39.392543+0000","last_clean":"2024-05-08T23:30:39.392543+0000","last_became_active":"2024-05-08T23:30:38.690525+0000","last_became_peered":"2024-05-08T23:30:38.690525+0000","last_unstale":"2024-05-08T23:30:39.392543+0000","last_undegraded":"2024-05-08T23:30:39.392543+0000","last_fullsized":"2024-05-08T23:30:39.392543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:21.833335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388976+0000","last_change":"2024-05-08T23:30:38.317224+0000","last_active":"2024-05-08T23:30:40.388976+0000","last_peered":"2024-05-08T23:30:40.388976+0000","last_clean":"2024-05-08T23:30:40.388976+0000","last_became_active":"2024-05-08T23:30:38.316980+0000","last_became_peered":"2024-05-08T23:30:38.316980+0000","last_unstale":"2024-05-08T23:30:40.388976+0000","last_undegraded":"2024-05-08T23:30:40.388976+0000","last_fullsized":"2024-05-08T23:30:40.388976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:10.501763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959964+0000","last_change":"2024-05-08T23:30:38.703383+0000","last_active":"2024-05-08T23:30:39.959964+0000","last_peered":"2024-05-08T23:30:39.959964+0000","last_clean":"2024-05-08T23:30:39.959964+0000","last_became_active":"2024-05-08T23:30:38.703119+0000","last_became_peered":"2024-05-08T23:30:38.703119+0000","last_unstale":"2024-05-08T23:30:39.959964+0000","last_undegraded":"2024-05-08T23:30:39.959964+0000","last_fullsized":"2024-05-08T23:30:39.959964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:01.319202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392765+0000","last_change":"2024-05-08T23:30:38.315586+0000","last_active":"2024-05-08T23:30:40.392765+0000","last_peered":"2024-05-08T23:30:40.392765+0000","last_clean":"2024-05-08T23:30:40.392765+0000","last_became_active":"2024-05-08T23:30:38.315274+0000","last_became_peered":"2024-05-08T23:30:38.315274+0000","last_unstale":"2024-05-08T23:30:40.392765+0000","last_undegraded":"2024-05-08T23:30:40.392765+0000","last_fullsized":"2024-05-08T23:30:40.392765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:31.924157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388710+0000","last_change":"2024-05-08T23:30:38.705407+0000","last_active":"2024-05-08T23:30:39.388710+0000","last_peered":"2024-05-08T23:30:39.388710+0000","last_clean":"2024-05-08T23:30:39.388710+0000","last_became_active":"2024-05-08T23:30:38.705035+0000","last_became_peered":"2024-05-08T23:30:38.705035+0000","last_unstale":"2024-05-08T23:30:39.388710+0000","last_undegraded":"2024-05-08T23:30:39.388710+0000","last_fullsized":"2024-05-08T23:30:39.388710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:49.850123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392334+0000","last_change":"2024-05-08T23:30:38.308541+0000","last_active":"2024-05-08T23:30:39.392334+0000","last_peered":"2024-05-08T23:30:39.392334+0000","last_clean":"2024-05-08T23:30:39.392334+0000","last_became_active":"2024-05-08T23:30:38.308285+0000","last_became_peered":"2024-05-08T23:30:38.308285+0000","last_unstale":"2024-05-08T23:30:39.392334+0000","last_undegraded":"2024-05-08T23:30:39.392334+0000","last_fullsized":"2024-05-08T23:30:39.392334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:12.981854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392148+0000","last_change":"2024-05-08T23:30:38.702752+0000","last_active":"2024-05-08T23:30:39.392148+0000","last_peered":"2024-05-08T23:30:39.392148+0000","last_clean":"2024-05-08T23:30:39.392148+0000","last_became_active":"2024-05-08T23:30:38.702534+0000","last_became_peered":"2024-05-08T23:30:38.702534+0000","last_unstale":"2024-05-08T23:30:39.392148+0000","last_undegraded":"2024-05-08T23:30:39.392148+0000","last_fullsized":"2024-05-08T23:30:39.392148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:40.230149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243826+0000","last_change":"2024-05-08T23:30:38.318065+0000","last_active":"2024-05-08T23:30:40.243826+0000","last_peered":"2024-05-08T23:30:40.243826+0000","last_clean":"2024-05-08T23:30:40.243826+0000","last_became_active":"2024-05-08T23:30:38.317810+0000","last_became_peered":"2024-05-08T23:30:38.317810+0000","last_unstale":"2024-05-08T23:30:40.243826+0000","last_undegraded":"2024-05-08T23:30:40.243826+0000","last_fullsized":"2024-05-08T23:30:40.243826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:39.460999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388629+0000","last_change":"2024-05-08T23:30:38.711628+0000","last_active":"2024-05-08T23:30:39.388629+0000","last_peered":"2024-05-08T23:30:39.388629+0000","last_clean":"2024-05-08T23:30:39.388629+0000","last_became_active":"2024-05-08T23:30:38.711306+0000","last_became_peered":"2024-05-08T23:30:38.711306+0000","last_unstale":"2024-05-08T23:30:39.388629+0000","last_undegraded":"2024-05-08T23:30:39.388629+0000","last_fullsized":"2024-05-08T23:30:39.388629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:34.727934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392656+0000","last_change":"2024-05-08T23:30:38.314564+0000","last_active":"2024-05-08T23:30:40.392656+0000","last_peered":"2024-05-08T23:30:40.392656+0000","last_clean":"2024-05-08T23:30:40.392656+0000","last_became_active":"2024-05-08T23:30:38.314293+0000","last_became_peered":"2024-05-08T23:30:38.314293+0000","last_unstale":"2024-05-08T23:30:40.392656+0000","last_undegraded":"2024-05-08T23:30:40.392656+0000","last_fullsized":"2024-05-08T23:30:40.392656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392627+0000","last_change":"2024-05-08T23:30:38.694428+0000","last_active":"2024-05-08T23:30:40.392627+0000","last_peered":"2024-05-08T23:30:40.392627+0000","last_clean":"2024-05-08T23:30:40.392627+0000","last_became_active":"2024-05-08T23:30:38.693827+0000","last_became_peered":"2024-05-08T23:30:38.693827+0000","last_unstale":"2024-05-08T23:30:40.392627+0000","last_undegraded":"2024-05-08T23:30:40.392627+0000","last_fullsized":"2024-05-08T23:30:40.392627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:01.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389000+0000","last_change":"2024-05-08T23:30:38.309066+0000","last_active":"2024-05-08T23:30:40.389000+0000","last_peered":"2024-05-08T23:30:40.389000+0000","last_clean":"2024-05-08T23:30:40.389000+0000","last_became_active":"2024-05-08T23:30:38.308581+0000","last_became_peered":"2024-05-08T23:30:38.308581+0000","last_unstale":"2024-05-08T23:30:40.389000+0000","last_undegraded":"2024-05-08T23:30:40.389000+0000","last_fullsized":"2024-05-08T23:30:40.389000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388543+0000","last_change":"2024-05-08T23:30:38.704429+0000","last_active":"2024-05-08T23:30:39.388543+0000","last_peered":"2024-05-08T23:30:39.388543+0000","last_clean":"2024-05-08T23:30:39.388543+0000","last_became_active":"2024-05-08T23:30:38.703989+0000","last_became_peered":"2024-05-08T23:30:38.703989+0000","last_unstale":"2024-05-08T23:30:39.388543+0000","last_undegraded":"2024-05-08T23:30:39.388543+0000","last_fullsized":"2024-05-08T23:30:39.388543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:56.584934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.960069+0000","last_change":"2024-05-08T23:30:38.309970+0000","last_active":"2024-05-08T23:30:39.960069+0000","last_peered":"2024-05-08T23:30:39.960069+0000","last_clean":"2024-05-08T23:30:39.960069+0000","last_became_active":"2024-05-08T23:30:38.308960+0000","last_became_peered":"2024-05-08T23:30:38.308960+0000","last_unstale":"2024-05-08T23:30:39.960069+0000","last_undegraded":"2024-05-08T23:30:39.960069+0000","last_fullsized":"2024-05-08T23:30:39.960069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:41.142396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388847+0000","last_change":"2024-05-08T23:30:38.705438+0000","last_active":"2024-05-08T23:30:39.388847+0000","last_peered":"2024-05-08T23:30:39.388847+0000","last_clean":"2024-05-08T23:30:39.388847+0000","last_became_active":"2024-05-08T23:30:38.705141+0000","last_became_peered":"2024-05-08T23:30:38.705141+0000","last_unstale":"2024-05-08T23:30:39.388847+0000","last_undegraded":"2024-05-08T23:30:39.388847+0000","last_fullsized":"2024-05-08T23:30:39.388847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:51:37.465394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389256+0000","last_change":"2024-05-08T23:30:38.312114+0000","last_active":"2024-05-08T23:30:40.389256+0000","last_peered":"2024-05-08T23:30:40.389256+0000","last_clean":"2024-05-08T23:30:40.389256+0000","last_became_active":"2024-05-08T23:30:38.311826+0000","last_became_peered":"2024-05-08T23:30:38.311826+0000","last_unstale":"2024-05-08T23:30:40.389256+0000","last_undegraded":"2024-05-08T23:30:40.389256+0000","last_fullsized":"2024-05-08T23:30:40.389256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:12:55.004696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388919+0000","last_change":"2024-05-08T23:30:38.708759+0000","last_active":"2024-05-08T23:30:39.388919+0000","last_peered":"2024-05-08T23:30:39.388919+0000","last_clean":"2024-05-08T23:30:39.388919+0000","last_became_active":"2024-05-08T23:30:38.708506+0000","last_became_peered":"2024-05-08T23:30:38.708506+0000","last_unstale":"2024-05-08T23:30:39.388919+0000","last_undegraded":"2024-05-08T23:30:39.388919+0000","last_fullsized":"2024-05-08T23:30:39.388919+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:15.958328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389048+0000","last_change":"2024-05-08T23:30:38.314282+0000","last_active":"2024-05-08T23:30:40.389048+0000","last_peered":"2024-05-08T23:30:40.389048+0000","last_clean":"2024-05-08T23:30:40.389048+0000","last_became_active":"2024-05-08T23:30:38.314040+0000","last_became_peered":"2024-05-08T23:30:38.314040+0000","last_unstale":"2024-05-08T23:30:40.389048+0000","last_undegraded":"2024-05-08T23:30:40.389048+0000","last_fullsized":"2024-05-08T23:30:40.389048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:39.460999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243979+0000","last_change":"2024-05-08T23:30:38.688690+0000","last_active":"2024-05-08T23:30:40.243979+0000","last_peered":"2024-05-08T23:30:40.243979+0000","last_clean":"2024-05-08T23:30:40.243979+0000","last_became_active":"2024-05-08T23:30:38.688342+0000","last_became_peered":"2024-05-08T23:30:38.688342+0000","last_unstale":"2024-05-08T23:30:40.243979+0000","last_undegraded":"2024-05-08T23:30:40.243979+0000","last_fullsized":"2024-05-08T23:30:40.243979+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:13.779218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389159+0000","last_change":"2024-05-08T23:30:38.314058+0000","last_active":"2024-05-08T23:30:40.389159+0000","last_peered":"2024-05-08T23:30:40.389159+0000","last_clean":"2024-05-08T23:30:40.389159+0000","last_became_active":"2024-05-08T23:30:38.313813+0000","last_became_peered":"2024-05-08T23:30:38.313813+0000","last_unstale":"2024-05-08T23:30:40.389159+0000","last_undegraded":"2024-05-08T23:30:40.389159+0000","last_fullsized":"2024-05-08T23:30:40.389159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:14.380730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388747+0000","last_change":"2024-05-08T23:30:38.707800+0000","last_active":"2024-05-08T23:30:39.388747+0000","last_peered":"2024-05-08T23:30:39.388747+0000","last_clean":"2024-05-08T23:30:39.388747+0000","last_became_active":"2024-05-08T23:30:38.706575+0000","last_became_peered":"2024-05-08T23:30:38.706575+0000","last_unstale":"2024-05-08T23:30:39.388747+0000","last_undegraded":"2024-05-08T23:30:39.388747+0000","last_fullsized":"2024-05-08T23:30:39.388747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:16.601918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:38.695881+0000","last_change":"2024-05-08T23:30:27.226484+0000","last_active":"2024-05-08T23:30:38.695881+0000","last_peered":"2024-05-08T23:30:38.695881+0000","last_clean":"2024-05-08T23:30:38.695881+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:30:38.695881+0000","last_undegraded":"2024-05-08T23:30:38.695881+0000","last_fullsized":"2024-05-08T23:30:38.695881+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_clean_scrub_stamp":"2024-05-08T23:30:26.204118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:14.267287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243573+0000","last_change":"2024-05-08T23:30:38.317146+0000","last_active":"2024-05-08T23:30:40.243573+0000","last_peered":"2024-05-08T23:30:40.243573+0000","last_clean":"2024-05-08T23:30:40.243573+0000","last_became_active":"2024-05-08T23:30:38.316813+0000","last_became_peered":"2024-05-08T23:30:38.316813+0000","last_unstale":"2024-05-08T23:30:40.243573+0000","last_undegraded":"2024-05-08T23:30:40.243573+0000","last_fullsized":"2024-05-08T23:30:40.243573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:03.938472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675574+0000","last_change":"2024-05-08T23:30:38.690153+0000","last_active":"2024-05-08T23:30:39.675574+0000","last_peered":"2024-05-08T23:30:39.675574+0000","last_clean":"2024-05-08T23:30:39.675574+0000","last_became_active":"2024-05-08T23:30:38.689929+0000","last_became_peered":"2024-05-08T23:30:38.689929+0000","last_unstale":"2024-05-08T23:30:39.675574+0000","last_undegraded":"2024-05-08T23:30:39.675574+0000","last_fullsized":"2024-05-08T23:30:39.675574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:07.682170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388454+0000","last_change":"2024-05-08T23:30:38.319918+0000","last_active":"2024-05-08T23:30:39.388454+0000","last_peered":"2024-05-08T23:30:39.388454+0000","last_clean":"2024-05-08T23:30:39.388454+0000","last_became_active":"2024-05-08T23:30:38.319606+0000","last_became_peered":"2024-05-08T23:30:38.319606+0000","last_unstale":"2024-05-08T23:30:39.388454+0000","last_undegraded":"2024-05-08T23:30:39.388454+0000","last_fullsized":"2024-05-08T23:30:39.388454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:24.905425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392035+0000","last_change":"2024-05-08T23:30:38.681448+0000","last_active":"2024-05-08T23:30:39.392035+0000","last_peered":"2024-05-08T23:30:39.392035+0000","last_clean":"2024-05-08T23:30:39.392035+0000","last_became_active":"2024-05-08T23:30:38.681002+0000","last_became_peered":"2024-05-08T23:30:38.681002+0000","last_unstale":"2024-05-08T23:30:39.392035+0000","last_undegraded":"2024-05-08T23:30:39.392035+0000","last_fullsized":"2024-05-08T23:30:39.392035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:40.104243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.389112+0000","last_change":"2024-05-08T23:30:38.312104+0000","last_active":"2024-05-08T23:30:40.389112+0000","last_peered":"2024-05-08T23:30:40.389112+0000","last_clean":"2024-05-08T23:30:40.389112+0000","last_became_active":"2024-05-08T23:30:38.311833+0000","last_became_peered":"2024-05-08T23:30:38.311833+0000","last_unstale":"2024-05-08T23:30:40.389112+0000","last_undegraded":"2024-05-08T23:30:40.389112+0000","last_fullsized":"2024-05-08T23:30:40.389112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:31.924157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388624+0000","last_change":"2024-05-08T23:30:38.686436+0000","last_active":"2024-05-08T23:30:40.388624+0000","last_peered":"2024-05-08T23:30:40.388624+0000","last_clean":"2024-05-08T23:30:40.388624+0000","last_became_active":"2024-05-08T23:30:38.686176+0000","last_became_peered":"2024-05-08T23:30:38.686176+0000","last_unstale":"2024-05-08T23:30:40.388624+0000","last_undegraded":"2024-05-08T23:30:40.388624+0000","last_fullsized":"2024-05-08T23:30:40.388624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:41.133153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244021+0000","last_change":"2024-05-08T23:30:38.317558+0000","last_active":"2024-05-08T23:30:40.244021+0000","last_peered":"2024-05-08T23:30:40.244021+0000","last_clean":"2024-05-08T23:30:40.244021+0000","last_became_active":"2024-05-08T23:30:38.317296+0000","last_became_peered":"2024-05-08T23:30:38.317296+0000","last_unstale":"2024-05-08T23:30:40.244021+0000","last_undegraded":"2024-05-08T23:30:40.244021+0000","last_fullsized":"2024-05-08T23:30:40.244021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959765+0000","last_change":"2024-05-08T23:30:38.682924+0000","last_active":"2024-05-08T23:30:39.959765+0000","last_peered":"2024-05-08T23:30:39.959765+0000","last_clean":"2024-05-08T23:30:39.959765+0000","last_became_active":"2024-05-08T23:30:38.682112+0000","last_became_peered":"2024-05-08T23:30:38.682112+0000","last_unstale":"2024-05-08T23:30:39.959765+0000","last_undegraded":"2024-05-08T23:30:39.959765+0000","last_fullsized":"2024-05-08T23:30:39.959765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:04.948224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388873+0000","last_change":"2024-05-08T23:30:38.317398+0000","last_active":"2024-05-08T23:30:40.388873+0000","last_peered":"2024-05-08T23:30:40.388873+0000","last_clean":"2024-05-08T23:30:40.388873+0000","last_became_active":"2024-05-08T23:30:38.317167+0000","last_became_peered":"2024-05-08T23:30:38.317167+0000","last_unstale":"2024-05-08T23:30:40.388873+0000","last_undegraded":"2024-05-08T23:30:40.388873+0000","last_fullsized":"2024-05-08T23:30:40.388873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:14.409788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244386+0000","last_change":"2024-05-08T23:30:38.693098+0000","last_active":"2024-05-08T23:30:40.244386+0000","last_peered":"2024-05-08T23:30:40.244386+0000","last_clean":"2024-05-08T23:30:40.244386+0000","last_became_active":"2024-05-08T23:30:38.692852+0000","last_became_peered":"2024-05-08T23:30:38.692852+0000","last_unstale":"2024-05-08T23:30:40.244386+0000","last_undegraded":"2024-05-08T23:30:40.244386+0000","last_fullsized":"2024-05-08T23:30:40.244386+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:02.678862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388824+0000","last_change":"2024-05-08T23:30:38.320883+0000","last_active":"2024-05-08T23:30:39.388824+0000","last_peered":"2024-05-08T23:30:39.388824+0000","last_clean":"2024-05-08T23:30:39.388824+0000","last_became_active":"2024-05-08T23:30:38.320610+0000","last_became_peered":"2024-05-08T23:30:38.320610+0000","last_unstale":"2024-05-08T23:30:39.388824+0000","last_undegraded":"2024-05-08T23:30:39.388824+0000","last_fullsized":"2024-05-08T23:30:39.388824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:02.623735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388835+0000","last_change":"2024-05-08T23:30:38.687478+0000","last_active":"2024-05-08T23:30:40.388835+0000","last_peered":"2024-05-08T23:30:40.388835+0000","last_clean":"2024-05-08T23:30:40.388835+0000","last_became_active":"2024-05-08T23:30:38.686229+0000","last_became_peered":"2024-05-08T23:30:38.686229+0000","last_unstale":"2024-05-08T23:30:40.388835+0000","last_undegraded":"2024-05-08T23:30:40.388835+0000","last_fullsized":"2024-05-08T23:30:40.388835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:37.994049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.460801+0000","last_change":"2024-05-08T23:30:38.318046+0000","last_active":"2024-05-08T23:30:40.460801+0000","last_peered":"2024-05-08T23:30:40.460801+0000","last_clean":"2024-05-08T23:30:40.460801+0000","last_became_active":"2024-05-08T23:30:38.317757+0000","last_became_peered":"2024-05-08T23:30:38.317757+0000","last_unstale":"2024-05-08T23:30:40.460801+0000","last_undegraded":"2024-05-08T23:30:40.460801+0000","last_fullsized":"2024-05-08T23:30:40.460801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:22:00.309450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676059+0000","last_change":"2024-05-08T23:30:38.706013+0000","last_active":"2024-05-08T23:30:39.676059+0000","last_peered":"2024-05-08T23:30:39.676059+0000","last_clean":"2024-05-08T23:30:39.676059+0000","last_became_active":"2024-05-08T23:30:38.705709+0000","last_became_peered":"2024-05-08T23:30:38.705709+0000","last_unstale":"2024-05-08T23:30:39.676059+0000","last_undegraded":"2024-05-08T23:30:39.676059+0000","last_fullsized":"2024-05-08T23:30:39.676059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:25:01.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392885+0000","last_change":"2024-05-08T23:30:38.317850+0000","last_active":"2024-05-08T23:30:40.392885+0000","last_peered":"2024-05-08T23:30:40.392885+0000","last_clean":"2024-05-08T23:30:40.392885+0000","last_became_active":"2024-05-08T23:30:38.317598+0000","last_became_peered":"2024-05-08T23:30:38.317598+0000","last_unstale":"2024-05-08T23:30:40.392885+0000","last_undegraded":"2024-05-08T23:30:40.392885+0000","last_fullsized":"2024-05-08T23:30:40.392885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:39.460999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392452+0000","last_change":"2024-05-08T23:30:38.679213+0000","last_active":"2024-05-08T23:30:39.392452+0000","last_peered":"2024-05-08T23:30:39.392452+0000","last_clean":"2024-05-08T23:30:39.392452+0000","last_became_active":"2024-05-08T23:30:38.678935+0000","last_became_peered":"2024-05-08T23:30:38.678935+0000","last_unstale":"2024-05-08T23:30:39.392452+0000","last_undegraded":"2024-05-08T23:30:39.392452+0000","last_fullsized":"2024-05-08T23:30:39.392452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:39.006348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388422+0000","last_change":"2024-05-08T23:30:38.320666+0000","last_active":"2024-05-08T23:30:39.388422+0000","last_peered":"2024-05-08T23:30:39.388422+0000","last_clean":"2024-05-08T23:30:39.388422+0000","last_became_active":"2024-05-08T23:30:38.320387+0000","last_became_peered":"2024-05-08T23:30:38.320387+0000","last_unstale":"2024-05-08T23:30:39.388422+0000","last_undegraded":"2024-05-08T23:30:39.388422+0000","last_fullsized":"2024-05-08T23:30:39.388422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:35.811032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.959476+0000","last_change":"2024-05-08T23:30:38.692172+0000","last_active":"2024-05-08T23:30:39.959476+0000","last_peered":"2024-05-08T23:30:39.959476+0000","last_clean":"2024-05-08T23:30:39.959476+0000","last_became_active":"2024-05-08T23:30:38.688675+0000","last_became_peered":"2024-05-08T23:30:38.688675+0000","last_unstale":"2024-05-08T23:30:39.959476+0000","last_undegraded":"2024-05-08T23:30:39.959476+0000","last_fullsized":"2024-05-08T23:30:39.959476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:40.470751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388758+0000","last_change":"2024-05-08T23:30:38.319809+0000","last_active":"2024-05-08T23:30:40.388758+0000","last_peered":"2024-05-08T23:30:40.388758+0000","last_clean":"2024-05-08T23:30:40.388758+0000","last_became_active":"2024-05-08T23:30:38.319500+0000","last_became_peered":"2024-05-08T23:30:38.319500+0000","last_unstale":"2024-05-08T23:30:40.388758+0000","last_undegraded":"2024-05-08T23:30:40.388758+0000","last_fullsized":"2024-05-08T23:30:40.388758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:06.433390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675251+0000","last_change":"2024-05-08T23:30:38.694080+0000","last_active":"2024-05-08T23:30:39.675251+0000","last_peered":"2024-05-08T23:30:39.675251+0000","last_clean":"2024-05-08T23:30:39.675251+0000","last_became_active":"2024-05-08T23:30:38.693469+0000","last_became_peered":"2024-05-08T23:30:38.693469+0000","last_unstale":"2024-05-08T23:30:39.675251+0000","last_undegraded":"2024-05-08T23:30:39.675251+0000","last_fullsized":"2024-05-08T23:30:39.675251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:07.222797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.388797+0000","last_change":"2024-05-08T23:30:38.316969+0000","last_active":"2024-05-08T23:30:40.388797+0000","last_peered":"2024-05-08T23:30:40.388797+0000","last_clean":"2024-05-08T23:30:40.388797+0000","last_became_active":"2024-05-08T23:30:38.316648+0000","last_became_peered":"2024-05-08T23:30:38.316648+0000","last_unstale":"2024-05-08T23:30:40.388797+0000","last_undegraded":"2024-05-08T23:30:40.388797+0000","last_fullsized":"2024-05-08T23:30:40.388797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:38:03.938472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.675646+0000","last_change":"2024-05-08T23:30:38.696097+0000","last_active":"2024-05-08T23:30:39.675646+0000","last_peered":"2024-05-08T23:30:39.675646+0000","last_clean":"2024-05-08T23:30:39.675646+0000","last_became_active":"2024-05-08T23:30:38.695832+0000","last_became_peered":"2024-05-08T23:30:38.695832+0000","last_unstale":"2024-05-08T23:30:39.675646+0000","last_undegraded":"2024-05-08T23:30:39.675646+0000","last_fullsized":"2024-05-08T23:30:39.675646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:12.789913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388651+0000","last_change":"2024-05-08T23:30:38.320025+0000","last_active":"2024-05-08T23:30:39.388651+0000","last_peered":"2024-05-08T23:30:39.388651+0000","last_clean":"2024-05-08T23:30:39.388651+0000","last_became_active":"2024-05-08T23:30:38.319609+0000","last_became_peered":"2024-05-08T23:30:38.319609+0000","last_unstale":"2024-05-08T23:30:39.388651+0000","last_undegraded":"2024-05-08T23:30:39.388651+0000","last_fullsized":"2024-05-08T23:30:39.388651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:42.475179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.392678+0000","last_change":"2024-05-08T23:30:38.699456+0000","last_active":"2024-05-08T23:30:40.392678+0000","last_peered":"2024-05-08T23:30:40.392678+0000","last_clean":"2024-05-08T23:30:40.392678+0000","last_became_active":"2024-05-08T23:30:38.699013+0000","last_became_peered":"2024-05-08T23:30:38.699013+0000","last_unstale":"2024-05-08T23:30:40.392678+0000","last_undegraded":"2024-05-08T23:30:40.392678+0000","last_fullsized":"2024-05-08T23:30:40.392678+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:37.994049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.244260+0000","last_change":"2024-05-08T23:30:38.695946+0000","last_active":"2024-05-08T23:30:40.244260+0000","last_peered":"2024-05-08T23:30:40.244260+0000","last_clean":"2024-05-08T23:30:40.244260+0000","last_became_active":"2024-05-08T23:30:38.695704+0000","last_became_peered":"2024-05-08T23:30:38.695704+0000","last_unstale":"2024-05-08T23:30:40.244260+0000","last_undegraded":"2024-05-08T23:30:40.244260+0000","last_fullsized":"2024-05-08T23:30:40.244260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:18.614485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388680+0000","last_change":"2024-05-08T23:30:38.319876+0000","last_active":"2024-05-08T23:30:39.388680+0000","last_peered":"2024-05-08T23:30:39.388680+0000","last_clean":"2024-05-08T23:30:39.388680+0000","last_became_active":"2024-05-08T23:30:38.319589+0000","last_became_peered":"2024-05-08T23:30:38.319589+0000","last_unstale":"2024-05-08T23:30:39.388680+0000","last_undegraded":"2024-05-08T23:30:39.388680+0000","last_fullsized":"2024-05-08T23:30:39.388680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:12.479867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:40.243895+0000","last_change":"2024-05-08T23:30:38.697783+0000","last_active":"2024-05-08T23:30:40.243895+0000","last_peered":"2024-05-08T23:30:40.243895+0000","last_clean":"2024-05-08T23:30:40.243895+0000","last_became_active":"2024-05-08T23:30:38.697295+0000","last_became_peered":"2024-05-08T23:30:38.697295+0000","last_unstale":"2024-05-08T23:30:40.243895+0000","last_undegraded":"2024-05-08T23:30:40.243895+0000","last_fullsized":"2024-05-08T23:30:40.243895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:00:12.789913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.388775+0000","last_change":"2024-05-08T23:30:38.320543+0000","last_active":"2024-05-08T23:30:39.388775+0000","last_peered":"2024-05-08T23:30:39.388775+0000","last_clean":"2024-05-08T23:30:39.388775+0000","last_became_active":"2024-05-08T23:30:38.320231+0000","last_became_peered":"2024-05-08T23:30:38.320231+0000","last_unstale":"2024-05-08T23:30:39.388775+0000","last_undegraded":"2024-05-08T23:30:39.388775+0000","last_fullsized":"2024-05-08T23:30:39.388775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:08:09.987948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392499+0000","last_change":"2024-05-08T23:30:38.691260+0000","last_active":"2024-05-08T23:30:39.392499+0000","last_peered":"2024-05-08T23:30:39.392499+0000","last_clean":"2024-05-08T23:30:39.392499+0000","last_became_active":"2024-05-08T23:30:38.690856+0000","last_became_peered":"2024-05-08T23:30:38.690856+0000","last_unstale":"2024-05-08T23:30:39.392499+0000","last_undegraded":"2024-05-08T23:30:39.392499+0000","last_fullsized":"2024-05-08T23:30:39.392499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:58.926788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676224+0000","last_change":"2024-05-08T23:30:38.311855+0000","last_active":"2024-05-08T23:30:39.676224+0000","last_peered":"2024-05-08T23:30:39.676224+0000","last_clean":"2024-05-08T23:30:39.676224+0000","last_became_active":"2024-05-08T23:30:38.311542+0000","last_became_peered":"2024-05-08T23:30:38.311542+0000","last_unstale":"2024-05-08T23:30:39.676224+0000","last_undegraded":"2024-05-08T23:30:39.676224+0000","last_fullsized":"2024-05-08T23:30:39.676224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:30.035897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.676005+0000","last_change":"2024-05-08T23:30:38.688998+0000","last_active":"2024-05-08T23:30:39.676005+0000","last_peered":"2024-05-08T23:30:39.676005+0000","last_clean":"2024-05-08T23:30:39.676005+0000","last_became_active":"2024-05-08T23:30:38.688723+0000","last_became_peered":"2024-05-08T23:30:38.688723+0000","last_unstale":"2024-05-08T23:30:39.676005+0000","last_undegraded":"2024-05-08T23:30:39.676005+0000","last_fullsized":"2024-05-08T23:30:39.676005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:38.281246+0000","last_clean_scrub_stamp":"2024-05-08T23:30:38.281246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:41.133153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:30:39.392591+0000","last_change":"2024-05-08T23:30:38.315584+0000","last_active":"2024-05-08T23:30:39.392591+0000","last_peered":"2024-05-08T23:30:39.392591+0000","last_clean":"2024-05-08T23:30:39.392591+0000","last_became_active":"2024-05-08T23:30:38.315234+0000","last_became_peered":"2024-05-08T23:30:38.315234+0000","last_unstale":"2024-05-08T23:30:39.392591+0000","last_undegraded":"2024-05-08T23:30:39.392591+0000","last_fullsized":"2024-05-08T23:30:39.392591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:37.271494+0000","last_clean_scrub_stamp":"2024-05-08T23:30:37.271494+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:02.623735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":78378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":76904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":76940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":78378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":76974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":819200,"data_stored":667764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":667820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:30:47.051 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T23:30:47.051 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T23:30:47.051 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-08T23:30:47.051 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T23:30:47.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.145+0000 7fc976168700 1 Processor -- start 2024-05-08T23:30:47.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.145+0000 7fc976168700 1 -- start start 2024-05-08T23:30:47.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc976168700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc9700d00d0 0x7fc9700b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc976168700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9700d3e80 con 0x7fc970160ab0 2024-05-08T23:30:47.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc976168700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9700ce780 con 0x7fc9700b4610 2024-05-08T23:30:47.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc976168700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc970160e90 con 0x7fc9700d00d0 2024-05-08T23:30:47.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc975166700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc970160ab0 0x7fc9700cf9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38716/0 (socket says 172.21.15.81:38716) 2024-05-08T23:30:47.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc975166700 1 -- 172.21.15.81:0/3380861880 learned_addr learned my addr 172.21.15.81:0/3380861880 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:47.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc974965700 1 --2- 172.21.15.81:0/3380861880 >> v2:172.21.15.117:3300/0 conn(0x7fc9700d00d0 0x7fc9700b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc96ffff700 1 -- 172.21.15.81:0/3380861880 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763909365 0 0) 0x7fc9700d3e80 con 0x7fc970160ab0 2024-05-08T23:30:47.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc96ffff700 1 -- 172.21.15.81:0/3380861880 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9580035d0 con 0x7fc970160ab0 2024-05-08T23:30:47.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc974965700 1 -- 172.21.15.81:0/3380861880 >> v1:172.21.15.117:6789/0 conn(0x7fc9700b4610 legacy=0x7fc9700b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc96ffff700 1 -- 172.21.15.81:0/3380861880 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2689959867 0 0) 0x7fc9580035d0 con 0x7fc970160ab0 2024-05-08T23:30:47.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc974965700 1 -- 172.21.15.81:0/3380861880 >> v1:172.21.15.81:6789/0 conn(0x7fc970160ab0 legacy=0x7fc9700cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc974965700 1 -- 172.21.15.81:0/3380861880 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9700cea60 con 0x7fc9700d00d0 2024-05-08T23:30:47.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc974965700 1 --2- 172.21.15.81:0/3380861880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9700d00d0 0x7fc9700b1dc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc964009850 tx=0x7fc964009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61168cef1aa4b5eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:47.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc96ffff700 1 -- 172.21.15.81:0/3380861880 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960001670 con 0x7fc9700d00d0 2024-05-08T23:30:47.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc96ffff700 1 -- 172.21.15.81:0/3380861880 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc964004030 con 0x7fc9700d00d0 2024-05-08T23:30:47.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc96ffff700 1 -- 172.21.15.81:0/3380861880 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9640043c0 con 0x7fc9700d00d0 2024-05-08T23:30:47.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc976168700 1 -- 172.21.15.81:0/3380861880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9700d00d0 msgr2=0x7fc9700b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc976168700 1 --2- 172.21.15.81:0/3380861880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9700d00d0 0x7fc9700b1dc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc964009850 tx=0x7fc964009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:47.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc976168700 1 -- 172.21.15.81:0/3380861880 shutdown_connections 2024-05-08T23:30:47.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc976168700 1 --2- 172.21.15.81:0/3380861880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9700d00d0 0x7fc9700b1dc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc976168700 1 -- 172.21.15.81:0/3380861880 >> 172.21.15.81:0/3380861880 conn(0x7fc97009c390 msgr2=0x7fc9700d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:47.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc976168700 1 -- 172.21.15.81:0/3380861880 shutdown_connections 2024-05-08T23:30:47.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.149+0000 7fc976168700 1 -- 172.21.15.81:0/3380861880 wait complete. 2024-05-08T23:30:47.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc976168700 1 Processor -- start 2024-05-08T23:30:47.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc976168700 1 -- start start 2024-05-08T23:30:47.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc976168700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9700b4610 0x7fc9700ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc976168700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9700d00d0 0x7fc97015b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc976168700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc970160ab0 0x7fc9701f2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc976168700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9700ca840 con 0x7fc9700b4610 2024-05-08T23:30:47.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc976168700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97015b940 con 0x7fc970160ab0 2024-05-08T23:30:47.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc976168700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc970143fd0 con 0x7fc9700d00d0 2024-05-08T23:30:47.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc975166700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9700b4610 0x7fc9700ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc975166700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9700b4610 0x7fc9700ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51194/0 (socket says 172.21.15.81:51194) 2024-05-08T23:30:47.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc975166700 1 -- 172.21.15.81:0/143971390 learned_addr learned my addr 172.21.15.81:0/143971390 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:47.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc974965700 1 --2- 172.21.15.81:0/143971390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9700d00d0 0x7fc97015b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc975967700 1 --2- 172.21.15.81:0/143971390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc970160ab0 0x7fc9701f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc975166700 1 -- 172.21.15.81:0/143971390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9700d00d0 msgr2=0x7fc97015b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc975166700 1 --2- 172.21.15.81:0/143971390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9700d00d0 0x7fc97015b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc975166700 1 -- 172.21.15.81:0/143971390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc970160ab0 msgr2=0x7fc9701f2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc975166700 1 --2- 172.21.15.81:0/143971390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc970160ab0 0x7fc9701f2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc975166700 1 -- 172.21.15.81:0/143971390 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9640094b0 con 0x7fc9700b4610 2024-05-08T23:30:47.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc975967700 1 --2- 172.21.15.81:0/143971390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc970160ab0 0x7fc9701f2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:47.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc975166700 1 --2- 172.21.15.81:0/143971390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9700b4610 0x7fc9700ca300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc96000c540 tx=0x7fc96000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:47.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.153+0000 7fc96dffb700 1 -- 172.21.15.81:0/143971390 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960004ca0 con 0x7fc9700b4610 2024-05-08T23:30:47.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.157+0000 7fc96dffb700 1 -- 172.21.15.81:0/143971390 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc960004e00 con 0x7fc9700b4610 2024-05-08T23:30:47.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.157+0000 7fc976168700 1 -- 172.21.15.81:0/143971390 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97015bbc0 con 0x7fc9700b4610 2024-05-08T23:30:47.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.157+0000 7fc96dffb700 1 -- 172.21.15.81:0/143971390 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960007310 con 0x7fc9700b4610 2024-05-08T23:30:47.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.157+0000 7fc976168700 1 -- 172.21.15.81:0/143971390 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97015c180 con 0x7fc9700b4610 2024-05-08T23:30:47.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.157+0000 7fc976168700 1 -- 172.21.15.81:0/143971390 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97015c370 con 0x7fc9700b4610 2024-05-08T23:30:47.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.157+0000 7fc96dffb700 1 -- 172.21.15.81:0/143971390 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc960007470 con 0x7fc9700b4610 2024-05-08T23:30:47.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.157+0000 7fc96dffb700 1 --2- 172.21.15.81:0/143971390 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc95c066730 0x7fc95c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.157+0000 7fc974965700 1 --2- 172.21.15.81:0/143971390 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc95c066730 0x7fc95c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.157+0000 7fc96dffb700 1 -- 172.21.15.81:0/143971390 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc960083e70 con 0x7fc9700b4610 2024-05-08T23:30:47.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.161+0000 7fc974965700 1 --2- 172.21.15.81:0/143971390 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc95c066730 0x7fc95c068bf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc9640090b0 tx=0x7fc964008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:47.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.169+0000 7fc96dffb700 1 -- 172.21.15.81:0/143971390 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc9600511e0 con 0x7fc9700b4610 2024-05-08T23:30:47.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.357+0000 7fc976168700 1 -- 172.21.15.81:0/143971390 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc9700684b0 con 0x7fc9700b4610 2024-05-08T23:30:47.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.361+0000 7fc96dffb700 1 -- 172.21.15.81:0/143971390 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc960007720 con 0x7fc9700b4610 2024-05-08T23:30:47.364 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:47.364 INFO:teuthology.orchestra.run.smithi081.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T23:30:47.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.361+0000 7fc976168700 1 -- 172.21.15.81:0/143971390 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc95c066730 msgr2=0x7fc95c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.361+0000 7fc976168700 1 --2- 172.21.15.81:0/143971390 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc95c066730 0x7fc95c068bf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc9640090b0 tx=0x7fc964008030 comp rx=0 tx=0).stop 2024-05-08T23:30:47.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.361+0000 7fc976168700 1 -- 172.21.15.81:0/143971390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9700b4610 msgr2=0x7fc9700ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.361+0000 7fc976168700 1 --2- 172.21.15.81:0/143971390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9700b4610 0x7fc9700ca300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc96000c540 tx=0x7fc96000c850 comp rx=0 tx=0).stop 2024-05-08T23:30:47.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.361+0000 7fc976168700 1 -- 172.21.15.81:0/143971390 shutdown_connections 2024-05-08T23:30:47.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.361+0000 7fc976168700 1 --2- 172.21.15.81:0/143971390 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc95c066730 0x7fc95c068bf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.361+0000 7fc976168700 1 --2- 172.21.15.81:0/143971390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc970160ab0 0x7fc9701f2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.361+0000 7fc976168700 1 --2- 172.21.15.81:0/143971390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9700d00d0 0x7fc97015b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.361+0000 7fc976168700 1 --2- 172.21.15.81:0/143971390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9700b4610 0x7fc9700ca300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.361+0000 7fc976168700 1 -- 172.21.15.81:0/143971390 >> 172.21.15.81:0/143971390 conn(0x7fc97009c390 msgr2=0x7fc9700b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:47.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.361+0000 7fc976168700 1 -- 172.21.15.81:0/143971390 shutdown_connections 2024-05-08T23:30:47.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.361+0000 7fc976168700 1 -- 172.21.15.81:0/143971390 wait complete. 2024-05-08T23:30:47.377 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-08T23:30:47.377 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:30:47.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.477+0000 7f412359e700 1 Processor -- start 2024-05-08T23:30:47.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.477+0000 7f412359e700 1 -- start start 2024-05-08T23:30:47.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.477+0000 7f412359e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41240c3dd0 0x7f41240c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.477+0000 7f412359e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4124161960 0x7f41240cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.477+0000 7f412359e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412406bcd0 con 0x7f41240afd90 2024-05-08T23:30:47.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.477+0000 7f412359e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41240ce9d0 con 0x7f4124161960 2024-05-08T23:30:47.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.477+0000 7f412359e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41240cecb0 con 0x7f41240c3dd0 2024-05-08T23:30:47.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.477+0000 7f4122d9d700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f41240afd90 0x7f41240cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38724/0 (socket says 172.21.15.81:38724) 2024-05-08T23:30:47.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.477+0000 7f4122d9d700 1 -- 172.21.15.81:0/3455016035 learned_addr learned my addr 172.21.15.81:0/3455016035 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:47.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.477+0000 7f4121d9b700 1 --2- 172.21.15.81:0/3455016035 >> v2:172.21.15.117:3300/0 conn(0x7f4124161960 0x7f41240cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.477+0000 7f412259c700 1 --2- 172.21.15.81:0/3455016035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41240c3dd0 0x7f41240c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f412159a700 1 -- 172.21.15.81:0/3455016035 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864685830 0 0) 0x7f412406bcd0 con 0x7f41240afd90 2024-05-08T23:30:47.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f412159a700 1 -- 172.21.15.81:0/3455016035 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f410c0035d0 con 0x7f41240afd90 2024-05-08T23:30:47.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f4121d9b700 1 -- 172.21.15.81:0/3455016035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41240c3dd0 msgr2=0x7f41240c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:47.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f4121d9b700 1 --2- 172.21.15.81:0/3455016035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41240c3dd0 0x7f41240c41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f4121d9b700 1 -- 172.21.15.81:0/3455016035 >> v1:172.21.15.81:6789/0 conn(0x7f41240afd90 legacy=0x7f41240cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f4121d9b700 1 -- 172.21.15.81:0/3455016035 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412406b470 con 0x7f4124161960 2024-05-08T23:30:47.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f4121d9b700 1 --2- 172.21.15.81:0/3455016035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4124161960 0x7f41240cf140 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4114009850 tx=0x7f4114009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82cee71a30136d3c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:47.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f412159a700 1 -- 172.21.15.81:0/3455016035 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114011830 con 0x7f4124161960 2024-05-08T23:30:47.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f412159a700 1 -- 172.21.15.81:0/3455016035 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4114011990 con 0x7f4124161960 2024-05-08T23:30:47.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f412159a700 1 -- 172.21.15.81:0/3455016035 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114011d40 con 0x7f4124161960 2024-05-08T23:30:47.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f412359e700 1 -- 172.21.15.81:0/3455016035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4124161960 msgr2=0x7f41240cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f412359e700 1 --2- 172.21.15.81:0/3455016035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4124161960 0x7f41240cf140 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4114009850 tx=0x7f4114009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:47.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f412359e700 1 -- 172.21.15.81:0/3455016035 shutdown_connections 2024-05-08T23:30:47.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f412359e700 1 --2- 172.21.15.81:0/3455016035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4124161960 0x7f41240cf140 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f412359e700 1 --2- 172.21.15.81:0/3455016035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41240c3dd0 0x7f41240c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f412359e700 1 -- 172.21.15.81:0/3455016035 >> 172.21.15.81:0/3455016035 conn(0x7f41240b2970 msgr2=0x7f41240b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:47.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f412359e700 1 -- 172.21.15.81:0/3455016035 shutdown_connections 2024-05-08T23:30:47.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.481+0000 7f412359e700 1 -- 172.21.15.81:0/3455016035 wait complete. 2024-05-08T23:30:47.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.485+0000 7f412359e700 1 Processor -- start 2024-05-08T23:30:47.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.485+0000 7f412359e700 1 -- start start 2024-05-08T23:30:47.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.485+0000 7f412359e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41240afd90 0x7f4124149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.485+0000 7f412359e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41240c3dd0 0x7f41240c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.485+0000 7f412359e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4124161960 0x7f41240d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.485+0000 7f412359e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412414a4b0 con 0x7f41240c3dd0 2024-05-08T23:30:47.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.485+0000 7f412359e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41240c4e40 con 0x7f41240afd90 2024-05-08T23:30:47.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.485+0000 7f412359e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412414d370 con 0x7f4124161960 2024-05-08T23:30:47.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.485+0000 7f4121d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41240c3dd0 0x7f41240c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.485+0000 7f4121d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41240c3dd0 0x7f41240c4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51208/0 (socket says 172.21.15.81:51208) 2024-05-08T23:30:47.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.485+0000 7f4121d9b700 1 -- 172.21.15.81:0/2776180769 learned_addr learned my addr 172.21.15.81:0/2776180769 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:47.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.485+0000 7f412259c700 1 --2- 172.21.15.81:0/2776180769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41240afd90 0x7f4124149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.485+0000 7f4122d9d700 1 --2- 172.21.15.81:0/2776180769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4124161960 0x7f41240d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.489+0000 7f4122d9d700 1 -- 172.21.15.81:0/2776180769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41240afd90 msgr2=0x7f4124149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.489+0000 7f4122d9d700 1 --2- 172.21.15.81:0/2776180769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41240afd90 0x7f4124149f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.489+0000 7f4122d9d700 1 -- 172.21.15.81:0/2776180769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41240c3dd0 msgr2=0x7f41240c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.489+0000 7f4122d9d700 1 --2- 172.21.15.81:0/2776180769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41240c3dd0 0x7f41240c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.489+0000 7f4122d9d700 1 -- 172.21.15.81:0/2776180769 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41140094b0 con 0x7f4124161960 2024-05-08T23:30:47.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.489+0000 7f4121d9b700 1 --2- 172.21.15.81:0/2776180769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41240c3dd0 0x7f41240c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:47.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.489+0000 7f4122d9d700 1 --2- 172.21.15.81:0/2776180769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4124161960 0x7f41240d0c80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f411c00cad0 tx=0x7f411c00cb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:47.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.489+0000 7f4112ffd700 1 -- 172.21.15.81:0/2776180769 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c007480 con 0x7f4124161960 2024-05-08T23:30:47.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.489+0000 7f4112ffd700 1 -- 172.21.15.81:0/2776180769 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c0075e0 con 0x7f4124161960 2024-05-08T23:30:47.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.489+0000 7f412359e700 1 -- 172.21.15.81:0/2776180769 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41240c7750 con 0x7f4124161960 2024-05-08T23:30:47.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.489+0000 7f4112ffd700 1 -- 172.21.15.81:0/2776180769 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c007950 con 0x7f4124161960 2024-05-08T23:30:47.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.489+0000 7f412359e700 1 -- 172.21.15.81:0/2776180769 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41240c7470 con 0x7f4124161960 2024-05-08T23:30:47.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.489+0000 7f412359e700 1 -- 172.21.15.81:0/2776180769 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41240672a0 con 0x7f4124161960 2024-05-08T23:30:47.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.489+0000 7f4112ffd700 1 -- 172.21.15.81:0/2776180769 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f411c01c440 con 0x7f4124161960 2024-05-08T23:30:47.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.493+0000 7f4112ffd700 1 --2- 172.21.15.81:0/2776180769 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4104066730 0x7f4104068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.493+0000 7f412259c700 1 --2- 172.21.15.81:0/2776180769 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4104066730 0x7f4104068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.493+0000 7f4112ffd700 1 -- 172.21.15.81:0/2776180769 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f411c083de0 con 0x7f4124161960 2024-05-08T23:30:47.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.493+0000 7f412259c700 1 --2- 172.21.15.81:0/2776180769 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4104066730 0x7f4104068bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4118009bf0 tx=0x7f4118009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:47.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.501+0000 7f4112ffd700 1 -- 172.21.15.81:0/2776180769 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f411c051200 con 0x7f4124161960 2024-05-08T23:30:47.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.657+0000 7f412359e700 1 -- 172.21.15.81:0/2776180769 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f412406f7f0 con 0x7f4124161960 2024-05-08T23:30:47.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.661+0000 7f4112ffd700 1 -- 172.21.15.81:0/2776180769 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f411c054820 con 0x7f4124161960 2024-05-08T23:30:47.666 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:47.667 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:30:47.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.665+0000 7f412359e700 1 -- 172.21.15.81:0/2776180769 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4104066730 msgr2=0x7f4104068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.665+0000 7f412359e700 1 --2- 172.21.15.81:0/2776180769 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4104066730 0x7f4104068bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4118009bf0 tx=0x7f4118009240 comp rx=0 tx=0).stop 2024-05-08T23:30:47.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.665+0000 7f412359e700 1 -- 172.21.15.81:0/2776180769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4124161960 msgr2=0x7f41240d0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.665+0000 7f412359e700 1 --2- 172.21.15.81:0/2776180769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4124161960 0x7f41240d0c80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f411c00cad0 tx=0x7f411c00cb00 comp rx=0 tx=0).stop 2024-05-08T23:30:47.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.665+0000 7f412359e700 1 -- 172.21.15.81:0/2776180769 shutdown_connections 2024-05-08T23:30:47.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.665+0000 7f412359e700 1 --2- 172.21.15.81:0/2776180769 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4104066730 0x7f4104068bf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.665+0000 7f412359e700 1 --2- 172.21.15.81:0/2776180769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4124161960 0x7f41240d0c80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.665+0000 7f412359e700 1 --2- 172.21.15.81:0/2776180769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41240c3dd0 0x7f41240c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.665+0000 7f412359e700 1 --2- 172.21.15.81:0/2776180769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41240afd90 0x7f4124149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.665+0000 7f412359e700 1 -- 172.21.15.81:0/2776180769 >> 172.21.15.81:0/2776180769 conn(0x7f41240b2970 msgr2=0x7f41240cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:47.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.665+0000 7f412359e700 1 -- 172.21.15.81:0/2776180769 shutdown_connections 2024-05-08T23:30:47.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.665+0000 7f412359e700 1 -- 172.21.15.81:0/2776180769 wait complete. 2024-05-08T23:30:47.686 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:30:47.686 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:30:47.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.801+0000 7f725334e700 1 Processor -- start 2024-05-08T23:30:47.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.801+0000 7f725334e700 1 -- start start 2024-05-08T23:30:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.801+0000 7f725334e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f724c158d30 0x7f724c0ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.801+0000 7f725334e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f724c06ba80 0x7f724c06c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.801+0000 7f725334e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724c15c4e0 con 0x7f724c0aecc0 2024-05-08T23:30:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.801+0000 7f725334e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724c15b530 con 0x7f724c06ba80 2024-05-08T23:30:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.801+0000 7f725334e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724c15b720 con 0x7f724c158d30 2024-05-08T23:30:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.801+0000 7f7252b4d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f724c06ba80 0x7f724c06c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.801+0000 7f7252b4d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f724c06ba80 0x7f724c06c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:53978/0 (socket says 172.21.15.81:53978) 2024-05-08T23:30:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.801+0000 7f725234c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f724c158d30 0x7f724c0ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.801+0000 7f7252b4d700 1 -- 172.21.15.81:0/2758171424 learned_addr learned my addr 172.21.15.81:0/2758171424 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:47.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f7252b4d700 1 -- 172.21.15.81:0/2758171424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f724c158d30 msgr2=0x7f724c0ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:47.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f7252b4d700 1 --2- 172.21.15.81:0/2758171424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f724c158d30 0x7f724c0ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f7252b4d700 1 -- 172.21.15.81:0/2758171424 >> v1:172.21.15.117:6789/0 conn(0x7f724c0aecc0 legacy=0x7f724c06b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f7252b4d700 1 -- 172.21.15.81:0/2758171424 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c15ba00 con 0x7f724c06ba80 2024-05-08T23:30:47.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f7252b4d700 1 --2- 172.21.15.81:0/2758171424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c06ba80 0x7f724c06c9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7244009e50 tx=0x7f724400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9e4bf472a046193 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:47.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f725134a700 1 -- 172.21.15.81:0/2758171424 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724400e070 con 0x7f724c06ba80 2024-05-08T23:30:47.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f725134a700 1 -- 172.21.15.81:0/2758171424 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724400b910 con 0x7f724c06ba80 2024-05-08T23:30:47.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f725134a700 1 -- 172.21.15.81:0/2758171424 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724400bcc0 con 0x7f724c06ba80 2024-05-08T23:30:47.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f725334e700 1 -- 172.21.15.81:0/2758171424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c06ba80 msgr2=0x7f724c06c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f725334e700 1 --2- 172.21.15.81:0/2758171424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c06ba80 0x7f724c06c9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7244009e50 tx=0x7f724400b330 comp rx=0 tx=0).stop 2024-05-08T23:30:47.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f725334e700 1 -- 172.21.15.81:0/2758171424 shutdown_connections 2024-05-08T23:30:47.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f725334e700 1 --2- 172.21.15.81:0/2758171424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c06ba80 0x7f724c06c9a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f725334e700 1 --2- 172.21.15.81:0/2758171424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f724c158d30 0x7f724c0ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f725334e700 1 -- 172.21.15.81:0/2758171424 >> 172.21.15.81:0/2758171424 conn(0x7f724c070940 msgr2=0x7f724c070d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:47.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f725334e700 1 -- 172.21.15.81:0/2758171424 shutdown_connections 2024-05-08T23:30:47.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f725334e700 1 -- 172.21.15.81:0/2758171424 wait complete. 2024-05-08T23:30:47.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.805+0000 7f725334e700 1 Processor -- start 2024-05-08T23:30:47.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f725334e700 1 -- start start 2024-05-08T23:30:47.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f725334e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f724c06ba80 0x7f724c142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f725334e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f724c0aecc0 0x7f724c14fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f725334e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c158d30 0x7f724c14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f725334e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724c1516e0 con 0x7f724c0aecc0 2024-05-08T23:30:47.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f725334e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724c14f630 con 0x7f724c158d30 2024-05-08T23:30:47.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f725334e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724c1434d0 con 0x7f724c06ba80 2024-05-08T23:30:47.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f7251b4b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f724c0aecc0 0x7f724c14fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f7251b4b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f724c0aecc0 0x7f724c14fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51220/0 (socket says 172.21.15.81:51220) 2024-05-08T23:30:47.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f7251b4b700 1 -- 172.21.15.81:0/2738830083 learned_addr learned my addr 172.21.15.81:0/2738830083 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:47.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f7252b4d700 1 --2- 172.21.15.81:0/2738830083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c158d30 0x7f724c14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f725234c700 1 --2- 172.21.15.81:0/2738830083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f724c06ba80 0x7f724c142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f7251b4b700 1 -- 172.21.15.81:0/2738830083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f724c06ba80 msgr2=0x7f724c142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f7251b4b700 1 --2- 172.21.15.81:0/2738830083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f724c06ba80 0x7f724c142f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f7251b4b700 1 -- 172.21.15.81:0/2738830083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c158d30 msgr2=0x7f724c14ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f7251b4b700 1 --2- 172.21.15.81:0/2738830083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c158d30 0x7f724c14ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f7251b4b700 1 -- 172.21.15.81:0/2738830083 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7244009b30 con 0x7f724c0aecc0 2024-05-08T23:30:47.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f7251b4b700 1 --2- 172.21.15.81:0/2738830083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f724c0aecc0 0x7f724c14fa90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f723c00c500 tx=0x7f723c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:47.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f7242ffd700 1 -- 172.21.15.81:0/2738830083 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c0052d0 con 0x7f724c0aecc0 2024-05-08T23:30:47.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f7242ffd700 1 -- 172.21.15.81:0/2738830083 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c005430 con 0x7f724c0aecc0 2024-05-08T23:30:47.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f725334e700 1 -- 172.21.15.81:0/2738830083 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f724c151910 con 0x7f724c0aecc0 2024-05-08T23:30:47.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f7242ffd700 1 -- 172.21.15.81:0/2738830083 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c0055a0 con 0x7f724c0aecc0 2024-05-08T23:30:47.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.809+0000 7f725334e700 1 -- 172.21.15.81:0/2738830083 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f724c151ed0 con 0x7f724c0aecc0 2024-05-08T23:30:47.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.813+0000 7f725334e700 1 -- 172.21.15.81:0/2738830083 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f724c0672a0 con 0x7f724c0aecc0 2024-05-08T23:30:47.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.813+0000 7f7242ffd700 1 -- 172.21.15.81:0/2738830083 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f723c0038e0 con 0x7f724c0aecc0 2024-05-08T23:30:47.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.813+0000 7f7242ffd700 1 --2- 172.21.15.81:0/2738830083 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7230066760 0x7f7230068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:47.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.813+0000 7f7242ffd700 1 -- 172.21.15.81:0/2738830083 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f723c083500 con 0x7f724c0aecc0 2024-05-08T23:30:47.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.813+0000 7f725234c700 1 --2- 172.21.15.81:0/2738830083 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7230066760 0x7f7230068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:47.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.813+0000 7f725234c700 1 --2- 172.21.15.81:0/2738830083 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7230066760 0x7f7230068c20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7248009b50 tx=0x7f7248009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:47.824 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.821+0000 7f7242ffd700 1 -- 172.21.15.81:0/2738830083 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f723c050870 con 0x7f724c0aecc0 2024-05-08T23:30:47.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.989+0000 7f725334e700 1 -- 172.21.15.81:0/2738830083 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f724c071b50 con 0x7f724c0aecc0 2024-05-08T23:30:47.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.989+0000 7f7242ffd700 1 -- 172.21.15.81:0/2738830083 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f723c053e90 con 0x7f724c0aecc0 2024-05-08T23:30:47.993 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:30:47.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.989+0000 7f725334e700 1 -- 172.21.15.81:0/2738830083 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7230066760 msgr2=0x7f7230068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.989+0000 7f725334e700 1 --2- 172.21.15.81:0/2738830083 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7230066760 0x7f7230068c20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7248009b50 tx=0x7f7248009240 comp rx=0 tx=0).stop 2024-05-08T23:30:47.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.989+0000 7f725334e700 1 -- 172.21.15.81:0/2738830083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f724c0aecc0 msgr2=0x7f724c14fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:47.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.989+0000 7f725334e700 1 --2- 172.21.15.81:0/2738830083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f724c0aecc0 0x7f724c14fa90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f723c00c500 tx=0x7f723c00c810 comp rx=0 tx=0).stop 2024-05-08T23:30:47.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.989+0000 7f725334e700 1 -- 172.21.15.81:0/2738830083 shutdown_connections 2024-05-08T23:30:47.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.989+0000 7f725334e700 1 --2- 172.21.15.81:0/2738830083 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7230066760 0x7f7230068c20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.989+0000 7f725334e700 1 --2- 172.21.15.81:0/2738830083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c158d30 0x7f724c14ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.989+0000 7f725334e700 1 --2- 172.21.15.81:0/2738830083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f724c0aecc0 0x7f724c14fa90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.989+0000 7f725334e700 1 --2- 172.21.15.81:0/2738830083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f724c06ba80 0x7f724c142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:47.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.989+0000 7f725334e700 1 -- 172.21.15.81:0/2738830083 >> 172.21.15.81:0/2738830083 conn(0x7f724c070940 msgr2=0x7f724c070d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:47.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.989+0000 7f725334e700 1 -- 172.21.15.81:0/2738830083 shutdown_connections 2024-05-08T23:30:47.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:47.989+0000 7f725334e700 1 -- 172.21.15.81:0/2738830083 wait complete. 2024-05-08T23:30:48.007 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:30:48.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08e0dbb700 1 Processor -- start 2024-05-08T23:30:48.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08e0dbb700 1 -- start start 2024-05-08T23:30:48.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08e0dbb700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f08dc160920 0x7f08dc0ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08e0dbb700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08dc0d3d40 con 0x7f08dc0d1da0 2024-05-08T23:30:48.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08e0dbb700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08dc0d0d00 con 0x7f08dc0d0050 2024-05-08T23:30:48.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08e0dbb700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08dc0d0fe0 con 0x7f08dc160920 2024-05-08T23:30:48.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08db7fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f08dc160920 0x7f08dc0ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08daffd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f08dc0d1da0 0x7f08dc0cf940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38740/0 (socket says 172.21.15.81:38740) 2024-05-08T23:30:48.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08daffd700 1 -- 172.21.15.81:0/1422582304 learned_addr learned my addr 172.21.15.81:0/1422582304 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:48.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08db7fe700 1 -- 172.21.15.81:0/1422582304 >> v1:172.21.15.117:6789/0 conn(0x7f08dc0d0050 legacy=0x7f08dc0b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08db7fe700 1 -- 172.21.15.81:0/1422582304 >> v1:172.21.15.81:6789/0 conn(0x7f08dc0d1da0 legacy=0x7f08dc0cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08db7fe700 1 -- 172.21.15.81:0/1422582304 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc0d12c0 con 0x7f08dc160920 2024-05-08T23:30:48.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08db7fe700 1 --2- 172.21.15.81:0/1422582304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08dc160920 0x7f08dc0ce6f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009850 tx=0x7f08cc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee5a0284f576354 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:48.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08da7fc700 1 -- 172.21.15.81:0/1422582304 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc011830 con 0x7f08dc160920 2024-05-08T23:30:48.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08da7fc700 1 -- 172.21.15.81:0/1422582304 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08cc011990 con 0x7f08dc160920 2024-05-08T23:30:48.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08da7fc700 1 -- 172.21.15.81:0/1422582304 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc011d20 con 0x7f08dc160920 2024-05-08T23:30:48.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08e0dbb700 1 -- 172.21.15.81:0/1422582304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08dc160920 msgr2=0x7f08dc0ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.125+0000 7f08e0dbb700 1 --2- 172.21.15.81:0/1422582304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08dc160920 0x7f08dc0ce6f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009850 tx=0x7f08cc009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:48.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.129+0000 7f08e0dbb700 1 -- 172.21.15.81:0/1422582304 shutdown_connections 2024-05-08T23:30:48.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.129+0000 7f08e0dbb700 1 --2- 172.21.15.81:0/1422582304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08dc160920 0x7f08dc0ce6f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.129+0000 7f08e0dbb700 1 -- 172.21.15.81:0/1422582304 >> 172.21.15.81:0/1422582304 conn(0x7f08dc09c340 msgr2=0x7f08dc0d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:48.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.129+0000 7f08e0dbb700 1 -- 172.21.15.81:0/1422582304 shutdown_connections 2024-05-08T23:30:48.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.129+0000 7f08e0dbb700 1 -- 172.21.15.81:0/1422582304 wait complete. 2024-05-08T23:30:48.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.129+0000 7f08e0dbb700 1 Processor -- start 2024-05-08T23:30:48.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.129+0000 7f08e0dbb700 1 -- start start 2024-05-08T23:30:48.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08e0dbb700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08dc0d0050 0x7f08dc14e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08e0dbb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08dc0d1da0 0x7f08dc1e7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08e0dbb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08dc160920 0x7f08dc1e8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08e0dbb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc1e8f50 con 0x7f08dc160920 2024-05-08T23:30:48.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08e0dbb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc1e8990 con 0x7f08dc0d1da0 2024-05-08T23:30:48.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08e0dbb700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc1e8c70 con 0x7f08dc0d0050 2024-05-08T23:30:48.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08dbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08dc160920 0x7f08dc1e8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08dbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08dc160920 0x7f08dc1e8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51232/0 (socket says 172.21.15.81:51232) 2024-05-08T23:30:48.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08dbfff700 1 -- 172.21.15.81:0/1303134920 learned_addr learned my addr 172.21.15.81:0/1303134920 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:48.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08daffd700 1 --2- 172.21.15.81:0/1303134920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08dc0d1da0 0x7f08dc1e7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08db7fe700 1 --2- 172.21.15.81:0/1303134920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08dc0d0050 0x7f08dc14e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08dbfff700 1 -- 172.21.15.81:0/1303134920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08dc0d0050 msgr2=0x7f08dc14e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08dbfff700 1 --2- 172.21.15.81:0/1303134920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08dc0d0050 0x7f08dc14e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08dbfff700 1 -- 172.21.15.81:0/1303134920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08dc0d1da0 msgr2=0x7f08dc1e7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08dbfff700 1 --2- 172.21.15.81:0/1303134920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08dc0d1da0 0x7f08dc1e7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08dbfff700 1 -- 172.21.15.81:0/1303134920 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc0094b0 con 0x7f08dc160920 2024-05-08T23:30:48.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08dbfff700 1 --2- 172.21.15.81:0/1303134920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08dc160920 0x7f08dc1e8130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08d400cbc0 tx=0x7f08d400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:48.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08c3fff700 1 -- 172.21.15.81:0/1303134920 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4003c20 con 0x7f08dc160920 2024-05-08T23:30:48.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08c3fff700 1 -- 172.21.15.81:0/1303134920 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d4003d80 con 0x7f08dc160920 2024-05-08T23:30:48.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08e0dbb700 1 -- 172.21.15.81:0/1303134920 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08dc160e80 con 0x7f08dc160920 2024-05-08T23:30:48.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08c3fff700 1 -- 172.21.15.81:0/1303134920 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d400aa20 con 0x7f08dc160920 2024-05-08T23:30:48.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08e0dbb700 1 -- 172.21.15.81:0/1303134920 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc09e5e0 con 0x7f08dc160920 2024-05-08T23:30:48.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.133+0000 7f08e0dbb700 1 -- 172.21.15.81:0/1303134920 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08dc09e800 con 0x7f08dc160920 2024-05-08T23:30:48.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.137+0000 7f08c3fff700 1 -- 172.21.15.81:0/1303134920 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f08d400ab80 con 0x7f08dc160920 2024-05-08T23:30:48.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.137+0000 7f08c3fff700 1 --2- 172.21.15.81:0/1303134920 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f08c8066730 0x7f08c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.137+0000 7f08db7fe700 1 --2- 172.21.15.81:0/1303134920 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f08c8066730 0x7f08c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.137+0000 7f08c3fff700 1 -- 172.21.15.81:0/1303134920 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08d4084080 con 0x7f08dc160920 2024-05-08T23:30:48.142 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.137+0000 7f08db7fe700 1 --2- 172.21.15.81:0/1303134920 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f08c8066730 0x7f08c8068bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f08cc014040 tx=0x7f08cc01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:48.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.145+0000 7f08c3fff700 1 -- 172.21.15.81:0/1303134920 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f08d40513d0 con 0x7f08dc160920 2024-05-08T23:30:48.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.317+0000 7f08e0dbb700 1 -- 172.21.15.81:0/1303134920 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f08dc0684b0 con 0x7f08dc160920 2024-05-08T23:30:48.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.317+0000 7f08c3fff700 1 -- 172.21.15.81:0/1303134920 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f08d40549f0 con 0x7f08dc160920 2024-05-08T23:30:48.321 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:30:48.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.321+0000 7f08e0dbb700 1 -- 172.21.15.81:0/1303134920 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f08c8066730 msgr2=0x7f08c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.321+0000 7f08e0dbb700 1 --2- 172.21.15.81:0/1303134920 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f08c8066730 0x7f08c8068bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f08cc014040 tx=0x7f08cc01c040 comp rx=0 tx=0).stop 2024-05-08T23:30:48.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.321+0000 7f08e0dbb700 1 -- 172.21.15.81:0/1303134920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08dc160920 msgr2=0x7f08dc1e8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.321+0000 7f08e0dbb700 1 --2- 172.21.15.81:0/1303134920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08dc160920 0x7f08dc1e8130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08d400cbc0 tx=0x7f08d400ced0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.321+0000 7f08e0dbb700 1 -- 172.21.15.81:0/1303134920 shutdown_connections 2024-05-08T23:30:48.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.321+0000 7f08e0dbb700 1 --2- 172.21.15.81:0/1303134920 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f08c8066730 0x7f08c8068bf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.321+0000 7f08e0dbb700 1 --2- 172.21.15.81:0/1303134920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08dc160920 0x7f08dc1e8130 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.321+0000 7f08e0dbb700 1 --2- 172.21.15.81:0/1303134920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08dc0d1da0 0x7f08dc1e7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.321+0000 7f08e0dbb700 1 --2- 172.21.15.81:0/1303134920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08dc0d0050 0x7f08dc14e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.321+0000 7f08e0dbb700 1 -- 172.21.15.81:0/1303134920 >> 172.21.15.81:0/1303134920 conn(0x7f08dc09c340 msgr2=0x7f08dc0ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:48.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.321+0000 7f08e0dbb700 1 -- 172.21.15.81:0/1303134920 shutdown_connections 2024-05-08T23:30:48.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.321+0000 7f08e0dbb700 1 -- 172.21.15.81:0/1303134920 wait complete. 2024-05-08T23:30:48.336 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:30:48.458 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.453+0000 7ffa73214700 1 Processor -- start 2024-05-08T23:30:48.458 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.453+0000 7ffa73214700 1 -- start start 2024-05-08T23:30:48.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.453+0000 7ffa73214700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffa6c071dd0 0x7ffa6c06a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.453+0000 7ffa73214700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa6c15c0f0 0x7ffa6c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.453+0000 7ffa73214700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6c0cf720 con 0x7ffa6c158760 2024-05-08T23:30:48.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.453+0000 7ffa73214700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa6c0cfc50 con 0x7ffa6c071dd0 2024-05-08T23:30:48.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.453+0000 7ffa73214700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c15b400 con 0x7ffa6c15c0f0 2024-05-08T23:30:48.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa71a11700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffa6c071dd0 0x7ffa6c06a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa72a13700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa6c15c0f0 0x7ffa6c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa72a13700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa6c15c0f0 0x7ffa6c06a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:41126/0 (socket says 172.21.15.81:41126) 2024-05-08T23:30:48.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa72a13700 1 -- 172.21.15.81:0/2972211695 learned_addr learned my addr 172.21.15.81:0/2972211695 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:48.461 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa72a13700 1 -- 172.21.15.81:0/2972211695 >> v2:172.21.15.117:3300/0 conn(0x7ffa6c071dd0 msgr2=0x7ffa6c06a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:48.461 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa72a13700 1 --2- 172.21.15.81:0/2972211695 >> v2:172.21.15.117:3300/0 conn(0x7ffa6c071dd0 0x7ffa6c06a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa72a13700 1 -- 172.21.15.81:0/2972211695 >> v1:172.21.15.117:6789/0 conn(0x7ffa6c158760 legacy=0x7ffa6c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa72a13700 1 -- 172.21.15.81:0/2972211695 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6c0cf970 con 0x7ffa6c15c0f0 2024-05-08T23:30:48.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa72a13700 1 --2- 172.21.15.81:0/2972211695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa6c15c0f0 0x7ffa6c06a8a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009e50 tx=0x7ffa6400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f335d15dd7b56ac8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:48.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa71210700 1 -- 172.21.15.81:0/2972211695 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6400e070 con 0x7ffa6c15c0f0 2024-05-08T23:30:48.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa71210700 1 -- 172.21.15.81:0/2972211695 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6400b910 con 0x7ffa6c15c0f0 2024-05-08T23:30:48.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa71210700 1 -- 172.21.15.81:0/2972211695 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6400bcc0 con 0x7ffa6c15c0f0 2024-05-08T23:30:48.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa73214700 1 -- 172.21.15.81:0/2972211695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa6c15c0f0 msgr2=0x7ffa6c06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa73214700 1 --2- 172.21.15.81:0/2972211695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa6c15c0f0 0x7ffa6c06a8a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009e50 tx=0x7ffa6400b330 comp rx=0 tx=0).stop 2024-05-08T23:30:48.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa73214700 1 -- 172.21.15.81:0/2972211695 shutdown_connections 2024-05-08T23:30:48.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa73214700 1 --2- 172.21.15.81:0/2972211695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa6c15c0f0 0x7ffa6c06a8a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa73214700 1 --2- 172.21.15.81:0/2972211695 >> v2:172.21.15.117:3300/0 conn(0x7ffa6c071dd0 0x7ffa6c06a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa73214700 1 -- 172.21.15.81:0/2972211695 >> 172.21.15.81:0/2972211695 conn(0x7ffa6c06b470 msgr2=0x7ffa6c06b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:48.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa73214700 1 -- 172.21.15.81:0/2972211695 shutdown_connections 2024-05-08T23:30:48.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.457+0000 7ffa73214700 1 -- 172.21.15.81:0/2972211695 wait complete. 2024-05-08T23:30:48.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.461+0000 7ffa73214700 1 Processor -- start 2024-05-08T23:30:48.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.461+0000 7ffa73214700 1 -- start start 2024-05-08T23:30:48.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.461+0000 7ffa73214700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa6c071dd0 0x7ffa6c09d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.461+0000 7ffa73214700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa6c158760 0x7ffa6c09a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.461+0000 7ffa73214700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa6c15c0f0 0x7ffa6c152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.461+0000 7ffa73214700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c152e60 con 0x7ffa6c158760 2024-05-08T23:30:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.461+0000 7ffa73214700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c15c530 con 0x7ffa6c071dd0 2024-05-08T23:30:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.461+0000 7ffa73214700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c15c780 con 0x7ffa6c15c0f0 2024-05-08T23:30:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.461+0000 7ffa71a11700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa6c158760 0x7ffa6c09a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.461+0000 7ffa71a11700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa6c158760 0x7ffa6c09a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51236/0 (socket says 172.21.15.81:51236) 2024-05-08T23:30:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.461+0000 7ffa71a11700 1 -- 172.21.15.81:0/2358454594 learned_addr learned my addr 172.21.15.81:0/2358454594 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.461+0000 7ffa72a13700 1 --2- 172.21.15.81:0/2358454594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa6c15c0f0 0x7ffa6c152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.461+0000 7ffa72212700 1 --2- 172.21.15.81:0/2358454594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa6c071dd0 0x7ffa6c09d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.465+0000 7ffa71a11700 1 -- 172.21.15.81:0/2358454594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa6c15c0f0 msgr2=0x7ffa6c152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.465+0000 7ffa71a11700 1 --2- 172.21.15.81:0/2358454594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa6c15c0f0 0x7ffa6c152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.465+0000 7ffa71a11700 1 -- 172.21.15.81:0/2358454594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa6c071dd0 msgr2=0x7ffa6c09d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.465+0000 7ffa71a11700 1 --2- 172.21.15.81:0/2358454594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa6c071dd0 0x7ffa6c09d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.465+0000 7ffa71a11700 1 -- 172.21.15.81:0/2358454594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa64009b30 con 0x7ffa6c158760 2024-05-08T23:30:48.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.465+0000 7ffa71a11700 1 --2- 172.21.15.81:0/2358454594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa6c158760 0x7ffa6c09a730 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00e9d0 tx=0x7ffa5c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:48.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.465+0000 7ffa62ffd700 1 -- 172.21.15.81:0/2358454594 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c009720 con 0x7ffa6c158760 2024-05-08T23:30:48.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.465+0000 7ffa62ffd700 1 -- 172.21.15.81:0/2358454594 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c00ee70 con 0x7ffa6c158760 2024-05-08T23:30:48.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.465+0000 7ffa73214700 1 -- 172.21.15.81:0/2358454594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6c15ca60 con 0x7ffa6c158760 2024-05-08T23:30:48.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.465+0000 7ffa62ffd700 1 -- 172.21.15.81:0/2358454594 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c00ca50 con 0x7ffa6c158760 2024-05-08T23:30:48.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.465+0000 7ffa73214700 1 -- 172.21.15.81:0/2358454594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6c15d020 con 0x7ffa6c158760 2024-05-08T23:30:48.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.465+0000 7ffa73214700 1 -- 172.21.15.81:0/2358454594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6c0672a0 con 0x7ffa6c158760 2024-05-08T23:30:48.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.469+0000 7ffa62ffd700 1 -- 172.21.15.81:0/2358454594 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffa5c00cbb0 con 0x7ffa6c158760 2024-05-08T23:30:48.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.469+0000 7ffa62ffd700 1 --2- 172.21.15.81:0/2358454594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa50066730 0x7ffa50068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.469+0000 7ffa62ffd700 1 -- 172.21.15.81:0/2358454594 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa5c084940 con 0x7ffa6c158760 2024-05-08T23:30:48.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.469+0000 7ffa72212700 1 --2- 172.21.15.81:0/2358454594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa50066730 0x7ffa50068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.469+0000 7ffa72212700 1 --2- 172.21.15.81:0/2358454594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa50066730 0x7ffa50068bf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0b22c0 tx=0x7ffa68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:48.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.477+0000 7ffa62ffd700 1 -- 172.21.15.81:0/2358454594 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffa5c051cb0 con 0x7ffa6c158760 2024-05-08T23:30:48.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.641+0000 7ffa73214700 1 -- 172.21.15.81:0/2358454594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffa6c15bba0 con 0x7ffa6c158760 2024-05-08T23:30:48.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.641+0000 7ffa62ffd700 1 -- 172.21.15.81:0/2358454594 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffa5c0170a0 con 0x7ffa6c158760 2024-05-08T23:30:48.648 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:30:48.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.645+0000 7ffa73214700 1 -- 172.21.15.81:0/2358454594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa50066730 msgr2=0x7ffa50068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.645+0000 7ffa73214700 1 --2- 172.21.15.81:0/2358454594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa50066730 0x7ffa50068bf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0b22c0 tx=0x7ffa68009040 comp rx=0 tx=0).stop 2024-05-08T23:30:48.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.645+0000 7ffa73214700 1 -- 172.21.15.81:0/2358454594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa6c158760 msgr2=0x7ffa6c09a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.645+0000 7ffa73214700 1 --2- 172.21.15.81:0/2358454594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa6c158760 0x7ffa6c09a730 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00e9d0 tx=0x7ffa5c00ece0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.645+0000 7ffa73214700 1 -- 172.21.15.81:0/2358454594 shutdown_connections 2024-05-08T23:30:48.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.645+0000 7ffa73214700 1 --2- 172.21.15.81:0/2358454594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa50066730 0x7ffa50068bf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.645+0000 7ffa73214700 1 --2- 172.21.15.81:0/2358454594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa6c15c0f0 0x7ffa6c152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.645+0000 7ffa73214700 1 --2- 172.21.15.81:0/2358454594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa6c158760 0x7ffa6c09a730 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.651 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.645+0000 7ffa73214700 1 --2- 172.21.15.81:0/2358454594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa6c071dd0 0x7ffa6c09d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.651 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.645+0000 7ffa73214700 1 -- 172.21.15.81:0/2358454594 >> 172.21.15.81:0/2358454594 conn(0x7ffa6c06b470 msgr2=0x7ffa6c06b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:48.651 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.645+0000 7ffa73214700 1 -- 172.21.15.81:0/2358454594 shutdown_connections 2024-05-08T23:30:48.651 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.645+0000 7ffa73214700 1 -- 172.21.15.81:0/2358454594 wait complete. 2024-05-08T23:30:48.662 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:30:48.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.769+0000 7fdb44e73700 1 Processor -- start 2024-05-08T23:30:48.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.769+0000 7fdb44e73700 1 -- start start 2024-05-08T23:30:48.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.769+0000 7fdb44e73700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400ce990 0x7fdb4015f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.769+0000 7fdb44e73700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdb4015f770 0x7fdb400b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.769+0000 7fdb44e73700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4015eda0 con 0x7fdb4015afa0 2024-05-08T23:30:48.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.769+0000 7fdb44e73700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb4015ea70 con 0x7fdb4015f770 2024-05-08T23:30:48.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.769+0000 7fdb44e73700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4015ba90 con 0x7fdb400ce990 2024-05-08T23:30:48.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb3f7fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fdb4015afa0 0x7fdb4015b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38754/0 (socket says 172.21.15.81:38754) 2024-05-08T23:30:48.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb3f7fe700 1 -- 172.21.15.81:0/3992829623 learned_addr learned my addr 172.21.15.81:0/3992829623 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:48.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb3ffff700 1 --2- 172.21.15.81:0/3992829623 >> v2:172.21.15.117:3300/0 conn(0x7fdb4015f770 0x7fdb400b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb3effd700 1 --2- 172.21.15.81:0/3992829623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400ce990 0x7fdb4015f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb3e7fc700 1 -- 172.21.15.81:0/3992829623 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380586926 0 0) 0x7fdb4015eda0 con 0x7fdb4015afa0 2024-05-08T23:30:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb3e7fc700 1 -- 172.21.15.81:0/3992829623 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb2c0035d0 con 0x7fdb4015afa0 2024-05-08T23:30:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb3ffff700 1 -- 172.21.15.81:0/3992829623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400ce990 msgr2=0x7fdb4015f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb3ffff700 1 --2- 172.21.15.81:0/3992829623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400ce990 0x7fdb4015f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb3ffff700 1 -- 172.21.15.81:0/3992829623 >> v1:172.21.15.81:6789/0 conn(0x7fdb4015afa0 legacy=0x7fdb4015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb3ffff700 1 -- 172.21.15.81:0/3992829623 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4015bd40 con 0x7fdb4015f770 2024-05-08T23:30:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb3ffff700 1 --2- 172.21.15.81:0/3992829623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4015f770 0x7fdb400b4270 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009850 tx=0x7fdb38009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e73fc72e7b6959a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb3e7fc700 1 -- 172.21.15.81:0/3992829623 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38011830 con 0x7fdb4015f770 2024-05-08T23:30:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb3e7fc700 1 -- 172.21.15.81:0/3992829623 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb38011990 con 0x7fdb4015f770 2024-05-08T23:30:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb3e7fc700 1 -- 172.21.15.81:0/3992829623 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38011d40 con 0x7fdb4015f770 2024-05-08T23:30:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb44e73700 1 -- 172.21.15.81:0/3992829623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4015f770 msgr2=0x7fdb400b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb44e73700 1 --2- 172.21.15.81:0/3992829623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4015f770 0x7fdb400b4270 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009850 tx=0x7fdb38009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb44e73700 1 -- 172.21.15.81:0/3992829623 shutdown_connections 2024-05-08T23:30:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb44e73700 1 --2- 172.21.15.81:0/3992829623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4015f770 0x7fdb400b4270 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb44e73700 1 --2- 172.21.15.81:0/3992829623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400ce990 0x7fdb4015f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb44e73700 1 -- 172.21.15.81:0/3992829623 >> 172.21.15.81:0/3992829623 conn(0x7fdb4006c330 msgr2=0x7fdb4006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb44e73700 1 -- 172.21.15.81:0/3992829623 shutdown_connections 2024-05-08T23:30:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb44e73700 1 -- 172.21.15.81:0/3992829623 wait complete. 2024-05-08T23:30:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb44e73700 1 Processor -- start 2024-05-08T23:30:48.780 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.773+0000 7fdb44e73700 1 -- start start 2024-05-08T23:30:48.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb44e73700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb400ce990 0x7fdb4014b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb44e73700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4015afa0 0x7fdb4015db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb44e73700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4015f770 0x7fdb4015e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb44e73700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4014e0c0 con 0x7fdb400ce990 2024-05-08T23:30:48.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb44e73700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4014e3a0 con 0x7fdb4015f770 2024-05-08T23:30:48.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb44e73700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4014e680 con 0x7fdb4015afa0 2024-05-08T23:30:48.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb3f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb400ce990 0x7fdb4014b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb3f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb400ce990 0x7fdb4014b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51242/0 (socket says 172.21.15.81:51242) 2024-05-08T23:30:48.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb3f7fe700 1 -- 172.21.15.81:0/382925422 learned_addr learned my addr 172.21.15.81:0/382925422 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:48.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb3ffff700 1 --2- 172.21.15.81:0/382925422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4015f770 0x7fdb4015e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb3effd700 1 --2- 172.21.15.81:0/382925422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4015afa0 0x7fdb4015db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb3f7fe700 1 -- 172.21.15.81:0/382925422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4015afa0 msgr2=0x7fdb4015db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb3f7fe700 1 --2- 172.21.15.81:0/382925422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4015afa0 0x7fdb4015db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb3f7fe700 1 -- 172.21.15.81:0/382925422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4015f770 msgr2=0x7fdb4015e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:48.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb3f7fe700 1 --2- 172.21.15.81:0/382925422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4015f770 0x7fdb4015e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:48.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb3f7fe700 1 -- 172.21.15.81:0/382925422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb380094b0 con 0x7fdb400ce990 2024-05-08T23:30:48.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb3f7fe700 1 --2- 172.21.15.81:0/382925422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb400ce990 0x7fdb4014b8d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000c700 tx=0x7fdb3000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:48.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb37fff700 1 -- 172.21.15.81:0/382925422 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30005d90 con 0x7fdb400ce990 2024-05-08T23:30:48.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb37fff700 1 -- 172.21.15.81:0/382925422 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb300070d0 con 0x7fdb400ce990 2024-05-08T23:30:48.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb44e73700 1 -- 172.21.15.81:0/382925422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4014e960 con 0x7fdb400ce990 2024-05-08T23:30:48.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb37fff700 1 -- 172.21.15.81:0/382925422 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30007480 con 0x7fdb400ce990 2024-05-08T23:30:48.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.777+0000 7fdb44e73700 1 -- 172.21.15.81:0/382925422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb40159b90 con 0x7fdb400ce990 2024-05-08T23:30:48.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.781+0000 7fdb44e73700 1 -- 172.21.15.81:0/382925422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4014ee80 con 0x7fdb400ce990 2024-05-08T23:30:48.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.781+0000 7fdb37fff700 1 -- 172.21.15.81:0/382925422 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdb300075e0 con 0x7fdb400ce990 2024-05-08T23:30:48.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.781+0000 7fdb37fff700 1 --2- 172.21.15.81:0/382925422 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdb20066730 0x7fdb20068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:48.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.781+0000 7fdb37fff700 1 -- 172.21.15.81:0/382925422 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb30083690 con 0x7fdb400ce990 2024-05-08T23:30:48.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.781+0000 7fdb3effd700 1 --2- 172.21.15.81:0/382925422 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdb20066730 0x7fdb20068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:48.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.781+0000 7fdb3effd700 1 --2- 172.21.15.81:0/382925422 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdb20066730 0x7fdb20068bf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdb400cb960 tx=0x7fdb2800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:48.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.789+0000 7fdb37fff700 1 -- 172.21.15.81:0/382925422 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdb4014ee80 con 0x7fdb400ce990 2024-05-08T23:30:49.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.997+0000 7fdb44e73700 1 -- 172.21.15.81:0/382925422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb4014ee80 con 0x7fdb400ce990 2024-05-08T23:30:49.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.997+0000 7fdb37fff700 1 -- 172.21.15.81:0/382925422 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7fdb4014ee80 con 0x7fdb400ce990 2024-05-08T23:30:49.001 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:49.001 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1475726755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1475726755},{"type":"v1","addr":"172.21.15.81:6835","nonce":1475726755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3535820540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3535820540},{"type":"v1","addr":"172.21.15.81:6837","nonce":3535820540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1531019955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1531019955},{"type":"v1","addr":"172.21.15.117:6833","nonce":1531019955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:30:38.665368+0000","modified":"2024-05-08T23:30:41.672722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/491911303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":491911303},{"type":"v1","addr":"172.21.15.117:6835","nonce":491911303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:30:49.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.997+0000 7fdb44e73700 1 -- 172.21.15.81:0/382925422 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdb20066730 msgr2=0x7fdb20068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.997+0000 7fdb44e73700 1 --2- 172.21.15.81:0/382925422 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdb20066730 0x7fdb20068bf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdb400cb960 tx=0x7fdb2800b040 comp rx=0 tx=0).stop 2024-05-08T23:30:49.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.997+0000 7fdb44e73700 1 -- 172.21.15.81:0/382925422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb400ce990 msgr2=0x7fdb4014b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.997+0000 7fdb44e73700 1 --2- 172.21.15.81:0/382925422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb400ce990 0x7fdb4014b8d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000c700 tx=0x7fdb3000ca10 comp rx=0 tx=0).stop 2024-05-08T23:30:49.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.997+0000 7fdb44e73700 1 -- 172.21.15.81:0/382925422 shutdown_connections 2024-05-08T23:30:49.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.997+0000 7fdb44e73700 1 --2- 172.21.15.81:0/382925422 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdb20066730 0x7fdb20068bf0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.997+0000 7fdb44e73700 1 --2- 172.21.15.81:0/382925422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4015f770 0x7fdb4015e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.997+0000 7fdb44e73700 1 --2- 172.21.15.81:0/382925422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4015afa0 0x7fdb4015db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.997+0000 7fdb44e73700 1 --2- 172.21.15.81:0/382925422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb400ce990 0x7fdb4014b8d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.997+0000 7fdb44e73700 1 -- 172.21.15.81:0/382925422 >> 172.21.15.81:0/382925422 conn(0x7fdb4006c330 msgr2=0x7fdb400cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:49.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.997+0000 7fdb44e73700 1 -- 172.21.15.81:0/382925422 shutdown_connections 2024-05-08T23:30:49.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:48.997+0000 7fdb44e73700 1 -- 172.21.15.81:0/382925422 wait complete. 2024-05-08T23:30:49.005 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 7 2024-05-08T23:30:49.016 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:30:49.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.121+0000 7f0670db6700 1 Processor -- start 2024-05-08T23:30:49.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.121+0000 7f0670db6700 1 -- start start 2024-05-08T23:30:49.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.121+0000 7f0670db6700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f066c0d37c0 0x7f066c0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.121+0000 7f0670db6700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066c0d3c20 con 0x7f066c0d1690 2024-05-08T23:30:49.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.121+0000 7f0670db6700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066c0d1460 con 0x7f066c0b2ea0 2024-05-08T23:30:49.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.121+0000 7f0670db6700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066c0d3e50 con 0x7f066c0d37c0 2024-05-08T23:30:49.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.121+0000 7f066b7fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f066c0d1690 0x7f066c0d1a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38764/0 (socket says 172.21.15.81:38764) 2024-05-08T23:30:49.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.121+0000 7f066b7fe700 1 -- 172.21.15.81:0/1887229124 learned_addr learned my addr 172.21.15.81:0/1887229124 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:49.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.121+0000 7f066affd700 1 --2- 172.21.15.81:0/1887229124 >> v2:172.21.15.117:3300/0 conn(0x7f066c0d37c0 0x7f066c0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.121+0000 7f066a7fc700 1 -- 172.21.15.81:0/1887229124 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075109703 0 0) 0x7f066c0d3c20 con 0x7f066c0d1690 2024-05-08T23:30:49.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.125+0000 7f066a7fc700 1 -- 172.21.15.81:0/1887229124 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06580035d0 con 0x7f066c0d1690 2024-05-08T23:30:49.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.125+0000 7f066affd700 1 -- 172.21.15.81:0/1887229124 >> v1:172.21.15.117:6789/0 conn(0x7f066c0b2ea0 legacy=0x7f066c0b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.125+0000 7f066affd700 1 -- 172.21.15.81:0/1887229124 >> v1:172.21.15.81:6789/0 conn(0x7f066c0d1690 legacy=0x7f066c0d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.125+0000 7f066affd700 1 -- 172.21.15.81:0/1887229124 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c0ce870 con 0x7f066c0d37c0 2024-05-08T23:30:49.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.125+0000 7f066affd700 1 --2- 172.21.15.81:0/1887229124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f066c0d37c0 0x7f066c0d0f20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f065c009d90 tx=0x7f065c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5a2092feb8cff60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:49.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.125+0000 7f066a7fc700 1 -- 172.21.15.81:0/1887229124 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c00e070 con 0x7f066c0d37c0 2024-05-08T23:30:49.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.125+0000 7f066a7fc700 1 -- 172.21.15.81:0/1887229124 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c00bb20 con 0x7f066c0d37c0 2024-05-08T23:30:49.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.125+0000 7f066a7fc700 1 -- 172.21.15.81:0/1887229124 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c00be90 con 0x7f066c0d37c0 2024-05-08T23:30:49.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.125+0000 7f0670db6700 1 -- 172.21.15.81:0/1887229124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f066c0d37c0 msgr2=0x7f066c0d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.125+0000 7f0670db6700 1 --2- 172.21.15.81:0/1887229124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f066c0d37c0 0x7f066c0d0f20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f065c009d90 tx=0x7f065c00b330 comp rx=0 tx=0).stop 2024-05-08T23:30:49.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.125+0000 7f0670db6700 1 -- 172.21.15.81:0/1887229124 shutdown_connections 2024-05-08T23:30:49.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.125+0000 7f0670db6700 1 --2- 172.21.15.81:0/1887229124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f066c0d37c0 0x7f066c0d0f20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.125+0000 7f0670db6700 1 -- 172.21.15.81:0/1887229124 >> 172.21.15.81:0/1887229124 conn(0x7f066c09c3c0 msgr2=0x7f066c0d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:49.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.125+0000 7f0670db6700 1 -- 172.21.15.81:0/1887229124 shutdown_connections 2024-05-08T23:30:49.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.125+0000 7f0670db6700 1 -- 172.21.15.81:0/1887229124 wait complete. 2024-05-08T23:30:49.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f0670db6700 1 Processor -- start 2024-05-08T23:30:49.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f0670db6700 1 -- start start 2024-05-08T23:30:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f0670db6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f066c0b2ea0 0x7f066c15f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f0670db6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f066c0d1690 0x7f066c15b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f0670db6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f066c0d37c0 0x7f066c15bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f0670db6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c15fd10 con 0x7f066c0b2ea0 2024-05-08T23:30:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f0670db6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c0b6e50 con 0x7f066c0d37c0 2024-05-08T23:30:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f0670db6700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066c0b7130 con 0x7f066c0d1690 2024-05-08T23:30:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f066b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f066c0b2ea0 0x7f066c15f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f066b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f066c0b2ea0 0x7f066c15f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51246/0 (socket says 172.21.15.81:51246) 2024-05-08T23:30:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f066bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f066c0d37c0 0x7f066c15bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f066b7fe700 1 -- 172.21.15.81:0/4212912990 learned_addr learned my addr 172.21.15.81:0/4212912990 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f066affd700 1 --2- 172.21.15.81:0/4212912990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f066c0d1690 0x7f066c15b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f066b7fe700 1 -- 172.21.15.81:0/4212912990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f066c0d1690 msgr2=0x7f066c15b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f066b7fe700 1 --2- 172.21.15.81:0/4212912990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f066c0d1690 0x7f066c15b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f066b7fe700 1 -- 172.21.15.81:0/4212912990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f066c0d37c0 msgr2=0x7f066c15bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f066b7fe700 1 --2- 172.21.15.81:0/4212912990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f066c0d37c0 0x7f066c15bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f066b7fe700 1 -- 172.21.15.81:0/4212912990 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c009a70 con 0x7f066c0b2ea0 2024-05-08T23:30:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f066b7fe700 1 --2- 172.21.15.81:0/4212912990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f066c0b2ea0 0x7f066c15f350 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f065400c4d0 tx=0x7f065400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f0663fff700 1 -- 172.21.15.81:0/4212912990 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654005d90 con 0x7f066c0b2ea0 2024-05-08T23:30:49.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f0663fff700 1 -- 172.21.15.81:0/4212912990 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06540070d0 con 0x7f066c0b2ea0 2024-05-08T23:30:49.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f0670db6700 1 -- 172.21.15.81:0/4212912990 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066c0b7410 con 0x7f066c0b2ea0 2024-05-08T23:30:49.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f0663fff700 1 -- 172.21.15.81:0/4212912990 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654007480 con 0x7f066c0b2ea0 2024-05-08T23:30:49.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.129+0000 7f0670db6700 1 -- 172.21.15.81:0/4212912990 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066c0b7c10 con 0x7f066c0b2ea0 2024-05-08T23:30:49.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.133+0000 7f0670db6700 1 -- 172.21.15.81:0/4212912990 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066c0b7e00 con 0x7f066c0b2ea0 2024-05-08T23:30:49.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.133+0000 7f0663fff700 1 -- 172.21.15.81:0/4212912990 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f06540075e0 con 0x7f066c0b2ea0 2024-05-08T23:30:49.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.133+0000 7f0663fff700 1 --2- 172.21.15.81:0/4212912990 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f064c066730 0x7f064c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.133+0000 7f066affd700 1 --2- 172.21.15.81:0/4212912990 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f064c066730 0x7f064c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.133+0000 7f0663fff700 1 -- 172.21.15.81:0/4212912990 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0654083590 con 0x7f066c0b2ea0 2024-05-08T23:30:49.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.133+0000 7f066affd700 1 --2- 172.21.15.81:0/4212912990 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f064c066730 0x7f064c068bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f065c00b370 tx=0x7f065c011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:49.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.141+0000 7f0663fff700 1 -- 172.21.15.81:0/4212912990 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f06540508f0 con 0x7f066c0b2ea0 2024-05-08T23:30:49.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.313+0000 7f0670db6700 1 -- 172.21.15.81:0/4212912990 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f066c0684b0 con 0x7f066c0b2ea0 2024-05-08T23:30:49.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.317+0000 7f0663fff700 1 -- 172.21.15.81:0/4212912990 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f0654053f10 con 0x7f066c0b2ea0 2024-05-08T23:30:49.322 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:49.322 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:30:49.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.321+0000 7f0670db6700 1 -- 172.21.15.81:0/4212912990 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f064c066730 msgr2=0x7f064c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.321+0000 7f0670db6700 1 --2- 172.21.15.81:0/4212912990 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f064c066730 0x7f064c068bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f065c00b370 tx=0x7f065c011000 comp rx=0 tx=0).stop 2024-05-08T23:30:49.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.321+0000 7f0670db6700 1 -- 172.21.15.81:0/4212912990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f066c0b2ea0 msgr2=0x7f066c15f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.321+0000 7f0670db6700 1 --2- 172.21.15.81:0/4212912990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f066c0b2ea0 0x7f066c15f350 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f065400c4d0 tx=0x7f065400c7e0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.321+0000 7f0670db6700 1 -- 172.21.15.81:0/4212912990 shutdown_connections 2024-05-08T23:30:49.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.321+0000 7f0670db6700 1 --2- 172.21.15.81:0/4212912990 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f064c066730 0x7f064c068bf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.321+0000 7f0670db6700 1 --2- 172.21.15.81:0/4212912990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f066c0d37c0 0x7f066c15bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.321+0000 7f0670db6700 1 --2- 172.21.15.81:0/4212912990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f066c0d1690 0x7f066c15b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.321+0000 7f0670db6700 1 --2- 172.21.15.81:0/4212912990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f066c0b2ea0 0x7f066c15f350 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.321+0000 7f0670db6700 1 -- 172.21.15.81:0/4212912990 >> 172.21.15.81:0/4212912990 conn(0x7f066c09c3c0 msgr2=0x7f066c06b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:49.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.321+0000 7f0670db6700 1 -- 172.21.15.81:0/4212912990 shutdown_connections 2024-05-08T23:30:49.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.321+0000 7f0670db6700 1 -- 172.21.15.81:0/4212912990 wait complete. 2024-05-08T23:30:49.338 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:30:49.339 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:30:49.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a9174700 1 Processor -- start 2024-05-08T23:30:49.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a9174700 1 -- start start 2024-05-08T23:30:49.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a9174700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8a40d1da0 0x7fc8a40cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a9174700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a40d3d40 con 0x7fc8a40cfe80 2024-05-08T23:30:49.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a9174700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a40d0d00 con 0x7fc8a4160920 2024-05-08T23:30:49.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a9174700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a40d0310 con 0x7fc8a40d1da0 2024-05-08T23:30:49.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a8973700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc8a40cfe80 0x7fc8a40b5e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38768/0 (socket says 172.21.15.81:38768) 2024-05-08T23:30:49.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a8973700 1 -- 172.21.15.81:0/3263000502 learned_addr learned my addr 172.21.15.81:0/3263000502 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:49.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc89b7fe700 1 --2- 172.21.15.81:0/3263000502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8a40d1da0 0x7fc8a40cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a37fe700 1 -- 172.21.15.81:0/3263000502 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1603809571 0 0) 0x7fc8a40d3d40 con 0x7fc8a40cfe80 2024-05-08T23:30:49.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a37fe700 1 -- 172.21.15.81:0/3263000502 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8900035d0 con 0x7fc8a40cfe80 2024-05-08T23:30:49.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a37fe700 1 -- 172.21.15.81:0/3263000502 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1902058520 0 0) 0x7fc8900035d0 con 0x7fc8a40cfe80 2024-05-08T23:30:49.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a37fe700 1 -- 172.21.15.81:0/3263000502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8a40d1da0 msgr2=0x7fc8a40cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:49.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a37fe700 1 --2- 172.21.15.81:0/3263000502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8a40d1da0 0x7fc8a40cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a37fe700 1 -- 172.21.15.81:0/3263000502 >> v1:172.21.15.117:6789/0 conn(0x7fc8a4160920 legacy=0x7fc8a40ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a37fe700 1 -- 172.21.15.81:0/3263000502 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a40d0fe0 con 0x7fc8a40cfe80 2024-05-08T23:30:49.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a37fe700 1 -- 172.21.15.81:0/3263000502 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7fc89c0019f0 con 0x7fc8a40cfe80 2024-05-08T23:30:49.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.461+0000 7fc8a37fe700 1 -- 172.21.15.81:0/3263000502 >> v1:172.21.15.81:6789/0 conn(0x7fc8a40cfe80 legacy=0x7fc8a40b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a37fe700 1 --2- 172.21.15.81:0/3263000502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc890003a50 0x7fc890003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a37fe700 1 --2- 172.21.15.81:0/3263000502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8a40d1da0 0x7fc890004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a37fe700 1 --2- 172.21.15.81:0/3263000502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc890004db0 0x7fc890005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a37fe700 1 -- 172.21.15.81:0/3263000502 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a40d0310 con 0x7fc890004db0 2024-05-08T23:30:49.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a37fe700 1 -- 172.21.15.81:0/3263000502 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89000bb60 con 0x7fc890003a50 2024-05-08T23:30:49.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a37fe700 1 -- 172.21.15.81:0/3263000502 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89000bd20 con 0x7fc8a40d1da0 2024-05-08T23:30:49.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a8973700 1 --2- 172.21.15.81:0/3263000502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc890004db0 0x7fc890005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc89b7fe700 1 --2- 172.21.15.81:0/3263000502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8a40d1da0 0x7fc890004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a3fff700 1 --2- 172.21.15.81:0/3263000502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc890003a50 0x7fc890003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a8973700 1 -- 172.21.15.81:0/3263000502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8a40d1da0 msgr2=0x7fc890004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:49.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a8973700 1 --2- 172.21.15.81:0/3263000502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8a40d1da0 0x7fc890004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a8973700 1 -- 172.21.15.81:0/3263000502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc890003a50 msgr2=0x7fc890003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:49.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a8973700 1 --2- 172.21.15.81:0/3263000502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc890003a50 0x7fc890003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a8973700 1 -- 172.21.15.81:0/3263000502 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8900035d0 con 0x7fc890004db0 2024-05-08T23:30:49.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a8973700 1 --2- 172.21.15.81:0/3263000502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc890004db0 0x7fc890005230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00a830 tx=0x7fc8900058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32da1a45ce7e21b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:49.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a37fe700 1 -- 172.21.15.81:0/3263000502 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c00a2f0 con 0x7fc890004db0 2024-05-08T23:30:49.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a37fe700 1 -- 172.21.15.81:0/3263000502 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89c0072f0 con 0x7fc890004db0 2024-05-08T23:30:49.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a37fe700 1 -- 172.21.15.81:0/3263000502 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c0076a0 con 0x7fc890004db0 2024-05-08T23:30:49.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a9174700 1 -- 172.21.15.81:0/3263000502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc890004db0 msgr2=0x7fc890005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a9174700 1 --2- 172.21.15.81:0/3263000502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc890004db0 0x7fc890005230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00a830 tx=0x7fc8900058a0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a8973700 1 -- 172.21.15.81:0/3263000502 reap_dead start 2024-05-08T23:30:49.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a9174700 1 -- 172.21.15.81:0/3263000502 shutdown_connections 2024-05-08T23:30:49.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a9174700 1 -- 172.21.15.81:0/3263000502 >> 172.21.15.81:0/3263000502 conn(0x7fc8a409c340 msgr2=0x7fc8a40d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:49.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a9174700 1 -- 172.21.15.81:0/3263000502 shutdown_connections 2024-05-08T23:30:49.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.465+0000 7fc8a9174700 1 -- 172.21.15.81:0/3263000502 wait complete. 2024-05-08T23:30:49.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a9174700 1 Processor -- start 2024-05-08T23:30:49.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a9174700 1 -- start start 2024-05-08T23:30:49.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a9174700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc890004db0 0x7fc8a415d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a9174700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8a415e770 0x7fc8a4158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a9174700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8a41f6ba0 0x7fc8a41f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a9174700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a4159090 con 0x7fc8a415e770 2024-05-08T23:30:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a9174700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a415d6d0 con 0x7fc890004db0 2024-05-08T23:30:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a9174700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a415d9b0 con 0x7fc8a41f6ba0 2024-05-08T23:30:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc89b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8a415e770 0x7fc8a4158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc89b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8a415e770 0x7fc8a4158b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51264/0 (socket says 172.21.15.81:51264) 2024-05-08T23:30:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc89b7fe700 1 -- 172.21.15.81:0/3859863443 learned_addr learned my addr 172.21.15.81:0/3859863443 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a3fff700 1 --2- 172.21.15.81:0/3859863443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc890004db0 0x7fc8a415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a8973700 1 --2- 172.21.15.81:0/3859863443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8a41f6ba0 0x7fc8a41f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc89b7fe700 1 -- 172.21.15.81:0/3859863443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8a41f6ba0 msgr2=0x7fc8a41f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc89b7fe700 1 --2- 172.21.15.81:0/3859863443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8a41f6ba0 0x7fc8a41f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc89b7fe700 1 -- 172.21.15.81:0/3859863443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc890004db0 msgr2=0x7fc8a415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc89b7fe700 1 --2- 172.21.15.81:0/3859863443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc890004db0 0x7fc8a415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc89b7fe700 1 -- 172.21.15.81:0/3859863443 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a415dc90 con 0x7fc8a415e770 2024-05-08T23:30:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a3fff700 1 --2- 172.21.15.81:0/3859863443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc890004db0 0x7fc8a415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:49.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc89b7fe700 1 --2- 172.21.15.81:0/3859863443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8a415e770 0x7fc8a4158b50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc88c011a20 tx=0x7fc88c011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:49.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a17fa700 1 -- 172.21.15.81:0/3859863443 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c00fa00 con 0x7fc8a415e770 2024-05-08T23:30:49.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a17fa700 1 -- 172.21.15.81:0/3859863443 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c00fb60 con 0x7fc8a415e770 2024-05-08T23:30:49.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a9174700 1 -- 172.21.15.81:0/3859863443 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a415df70 con 0x7fc8a415e770 2024-05-08T23:30:49.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a17fa700 1 -- 172.21.15.81:0/3859863443 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c014440 con 0x7fc8a415e770 2024-05-08T23:30:49.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.469+0000 7fc8a9174700 1 -- 172.21.15.81:0/3859863443 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a41f7550 con 0x7fc8a415e770 2024-05-08T23:30:49.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.473+0000 7fc8a9174700 1 -- 172.21.15.81:0/3859863443 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a40684b0 con 0x7fc8a415e770 2024-05-08T23:30:49.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.473+0000 7fc8a17fa700 1 -- 172.21.15.81:0/3859863443 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc88c01d400 con 0x7fc8a415e770 2024-05-08T23:30:49.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.473+0000 7fc8a17fa700 1 --2- 172.21.15.81:0/3859863443 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc884066760 0x7fc884068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.473+0000 7fc8a17fa700 1 -- 172.21.15.81:0/3859863443 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc88c084970 con 0x7fc8a415e770 2024-05-08T23:30:49.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.473+0000 7fc8a3fff700 1 --2- 172.21.15.81:0/3859863443 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc884066760 0x7fc884068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.473+0000 7fc8a3fff700 1 --2- 172.21.15.81:0/3859863443 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc884066760 0x7fc884068c20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc894007fd0 tx=0x7fc894009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:49.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.481+0000 7fc8a17fa700 1 -- 172.21.15.81:0/3859863443 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc88c051d10 con 0x7fc8a415e770 2024-05-08T23:30:49.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.649+0000 7fc8a9174700 1 -- 172.21.15.81:0/3859863443 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8a40cc690 con 0x7fc8a415e770 2024-05-08T23:30:49.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.649+0000 7fc8a17fa700 1 -- 172.21.15.81:0/3859863443 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc88c055330 con 0x7fc8a415e770 2024-05-08T23:30:49.654 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:30:49.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.653+0000 7fc8a9174700 1 -- 172.21.15.81:0/3859863443 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc884066760 msgr2=0x7fc884068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.653+0000 7fc8a9174700 1 --2- 172.21.15.81:0/3859863443 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc884066760 0x7fc884068c20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc894007fd0 tx=0x7fc894009040 comp rx=0 tx=0).stop 2024-05-08T23:30:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.653+0000 7fc8a9174700 1 -- 172.21.15.81:0/3859863443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8a415e770 msgr2=0x7fc8a4158b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.653+0000 7fc8a9174700 1 --2- 172.21.15.81:0/3859863443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8a415e770 0x7fc8a4158b50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc88c011a20 tx=0x7fc88c011de0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.653+0000 7fc8a9174700 1 -- 172.21.15.81:0/3859863443 shutdown_connections 2024-05-08T23:30:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.653+0000 7fc8a9174700 1 --2- 172.21.15.81:0/3859863443 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc884066760 0x7fc884068c20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.653+0000 7fc8a9174700 1 --2- 172.21.15.81:0/3859863443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8a41f6ba0 0x7fc8a41f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.653+0000 7fc8a9174700 1 --2- 172.21.15.81:0/3859863443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8a415e770 0x7fc8a4158b50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.653+0000 7fc8a9174700 1 --2- 172.21.15.81:0/3859863443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc890004db0 0x7fc8a415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.653+0000 7fc8a9174700 1 -- 172.21.15.81:0/3859863443 >> 172.21.15.81:0/3859863443 conn(0x7fc8a409c340 msgr2=0x7fc8a40cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.653+0000 7fc8a9174700 1 -- 172.21.15.81:0/3859863443 shutdown_connections 2024-05-08T23:30:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.653+0000 7fc8a9174700 1 -- 172.21.15.81:0/3859863443 wait complete. 2024-05-08T23:30:49.669 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:30:49.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.781+0000 7f4500980700 1 Processor -- start 2024-05-08T23:30:49.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.781+0000 7f4500980700 1 -- start start 2024-05-08T23:30:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f4500980700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44f403ebd0 0x7f44f4058950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f4500980700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f44f4007280 0x7f44f4007710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f4500980700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f40578b0 con 0x7f44f40e33a0 2024-05-08T23:30:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f4500980700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f40576c0 con 0x7f44f4007280 2024-05-08T23:30:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f4500980700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4007c50 con 0x7f44f403ebd0 2024-05-08T23:30:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f44fb59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f44f4007280 0x7f44f4007710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f44fb59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f44f4007280 0x7f44f4007710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51270/0 (socket says 172.21.15.81:51270) 2024-05-08T23:30:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f44fb59e700 1 -- 172.21.15.81:0/1905174072 learned_addr learned my addr 172.21.15.81:0/1905174072 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f44fad9d700 1 --1- 172.21.15.81:0/1905174072 >> v1:172.21.15.81:6789/0 conn(0x7f44f40e33a0 0x7f44f4056fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38784/0 (socket says 172.21.15.81:38784) 2024-05-08T23:30:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f44fa59c700 1 --2- 172.21.15.81:0/1905174072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44f403ebd0 0x7f44f4058950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f44fb59e700 1 -- 172.21.15.81:0/1905174072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44f403ebd0 msgr2=0x7f44f4058950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f44fb59e700 1 --2- 172.21.15.81:0/1905174072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44f403ebd0 0x7f44f4058950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f44fb59e700 1 -- 172.21.15.81:0/1905174072 >> v1:172.21.15.81:6789/0 conn(0x7f44f40e33a0 legacy=0x7f44f4056fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f44fb59e700 1 -- 172.21.15.81:0/1905174072 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f4057b90 con 0x7f44f4007280 2024-05-08T23:30:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f44fb59e700 1 --2- 172.21.15.81:0/1905174072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44f4007280 0x7f44f4007710 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0698a0 tx=0x7f44fc068970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cb1f7935d2a9f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f44f9d9b700 1 -- 172.21.15.81:0/1905174072 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0679b0 con 0x7f44f4007280 2024-05-08T23:30:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f44f9d9b700 1 -- 172.21.15.81:0/1905174072 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc067b10 con 0x7f44f4007280 2024-05-08T23:30:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f44f9d9b700 1 -- 172.21.15.81:0/1905174072 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc072130 con 0x7f44f4007280 2024-05-08T23:30:49.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f4500980700 1 -- 172.21.15.81:0/1905174072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44f4007280 msgr2=0x7f44f4007710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f4500980700 1 --2- 172.21.15.81:0/1905174072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44f4007280 0x7f44f4007710 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0698a0 tx=0x7f44fc068970 comp rx=0 tx=0).stop 2024-05-08T23:30:49.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f4500980700 1 -- 172.21.15.81:0/1905174072 shutdown_connections 2024-05-08T23:30:49.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f4500980700 1 --2- 172.21.15.81:0/1905174072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44f4007280 0x7f44f4007710 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f4500980700 1 --2- 172.21.15.81:0/1905174072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44f403ebd0 0x7f44f4058950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f4500980700 1 -- 172.21.15.81:0/1905174072 >> 172.21.15.81:0/1905174072 conn(0x7f44f4055a10 msgr2=0x7f44f4055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:49.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f4500980700 1 -- 172.21.15.81:0/1905174072 shutdown_connections 2024-05-08T23:30:49.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f4500980700 1 -- 172.21.15.81:0/1905174072 wait complete. 2024-05-08T23:30:49.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f4500980700 1 Processor -- start 2024-05-08T23:30:49.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.785+0000 7f4500980700 1 -- start start 2024-05-08T23:30:49.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f4500980700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f4007280 0x7f44f4039390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f4500980700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44f403ebd0 0x7f44f40dcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f4500980700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44f40e33a0 0x7f44f4038b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f4500980700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4039970 con 0x7f44f403ebd0 2024-05-08T23:30:49.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f4500980700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4039c50 con 0x7f44f4007280 2024-05-08T23:30:49.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f4500980700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40512c0 con 0x7f44f40e33a0 2024-05-08T23:30:49.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f44fa59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44f403ebd0 0x7f44f40dcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f44fa59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44f403ebd0 0x7f44f40dcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51286/0 (socket says 172.21.15.81:51286) 2024-05-08T23:30:49.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f44fa59c700 1 -- 172.21.15.81:0/2624109434 learned_addr learned my addr 172.21.15.81:0/2624109434 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f44fb59e700 1 --2- 172.21.15.81:0/2624109434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44f40e33a0 0x7f44f4038b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f44fad9d700 1 --2- 172.21.15.81:0/2624109434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f4007280 0x7f44f4039390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f44fa59c700 1 -- 172.21.15.81:0/2624109434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44f40e33a0 msgr2=0x7f44f4038b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f44fa59c700 1 --2- 172.21.15.81:0/2624109434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44f40e33a0 0x7f44f4038b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f44fa59c700 1 -- 172.21.15.81:0/2624109434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f4007280 msgr2=0x7f44f4039390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f44fa59c700 1 --2- 172.21.15.81:0/2624109434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f4007280 0x7f44f4039390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f44fa59c700 1 -- 172.21.15.81:0/2624109434 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc04f5c0 con 0x7f44f403ebd0 2024-05-08T23:30:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f44fa59c700 1 --2- 172.21.15.81:0/2624109434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44f403ebd0 0x7f44f40dcc10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00e760 tx=0x7f44ec00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f44e37fe700 1 -- 172.21.15.81:0/2624109434 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec013610 con 0x7f44f403ebd0 2024-05-08T23:30:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f4500980700 1 -- 172.21.15.81:0/2624109434 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f40515a0 con 0x7f44f403ebd0 2024-05-08T23:30:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f44e37fe700 1 -- 172.21.15.81:0/2624109434 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ec013770 con 0x7f44f403ebd0 2024-05-08T23:30:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f44e37fe700 1 -- 172.21.15.81:0/2624109434 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec013ae0 con 0x7f44f403ebd0 2024-05-08T23:30:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f4500980700 1 -- 172.21.15.81:0/2624109434 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f40e4fe0 con 0x7f44f403ebd0 2024-05-08T23:30:49.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.789+0000 7f4500980700 1 -- 172.21.15.81:0/2624109434 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f4005440 con 0x7f44f403ebd0 2024-05-08T23:30:49.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.793+0000 7f44e37fe700 1 -- 172.21.15.81:0/2624109434 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f44ec013c40 con 0x7f44f403ebd0 2024-05-08T23:30:49.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.793+0000 7f44e37fe700 1 --2- 172.21.15.81:0/2624109434 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f44e8066730 0x7f44e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:49.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.793+0000 7f44fad9d700 1 --2- 172.21.15.81:0/2624109434 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f44e8066730 0x7f44e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:49.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.793+0000 7f44e37fe700 1 -- 172.21.15.81:0/2624109434 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44ec084220 con 0x7f44f403ebd0 2024-05-08T23:30:49.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.793+0000 7f44fad9d700 1 --2- 172.21.15.81:0/2624109434 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f44e8066730 0x7f44e8068bf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f44f0000f60 tx=0x7f44f0007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:49.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.801+0000 7f44e37fe700 1 -- 172.21.15.81:0/2624109434 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f44ec051500 con 0x7f44f403ebd0 2024-05-08T23:30:49.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.969+0000 7f4500980700 1 -- 172.21.15.81:0/2624109434 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f44f4038710 con 0x7f44f403ebd0 2024-05-08T23:30:49.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.969+0000 7f44e37fe700 1 -- 172.21.15.81:0/2624109434 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f44ec054b20 con 0x7f44f403ebd0 2024-05-08T23:30:49.975 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:30:49.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.973+0000 7f4500980700 1 -- 172.21.15.81:0/2624109434 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f44e8066730 msgr2=0x7f44e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.973+0000 7f4500980700 1 --2- 172.21.15.81:0/2624109434 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f44e8066730 0x7f44e8068bf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f44f0000f60 tx=0x7f44f0007d30 comp rx=0 tx=0).stop 2024-05-08T23:30:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.973+0000 7f4500980700 1 -- 172.21.15.81:0/2624109434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44f403ebd0 msgr2=0x7f44f40dcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.973+0000 7f4500980700 1 --2- 172.21.15.81:0/2624109434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44f403ebd0 0x7f44f40dcc10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00e760 tx=0x7f44ec00eb20 comp rx=0 tx=0).stop 2024-05-08T23:30:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.973+0000 7f4500980700 1 -- 172.21.15.81:0/2624109434 shutdown_connections 2024-05-08T23:30:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.973+0000 7f4500980700 1 --2- 172.21.15.81:0/2624109434 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f44e8066730 0x7f44e8068bf0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.973+0000 7f4500980700 1 --2- 172.21.15.81:0/2624109434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44f40e33a0 0x7f44f4038b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.973+0000 7f4500980700 1 --2- 172.21.15.81:0/2624109434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44f403ebd0 0x7f44f40dcc10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.973+0000 7f4500980700 1 --2- 172.21.15.81:0/2624109434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f4007280 0x7f44f4039390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.973+0000 7f4500980700 1 -- 172.21.15.81:0/2624109434 >> 172.21.15.81:0/2624109434 conn(0x7f44f4055a10 msgr2=0x7f44f404d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.973+0000 7f4500980700 1 -- 172.21.15.81:0/2624109434 shutdown_connections 2024-05-08T23:30:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:49.973+0000 7f4500980700 1 -- 172.21.15.81:0/2624109434 wait complete. 2024-05-08T23:30:49.991 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:30:50.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.109+0000 7fb56ce07700 1 Processor -- start 2024-05-08T23:30:50.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.109+0000 7fb56ce07700 1 -- start start 2024-05-08T23:30:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.109+0000 7fb56ce07700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb568159730 0x7fb56815d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.109+0000 7fb56ce07700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb56806ed70 0x7fb5680b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.109+0000 7fb56ce07700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb56815e220 con 0x7fb5680cb770 2024-05-08T23:30:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.109+0000 7fb56ce07700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5680b2790 con 0x7fb568159730 2024-05-08T23:30:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.109+0000 7fb56ce07700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56806f340 con 0x7fb56806ed70 2024-05-08T23:30:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb566ffd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb5680cb770 0x7fb56806e660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38786/0 (socket says 172.21.15.81:38786) 2024-05-08T23:30:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb566ffd700 1 -- 172.21.15.81:0/3950446014 learned_addr learned my addr 172.21.15.81:0/3950446014 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb567fff700 1 --2- 172.21.15.81:0/3950446014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb56806ed70 0x7fb5680b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5677fe700 1 --2- 172.21.15.81:0/3950446014 >> v2:172.21.15.117:3300/0 conn(0x7fb568159730 0x7fb56815d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 -- 172.21.15.81:0/3950446014 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043300239 0 0) 0x7fb56815e220 con 0x7fb5680cb770 2024-05-08T23:30:50.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 -- 172.21.15.81:0/3950446014 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5500057e0 con 0x7fb5680cb770 2024-05-08T23:30:50.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 -- 172.21.15.81:0/3950446014 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 48314816 0 0) 0x7fb5500057e0 con 0x7fb5680cb770 2024-05-08T23:30:50.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 -- 172.21.15.81:0/3950446014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb56806ed70 msgr2=0x7fb5680b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:50.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 --2- 172.21.15.81:0/3950446014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb56806ed70 0x7fb5680b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 -- 172.21.15.81:0/3950446014 >> v2:172.21.15.117:3300/0 conn(0x7fb568159730 msgr2=0x7fb56815d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:50.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 --2- 172.21.15.81:0/3950446014 >> v2:172.21.15.117:3300/0 conn(0x7fb568159730 0x7fb56815d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 -- 172.21.15.81:0/3950446014 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb568159b10 con 0x7fb5680cb770 2024-05-08T23:30:50.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 -- 172.21.15.81:0/3950446014 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7fb55c001670 con 0x7fb5680cb770 2024-05-08T23:30:50.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 -- 172.21.15.81:0/3950446014 >> v1:172.21.15.81:6789/0 conn(0x7fb5680cb770 legacy=0x7fb56806e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 --2- 172.21.15.81:0/3950446014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb550005c60 0x7fb550006080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 --2- 172.21.15.81:0/3950446014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb56806ed70 0x7fb550006a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 --2- 172.21.15.81:0/3950446014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb550006fc0 0x7fb550007440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 -- 172.21.15.81:0/3950446014 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5680b2790 con 0x7fb550005c60 2024-05-08T23:30:50.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 -- 172.21.15.81:0/3950446014 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56806f340 con 0x7fb550006fc0 2024-05-08T23:30:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 -- 172.21.15.81:0/3950446014 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55000dc40 con 0x7fb56806ed70 2024-05-08T23:30:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5677fe700 1 --2- 172.21.15.81:0/3950446014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb550005c60 0x7fb550006080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb566ffd700 1 --2- 172.21.15.81:0/3950446014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb56806ed70 0x7fb550006a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb567fff700 1 --2- 172.21.15.81:0/3950446014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb550006fc0 0x7fb550007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5677fe700 1 -- 172.21.15.81:0/3950446014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb56806ed70 msgr2=0x7fb550006a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5677fe700 1 --2- 172.21.15.81:0/3950446014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb56806ed70 0x7fb550006a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5677fe700 1 -- 172.21.15.81:0/3950446014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb550006fc0 msgr2=0x7fb550007440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5677fe700 1 --2- 172.21.15.81:0/3950446014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb550006fc0 0x7fb550007440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5677fe700 1 -- 172.21.15.81:0/3950446014 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5500057e0 con 0x7fb550005c60 2024-05-08T23:30:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5677fe700 1 --2- 172.21.15.81:0/3950446014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb550005c60 0x7fb550006080 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb558009b70 tx=0x7fb550007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adac4c2d5ee5f38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 -- 172.21.15.81:0/3950446014 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55800e660 con 0x7fb550005c60 2024-05-08T23:30:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 -- 172.21.15.81:0/3950446014 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55800e7c0 con 0x7fb550005c60 2024-05-08T23:30:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb5667fc700 1 -- 172.21.15.81:0/3950446014 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55800eb30 con 0x7fb550005c60 2024-05-08T23:30:50.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb56ce07700 1 -- 172.21.15.81:0/3950446014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb550005c60 msgr2=0x7fb550006080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb56ce07700 1 --2- 172.21.15.81:0/3950446014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb550005c60 0x7fb550006080 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb558009b70 tx=0x7fb550007ab0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb567fff700 1 -- 172.21.15.81:0/3950446014 reap_dead start 2024-05-08T23:30:50.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb56ce07700 1 -- 172.21.15.81:0/3950446014 shutdown_connections 2024-05-08T23:30:50.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb56ce07700 1 -- 172.21.15.81:0/3950446014 >> 172.21.15.81:0/3950446014 conn(0x7fb5680717b0 msgr2=0x7fb56806a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:50.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb56ce07700 1 -- 172.21.15.81:0/3950446014 shutdown_connections 2024-05-08T23:30:50.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.113+0000 7fb56ce07700 1 -- 172.21.15.81:0/3950446014 wait complete. 2024-05-08T23:30:50.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb56ce07700 1 Processor -- start 2024-05-08T23:30:50.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb56ce07700 1 -- start start 2024-05-08T23:30:50.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb56ce07700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb550005c60 0x7fb568159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb56ce07700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb56813edb0 0x7fb56815a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb56ce07700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb56815aea0 0x7fb5681eea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb56ce07700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56815b320 con 0x7fb56813edb0 2024-05-08T23:30:50.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb56ce07700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5680cec00 con 0x7fb550005c60 2024-05-08T23:30:50.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb56ce07700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5680cdff0 con 0x7fb56815aea0 2024-05-08T23:30:50.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb566ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb56813edb0 0x7fb56815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb566ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb56813edb0 0x7fb56815a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51294/0 (socket says 172.21.15.81:51294) 2024-05-08T23:30:50.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb566ffd700 1 -- 172.21.15.81:0/2577266982 learned_addr learned my addr 172.21.15.81:0/2577266982 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:50.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb5677fe700 1 --2- 172.21.15.81:0/2577266982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb550005c60 0x7fb568159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb567fff700 1 --2- 172.21.15.81:0/2577266982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb56815aea0 0x7fb5681eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb566ffd700 1 -- 172.21.15.81:0/2577266982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb56815aea0 msgr2=0x7fb5681eea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb566ffd700 1 --2- 172.21.15.81:0/2577266982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb56815aea0 0x7fb5681eea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb566ffd700 1 -- 172.21.15.81:0/2577266982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb550005c60 msgr2=0x7fb568159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb566ffd700 1 --2- 172.21.15.81:0/2577266982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb550005c60 0x7fb568159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb566ffd700 1 -- 172.21.15.81:0/2577266982 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5680ce2d0 con 0x7fb56813edb0 2024-05-08T23:30:50.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.117+0000 7fb566ffd700 1 --2- 172.21.15.81:0/2577266982 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb56813edb0 0x7fb56815a960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb55c007310 tx=0x7fb55c007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:50.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.121+0000 7fb54ffff700 1 -- 172.21.15.81:0/2577266982 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c011580 con 0x7fb56813edb0 2024-05-08T23:30:50.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.121+0000 7fb54ffff700 1 -- 172.21.15.81:0/2577266982 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c011700 con 0x7fb56813edb0 2024-05-08T23:30:50.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.121+0000 7fb56ce07700 1 -- 172.21.15.81:0/2577266982 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5680ce5b0 con 0x7fb56813edb0 2024-05-08T23:30:50.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.121+0000 7fb54ffff700 1 -- 172.21.15.81:0/2577266982 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c011ab0 con 0x7fb56813edb0 2024-05-08T23:30:50.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.121+0000 7fb56ce07700 1 -- 172.21.15.81:0/2577266982 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56813f230 con 0x7fb56813edb0 2024-05-08T23:30:50.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.121+0000 7fb56ce07700 1 -- 172.21.15.81:0/2577266982 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5680672a0 con 0x7fb56813edb0 2024-05-08T23:30:50.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.121+0000 7fb54ffff700 1 -- 172.21.15.81:0/2577266982 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb55c011c10 con 0x7fb56813edb0 2024-05-08T23:30:50.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.121+0000 7fb54ffff700 1 --2- 172.21.15.81:0/2577266982 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb554066730 0x7fb554068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.125+0000 7fb5677fe700 1 --2- 172.21.15.81:0/2577266982 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb554066730 0x7fb554068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.125+0000 7fb54ffff700 1 -- 172.21.15.81:0/2577266982 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb55c084b50 con 0x7fb56813edb0 2024-05-08T23:30:50.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.125+0000 7fb5677fe700 1 --2- 172.21.15.81:0/2577266982 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb554066730 0x7fb554068bf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb5580090b0 tx=0x7fb55801e040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:50.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.129+0000 7fb54ffff700 1 -- 172.21.15.81:0/2577266982 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb55c051f50 con 0x7fb56813edb0 2024-05-08T23:30:50.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.297+0000 7fb56ce07700 1 -- 172.21.15.81:0/2577266982 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5680ced40 con 0x7fb56813edb0 2024-05-08T23:30:50.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.297+0000 7fb54ffff700 1 -- 172.21.15.81:0/2577266982 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb55c055570 con 0x7fb56813edb0 2024-05-08T23:30:50.303 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:30:50.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.301+0000 7fb56ce07700 1 -- 172.21.15.81:0/2577266982 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb554066730 msgr2=0x7fb554068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.301+0000 7fb56ce07700 1 --2- 172.21.15.81:0/2577266982 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb554066730 0x7fb554068bf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb5580090b0 tx=0x7fb55801e040 comp rx=0 tx=0).stop 2024-05-08T23:30:50.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.301+0000 7fb56ce07700 1 -- 172.21.15.81:0/2577266982 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb56813edb0 msgr2=0x7fb56815a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.301+0000 7fb56ce07700 1 --2- 172.21.15.81:0/2577266982 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb56813edb0 0x7fb56815a960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb55c007310 tx=0x7fb55c007620 comp rx=0 tx=0).stop 2024-05-08T23:30:50.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.301+0000 7fb56ce07700 1 -- 172.21.15.81:0/2577266982 shutdown_connections 2024-05-08T23:30:50.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.301+0000 7fb56ce07700 1 --2- 172.21.15.81:0/2577266982 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb554066730 0x7fb554068bf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.301+0000 7fb56ce07700 1 --2- 172.21.15.81:0/2577266982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb56815aea0 0x7fb5681eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.301+0000 7fb56ce07700 1 --2- 172.21.15.81:0/2577266982 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb56813edb0 0x7fb56815a960 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.301+0000 7fb56ce07700 1 --2- 172.21.15.81:0/2577266982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb550005c60 0x7fb568159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.301+0000 7fb56ce07700 1 -- 172.21.15.81:0/2577266982 >> 172.21.15.81:0/2577266982 conn(0x7fb5680717b0 msgr2=0x7fb56806a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:50.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.301+0000 7fb56ce07700 1 -- 172.21.15.81:0/2577266982 shutdown_connections 2024-05-08T23:30:50.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.301+0000 7fb56ce07700 1 -- 172.21.15.81:0/2577266982 wait complete. 2024-05-08T23:30:50.323 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:30:50.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.425+0000 7f2780948700 1 Processor -- start 2024-05-08T23:30:50.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.425+0000 7f2780948700 1 -- start start 2024-05-08T23:30:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2780948700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277c1545d0 0x7f277c15a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2780948700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f277c071230 0x7f277c0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2780948700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f277c0cdca0 con 0x7f277c0b3d00 2024-05-08T23:30:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2780948700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f277c0b4190 con 0x7f277c071230 2024-05-08T23:30:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2780948700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277c1549b0 con 0x7f277c1545d0 2024-05-08T23:30:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f277a59c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f277c071230 0x7f277c0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f277a59c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f277c071230 0x7f277c0b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51296/0 (socket says 172.21.15.81:51296) 2024-05-08T23:30:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f277a59c700 1 -- 172.21.15.81:0/3236314500 learned_addr learned my addr 172.21.15.81:0/3236314500 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f277ad9d700 1 --2- 172.21.15.81:0/3236314500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277c1545d0 0x7f277c15a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f277a59c700 1 -- 172.21.15.81:0/3236314500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277c1545d0 msgr2=0x7f277c15a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f277a59c700 1 --2- 172.21.15.81:0/3236314500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277c1545d0 0x7f277c15a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f277a59c700 1 -- 172.21.15.81:0/3236314500 >> v1:172.21.15.117:6789/0 conn(0x7f277c0b3d00 legacy=0x7f277c06efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f277a59c700 1 -- 172.21.15.81:0/3236314500 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c071a40 con 0x7f277c071230 2024-05-08T23:30:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f277ad9d700 1 --2- 172.21.15.81:0/3236314500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277c1545d0 0x7f277c15a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f277a59c700 1 --2- 172.21.15.81:0/3236314500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f277c071230 0x7f277c0b37c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2770009910 tx=0x7f2770009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07bbbd6f4e6656e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2779d9b700 1 -- 172.21.15.81:0/3236314500 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277000ba60 con 0x7f277c071230 2024-05-08T23:30:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2779d9b700 1 -- 172.21.15.81:0/3236314500 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277000bbc0 con 0x7f277c071230 2024-05-08T23:30:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2779d9b700 1 -- 172.21.15.81:0/3236314500 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770009e60 con 0x7f277c071230 2024-05-08T23:30:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2780948700 1 -- 172.21.15.81:0/3236314500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f277c071230 msgr2=0x7f277c0b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2780948700 1 --2- 172.21.15.81:0/3236314500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f277c071230 0x7f277c0b37c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2770009910 tx=0x7f2770009c20 comp rx=0 tx=0).stop 2024-05-08T23:30:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2780948700 1 -- 172.21.15.81:0/3236314500 shutdown_connections 2024-05-08T23:30:50.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2780948700 1 --2- 172.21.15.81:0/3236314500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f277c071230 0x7f277c0b37c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2780948700 1 --2- 172.21.15.81:0/3236314500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277c1545d0 0x7f277c15a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2780948700 1 -- 172.21.15.81:0/3236314500 >> 172.21.15.81:0/3236314500 conn(0x7f277c06b470 msgr2=0x7f277c06b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:50.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2780948700 1 -- 172.21.15.81:0/3236314500 shutdown_connections 2024-05-08T23:30:50.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.429+0000 7f2780948700 1 -- 172.21.15.81:0/3236314500 wait complete. 2024-05-08T23:30:50.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f2780948700 1 Processor -- start 2024-05-08T23:30:50.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f2780948700 1 -- start start 2024-05-08T23:30:50.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f2780948700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f277c071230 0x7f277c06da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f2780948700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277c0b3d00 0x7f277c1679e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f2780948700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f277c1545d0 0x7f277c14ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f2780948700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c167f20 con 0x7f277c071230 2024-05-08T23:30:50.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f2780948700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c14b380 con 0x7f277c1545d0 2024-05-08T23:30:50.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f2780948700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277c166000 con 0x7f277c0b3d00 2024-05-08T23:30:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f277ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f277c071230 0x7f277c06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f277ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f277c071230 0x7f277c06da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51304/0 (socket says 172.21.15.81:51304) 2024-05-08T23:30:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f277ad9d700 1 -- 172.21.15.81:0/2116876039 learned_addr learned my addr 172.21.15.81:0/2116876039 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f277a59c700 1 --2- 172.21.15.81:0/2116876039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277c0b3d00 0x7f277c1679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f277b59e700 1 --2- 172.21.15.81:0/2116876039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f277c1545d0 0x7f277c14ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f277ad9d700 1 -- 172.21.15.81:0/2116876039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277c0b3d00 msgr2=0x7f277c1679e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f277ad9d700 1 --2- 172.21.15.81:0/2116876039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277c0b3d00 0x7f277c1679e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f277ad9d700 1 -- 172.21.15.81:0/2116876039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f277c1545d0 msgr2=0x7f277c14ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f277ad9d700 1 --2- 172.21.15.81:0/2116876039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f277c1545d0 0x7f277c14ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f277ad9d700 1 -- 172.21.15.81:0/2116876039 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2770009570 con 0x7f277c071230 2024-05-08T23:30:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f277b59e700 1 --2- 172.21.15.81:0/2116876039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f277c1545d0 0x7f277c14ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f277ad9d700 1 --2- 172.21.15.81:0/2116876039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f277c071230 0x7f277c06da30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f276c00e810 tx=0x7f276c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f27637fe700 1 -- 172.21.15.81:0/2116876039 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c004230 con 0x7f277c071230 2024-05-08T23:30:50.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f27637fe700 1 -- 172.21.15.81:0/2116876039 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c004390 con 0x7f277c071230 2024-05-08T23:30:50.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f2780948700 1 -- 172.21.15.81:0/2116876039 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277c1662e0 con 0x7f277c071230 2024-05-08T23:30:50.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f27637fe700 1 -- 172.21.15.81:0/2116876039 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c0036c0 con 0x7f277c071230 2024-05-08T23:30:50.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f2780948700 1 -- 172.21.15.81:0/2116876039 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277c1668a0 con 0x7f277c071230 2024-05-08T23:30:50.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.433+0000 7f2780948700 1 -- 172.21.15.81:0/2116876039 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277c067e40 con 0x7f277c071230 2024-05-08T23:30:50.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.437+0000 7f27637fe700 1 -- 172.21.15.81:0/2116876039 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f276c003840 con 0x7f277c071230 2024-05-08T23:30:50.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.437+0000 7f27637fe700 1 --2- 172.21.15.81:0/2116876039 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2768066730 0x7f2768068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.437+0000 7f27637fe700 1 -- 172.21.15.81:0/2116876039 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f276c083b60 con 0x7f277c071230 2024-05-08T23:30:50.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.437+0000 7f277a59c700 1 --2- 172.21.15.81:0/2116876039 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2768066730 0x7f2768068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.437+0000 7f277a59c700 1 --2- 172.21.15.81:0/2116876039 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2768066730 0x7f2768068bf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2770009c60 tx=0x7f2770008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:50.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.441+0000 7f27637fe700 1 -- 172.21.15.81:0/2116876039 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f276c050f60 con 0x7f277c071230 2024-05-08T23:30:50.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.649+0000 7f2780948700 1 -- 172.21.15.81:0/2116876039 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f277c072a10 con 0x7f277c071230 2024-05-08T23:30:50.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.649+0000 7f27637fe700 1 -- 172.21.15.81:0/2116876039 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7f276c020090 con 0x7f277c071230 2024-05-08T23:30:50.653 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:50.653 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1475726755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1475726755},{"type":"v1","addr":"172.21.15.81:6835","nonce":1475726755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3535820540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3535820540},{"type":"v1","addr":"172.21.15.81:6837","nonce":3535820540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1531019955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1531019955},{"type":"v1","addr":"172.21.15.117:6833","nonce":1531019955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:30:38.665368+0000","modified":"2024-05-08T23:30:41.672722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/491911303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":491911303},{"type":"v1","addr":"172.21.15.117:6835","nonce":491911303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:30:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.649+0000 7f2780948700 1 -- 172.21.15.81:0/2116876039 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2768066730 msgr2=0x7f2768068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.649+0000 7f2780948700 1 --2- 172.21.15.81:0/2116876039 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2768066730 0x7f2768068bf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2770009c60 tx=0x7f2770008030 comp rx=0 tx=0).stop 2024-05-08T23:30:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.649+0000 7f2780948700 1 -- 172.21.15.81:0/2116876039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f277c071230 msgr2=0x7f277c06da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.649+0000 7f2780948700 1 --2- 172.21.15.81:0/2116876039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f277c071230 0x7f277c06da30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f276c00e810 tx=0x7f276c00ebd0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.649+0000 7f2780948700 1 -- 172.21.15.81:0/2116876039 shutdown_connections 2024-05-08T23:30:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.649+0000 7f2780948700 1 --2- 172.21.15.81:0/2116876039 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2768066730 0x7f2768068bf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.649+0000 7f2780948700 1 --2- 172.21.15.81:0/2116876039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f277c1545d0 0x7f277c14ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.649+0000 7f2780948700 1 --2- 172.21.15.81:0/2116876039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277c0b3d00 0x7f277c1679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.649+0000 7f2780948700 1 --2- 172.21.15.81:0/2116876039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f277c071230 0x7f277c06da30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.649+0000 7f2780948700 1 -- 172.21.15.81:0/2116876039 >> 172.21.15.81:0/2116876039 conn(0x7f277c06b470 msgr2=0x7f277c06a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.649+0000 7f2780948700 1 -- 172.21.15.81:0/2116876039 shutdown_connections 2024-05-08T23:30:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.653+0000 7f2780948700 1 -- 172.21.15.81:0/2116876039 wait complete. 2024-05-08T23:30:50.657 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 7 2024-05-08T23:30:50.667 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:30:50.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.765+0000 7ffa92043700 1 Processor -- start 2024-05-08T23:30:50.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.765+0000 7ffa92043700 1 -- start start 2024-05-08T23:30:50.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa92043700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffa8c15aed0 0x7ffa8c0cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa92043700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0cb970 0x7ffa8c0b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa92043700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ffa8c0b2e40 0x7ffa8c06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa92043700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8c0cd470 con 0x7ffa8c0b2e40 2024-05-08T23:30:50.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa92043700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8c0ced50 con 0x7ffa8c15aed0 2024-05-08T23:30:50.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa92043700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c0cceb0 con 0x7ffa8c0cb970 2024-05-08T23:30:50.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa8bfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ffa8c0b2e40 0x7ffa8c06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa8bfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ffa8c0b2e40 0x7ffa8c06c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51306/0 (socket says 172.21.15.81:51306) 2024-05-08T23:30:50.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa8bfff700 1 -- 172.21.15.81:0/1691333618 learned_addr learned my addr 172.21.15.81:0/1691333618 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:50.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa8b7fe700 1 --2- 172.21.15.81:0/1691333618 >> v2:172.21.15.117:3300/0 conn(0x7ffa8c15aed0 0x7ffa8c0cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa8affd700 1 --2- 172.21.15.81:0/1691333618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0cb970 0x7ffa8c0b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa8bfff700 1 -- 172.21.15.81:0/1691333618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0cb970 msgr2=0x7ffa8c0b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:50.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa8bfff700 1 --2- 172.21.15.81:0/1691333618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0cb970 0x7ffa8c0b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa8bfff700 1 -- 172.21.15.81:0/1691333618 >> v2:172.21.15.117:3300/0 conn(0x7ffa8c15aed0 msgr2=0x7ffa8c0cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:50.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa8bfff700 1 --2- 172.21.15.81:0/1691333618 >> v2:172.21.15.117:3300/0 conn(0x7ffa8c15aed0 0x7ffa8c0cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa8bfff700 1 -- 172.21.15.81:0/1691333618 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c0cd190 con 0x7ffa8c0b2e40 2024-05-08T23:30:50.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa8affd700 1 --2- 172.21.15.81:0/1691333618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0cb970 0x7ffa8c0b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:50.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa8bfff700 1 --2- 172.21.15.81:0/1691333618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c0b2e40 0x7ffa8c06c350 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c06e3e0 tx=0x7ffa8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532e7cb6d5430dd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:50.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa8a7fc700 1 -- 172.21.15.81:0/1691333618 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400e070 con 0x7ffa8c0b2e40 2024-05-08T23:30:50.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa8a7fc700 1 -- 172.21.15.81:0/1691333618 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8400b8d0 con 0x7ffa8c0b2e40 2024-05-08T23:30:50.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa8a7fc700 1 -- 172.21.15.81:0/1691333618 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400bc80 con 0x7ffa8c0b2e40 2024-05-08T23:30:50.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa92043700 1 -- 172.21.15.81:0/1691333618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c0b2e40 msgr2=0x7ffa8c06c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa92043700 1 --2- 172.21.15.81:0/1691333618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c0b2e40 0x7ffa8c06c350 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c06e3e0 tx=0x7ffa8400b330 comp rx=0 tx=0).stop 2024-05-08T23:30:50.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa92043700 1 -- 172.21.15.81:0/1691333618 shutdown_connections 2024-05-08T23:30:50.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa92043700 1 --2- 172.21.15.81:0/1691333618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0cb970 0x7ffa8c0b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa92043700 1 --2- 172.21.15.81:0/1691333618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c0b2e40 0x7ffa8c06c350 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa92043700 1 --2- 172.21.15.81:0/1691333618 >> v2:172.21.15.117:3300/0 conn(0x7ffa8c15aed0 0x7ffa8c0cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa92043700 1 -- 172.21.15.81:0/1691333618 >> 172.21.15.81:0/1691333618 conn(0x7ffa8c06a360 msgr2=0x7ffa8c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:50.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa92043700 1 -- 172.21.15.81:0/1691333618 shutdown_connections 2024-05-08T23:30:50.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.769+0000 7ffa92043700 1 -- 172.21.15.81:0/1691333618 wait complete. 2024-05-08T23:30:50.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa92043700 1 Processor -- start 2024-05-08T23:30:50.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa92043700 1 -- start start 2024-05-08T23:30:50.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa92043700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c0b2e40 0x7ffa8c14de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa92043700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0cb970 0x7ffa8c1476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa92043700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c15aed0 0x7ffa8c0ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa92043700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c0ab610 con 0x7ffa8c15aed0 2024-05-08T23:30:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa92043700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c157a20 con 0x7ffa8c0b2e40 2024-05-08T23:30:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa92043700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c0ab860 con 0x7ffa8c0cb970 2024-05-08T23:30:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa8bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c15aed0 0x7ffa8c0ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa8b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c0b2e40 0x7ffa8c14de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa8bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c15aed0 0x7ffa8c0ad920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51314/0 (socket says 172.21.15.81:51314) 2024-05-08T23:30:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa8bfff700 1 -- 172.21.15.81:0/217680298 learned_addr learned my addr 172.21.15.81:0/217680298 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa8affd700 1 --2- 172.21.15.81:0/217680298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0cb970 0x7ffa8c1476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa8bfff700 1 -- 172.21.15.81:0/217680298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0cb970 msgr2=0x7ffa8c1476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa8bfff700 1 --2- 172.21.15.81:0/217680298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0cb970 0x7ffa8c1476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa8bfff700 1 -- 172.21.15.81:0/217680298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c0b2e40 msgr2=0x7ffa8c14de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa8bfff700 1 --2- 172.21.15.81:0/217680298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c0b2e40 0x7ffa8c14de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa8bfff700 1 -- 172.21.15.81:0/217680298 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa740094b0 con 0x7ffa8c15aed0 2024-05-08T23:30:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.773+0000 7ffa8bfff700 1 --2- 172.21.15.81:0/217680298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c15aed0 0x7ffa8c0ad920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c074490 tx=0x7ffa84004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.777+0000 7ffa83fff700 1 -- 172.21.15.81:0/217680298 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84004880 con 0x7ffa8c15aed0 2024-05-08T23:30:50.781 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.777+0000 7ffa83fff700 1 -- 172.21.15.81:0/217680298 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa840092e0 con 0x7ffa8c15aed0 2024-05-08T23:30:50.781 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.777+0000 7ffa83fff700 1 -- 172.21.15.81:0/217680298 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8401c680 con 0x7ffa8c15aed0 2024-05-08T23:30:50.781 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.777+0000 7ffa92043700 1 -- 172.21.15.81:0/217680298 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa84009b30 con 0x7ffa8c15aed0 2024-05-08T23:30:50.781 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.777+0000 7ffa92043700 1 -- 172.21.15.81:0/217680298 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8c0abe20 con 0x7ffa8c15aed0 2024-05-08T23:30:50.781 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.777+0000 7ffa92043700 1 -- 172.21.15.81:0/217680298 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8c0ac040 con 0x7ffa8c15aed0 2024-05-08T23:30:50.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.777+0000 7ffa83fff700 1 -- 172.21.15.81:0/217680298 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffa8401c7e0 con 0x7ffa8c15aed0 2024-05-08T23:30:50.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.777+0000 7ffa83fff700 1 --2- 172.21.15.81:0/217680298 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa6c066760 0x7ffa6c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:50.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.777+0000 7ffa83fff700 1 -- 172.21.15.81:0/217680298 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa84085c00 con 0x7ffa8c15aed0 2024-05-08T23:30:50.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.777+0000 7ffa8b7fe700 1 --2- 172.21.15.81:0/217680298 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa6c066760 0x7ffa6c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:50.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.781+0000 7ffa8b7fe700 1 --2- 172.21.15.81:0/217680298 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa6c066760 0x7ffa6c068c20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0cb860 tx=0x7ffa7c009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:50.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.781+0000 7ffa83fff700 1 -- 172.21.15.81:0/217680298 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffa8408c080 con 0x7ffa8c15aed0 2024-05-08T23:30:50.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.953+0000 7ffa92043700 1 -- 172.21.15.81:0/217680298 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa8c0684b0 con 0x7ffa8c15aed0 2024-05-08T23:30:50.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.957+0000 7ffa83fff700 1 -- 172.21.15.81:0/217680298 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7ffa84052f80 con 0x7ffa8c15aed0 2024-05-08T23:30:50.961 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:50.961 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:30:50.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.957+0000 7ffa92043700 1 -- 172.21.15.81:0/217680298 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa6c066760 msgr2=0x7ffa6c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.957+0000 7ffa92043700 1 --2- 172.21.15.81:0/217680298 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa6c066760 0x7ffa6c068c20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0cb860 tx=0x7ffa7c009240 comp rx=0 tx=0).stop 2024-05-08T23:30:50.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.957+0000 7ffa92043700 1 -- 172.21.15.81:0/217680298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c15aed0 msgr2=0x7ffa8c0ad920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:50.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.957+0000 7ffa92043700 1 --2- 172.21.15.81:0/217680298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c15aed0 0x7ffa8c0ad920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c074490 tx=0x7ffa84004af0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.957+0000 7ffa92043700 1 -- 172.21.15.81:0/217680298 shutdown_connections 2024-05-08T23:30:50.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.957+0000 7ffa92043700 1 --2- 172.21.15.81:0/217680298 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa6c066760 0x7ffa6c068c20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.957+0000 7ffa92043700 1 --2- 172.21.15.81:0/217680298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c15aed0 0x7ffa8c0ad920 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.957+0000 7ffa92043700 1 --2- 172.21.15.81:0/217680298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0cb970 0x7ffa8c1476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.957+0000 7ffa92043700 1 --2- 172.21.15.81:0/217680298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c0b2e40 0x7ffa8c14de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:50.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.957+0000 7ffa92043700 1 -- 172.21.15.81:0/217680298 >> 172.21.15.81:0/217680298 conn(0x7ffa8c06a360 msgr2=0x7ffa8c1506b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:50.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.957+0000 7ffa92043700 1 -- 172.21.15.81:0/217680298 shutdown_connections 2024-05-08T23:30:50.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:50.957+0000 7ffa92043700 1 -- 172.21.15.81:0/217680298 wait complete. 2024-05-08T23:30:50.976 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:30:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 Processor -- start 2024-05-08T23:30:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 -- start start 2024-05-08T23:30:51.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2bf40cb3b0 0x7f2bf41478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bf4165240 0x7f2bf4162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2bf41628a0 0x7f2bf4144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf4161c50 con 0x7f2bf41628a0 2024-05-08T23:30:51.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf4166210 con 0x7f2bf40cb3b0 2024-05-08T23:30:51.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf4144ac0 con 0x7f2bf4165240 2024-05-08T23:30:51.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf2d9d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2bf41628a0 0x7f2bf4144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf2d9d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2bf41628a0 0x7f2bf4144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51326/0 (socket says 172.21.15.81:51326) 2024-05-08T23:30:51.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf2d9d700 1 -- 172.21.15.81:0/1308023162 learned_addr learned my addr 172.21.15.81:0/1308023162 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:51.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf1d9b700 1 --2- 172.21.15.81:0/1308023162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bf4165240 0x7f2bf4162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf259c700 1 --2- 172.21.15.81:0/1308023162 >> v2:172.21.15.117:3300/0 conn(0x7f2bf40cb3b0 0x7f2bf41478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf2d9d700 1 -- 172.21.15.81:0/1308023162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bf4165240 msgr2=0x7f2bf4162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf2d9d700 1 --2- 172.21.15.81:0/1308023162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bf4165240 0x7f2bf4162360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf2d9d700 1 -- 172.21.15.81:0/1308023162 >> v2:172.21.15.117:3300/0 conn(0x7f2bf40cb3b0 msgr2=0x7f2bf41478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf2d9d700 1 --2- 172.21.15.81:0/1308023162 >> v2:172.21.15.117:3300/0 conn(0x7f2bf40cb3b0 0x7f2bf41478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf2d9d700 1 -- 172.21.15.81:0/1308023162 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf41656a0 con 0x7f2bf41628a0 2024-05-08T23:30:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf259c700 1 --2- 172.21.15.81:0/1308023162 >> v2:172.21.15.117:3300/0 conn(0x7f2bf40cb3b0 0x7f2bf41478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf2d9d700 1 --2- 172.21.15.81:0/1308023162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf41628a0 0x7f2bf4144d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2bf406c110 tx=0x7f2be4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e390a78ecdb5fd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf159a700 1 -- 172.21.15.81:0/1308023162 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be400ba20 con 0x7f2bf41628a0 2024-05-08T23:30:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf159a700 1 -- 172.21.15.81:0/1308023162 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be400bb80 con 0x7f2bf41628a0 2024-05-08T23:30:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf159a700 1 -- 172.21.15.81:0/1308023162 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4009e60 con 0x7f2bf41628a0 2024-05-08T23:30:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 -- 172.21.15.81:0/1308023162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf41628a0 msgr2=0x7f2bf4144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 --2- 172.21.15.81:0/1308023162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf41628a0 0x7f2bf4144d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2bf406c110 tx=0x7f2be4009c20 comp rx=0 tx=0).stop 2024-05-08T23:30:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 -- 172.21.15.81:0/1308023162 shutdown_connections 2024-05-08T23:30:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 --2- 172.21.15.81:0/1308023162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bf4165240 0x7f2bf4162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 --2- 172.21.15.81:0/1308023162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf41628a0 0x7f2bf4144d40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 --2- 172.21.15.81:0/1308023162 >> v2:172.21.15.117:3300/0 conn(0x7f2bf40cb3b0 0x7f2bf41478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 -- 172.21.15.81:0/1308023162 >> 172.21.15.81:0/1308023162 conn(0x7f2bf406a360 msgr2=0x7f2bf406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 -- 172.21.15.81:0/1308023162 shutdown_connections 2024-05-08T23:30:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.081+0000 7f2bf89d2700 1 -- 172.21.15.81:0/1308023162 wait complete. 2024-05-08T23:30:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.085+0000 7f2bf89d2700 1 Processor -- start 2024-05-08T23:30:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.085+0000 7f2bf89d2700 1 -- start start 2024-05-08T23:30:51.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.085+0000 7f2bf89d2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf40cb3b0 0x7f2bf415d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.085+0000 7f2bf89d2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bf41628a0 0x7f2bf4145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.085+0000 7f2bf89d2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf4165240 0x7f2bf4145b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.085+0000 7f2bf89d2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf4160340 con 0x7f2bf40cb3b0 2024-05-08T23:30:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.085+0000 7f2bf89d2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf4147bc0 con 0x7f2bf4165240 2024-05-08T23:30:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.085+0000 7f2bf89d2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf4146090 con 0x7f2bf41628a0 2024-05-08T23:30:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.085+0000 7f2bf259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf40cb3b0 0x7f2bf415d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.085+0000 7f2bf259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf40cb3b0 0x7f2bf415d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51336/0 (socket says 172.21.15.81:51336) 2024-05-08T23:30:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.085+0000 7f2bf259c700 1 -- 172.21.15.81:0/1062489396 learned_addr learned my addr 172.21.15.81:0/1062489396 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.085+0000 7f2bf2d9d700 1 --2- 172.21.15.81:0/1062489396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf4165240 0x7f2bf4145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.085+0000 7f2bf1d9b700 1 --2- 172.21.15.81:0/1062489396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bf41628a0 0x7f2bf4145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2bf259c700 1 -- 172.21.15.81:0/1062489396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bf41628a0 msgr2=0x7f2bf4145610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2bf259c700 1 --2- 172.21.15.81:0/1062489396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bf41628a0 0x7f2bf4145610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2bf259c700 1 -- 172.21.15.81:0/1062489396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf4165240 msgr2=0x7f2bf4145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2bf259c700 1 --2- 172.21.15.81:0/1062489396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf4165240 0x7f2bf4145b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2bf259c700 1 -- 172.21.15.81:0/1062489396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be4009570 con 0x7f2bf40cb3b0 2024-05-08T23:30:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2bf1d9b700 1 --2- 172.21.15.81:0/1062489396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bf41628a0 0x7f2bf4145610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2bf259c700 1 --2- 172.21.15.81:0/1062489396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf40cb3b0 0x7f2bf415d580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2be800e880 tx=0x7f2be800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2be2ffd700 1 -- 172.21.15.81:0/1062489396 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8013610 con 0x7f2bf40cb3b0 2024-05-08T23:30:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2be2ffd700 1 -- 172.21.15.81:0/1062489396 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be8013770 con 0x7f2bf40cb3b0 2024-05-08T23:30:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2bf89d2700 1 -- 172.21.15.81:0/1062489396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf4142510 con 0x7f2bf40cb3b0 2024-05-08T23:30:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2be2ffd700 1 -- 172.21.15.81:0/1062489396 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8013b00 con 0x7f2bf40cb3b0 2024-05-08T23:30:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2bf89d2700 1 -- 172.21.15.81:0/1062489396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf415fc80 con 0x7f2bf40cb3b0 2024-05-08T23:30:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2bf89d2700 1 -- 172.21.15.81:0/1062489396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf41605b0 con 0x7f2bf40cb3b0 2024-05-08T23:30:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2be2ffd700 1 -- 172.21.15.81:0/1062489396 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2be8013c60 con 0x7f2bf40cb3b0 2024-05-08T23:30:51.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2be2ffd700 1 --2- 172.21.15.81:0/1062489396 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2bd4066730 0x7f2bd4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.089+0000 7f2bf1d9b700 1 --2- 172.21.15.81:0/1062489396 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2bd4066730 0x7f2bd4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.093+0000 7f2be2ffd700 1 -- 172.21.15.81:0/1062489396 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2be8083c90 con 0x7f2bf40cb3b0 2024-05-08T23:30:51.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.093+0000 7f2bf1d9b700 1 --2- 172.21.15.81:0/1062489396 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2bd4066730 0x7f2bd4068bf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2bec003fd0 tx=0x7f2bec003a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:51.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.097+0000 7f2be2ffd700 1 -- 172.21.15.81:0/1062489396 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2be8051090 con 0x7f2bf40cb3b0 2024-05-08T23:30:51.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.301+0000 7f2bf89d2700 1 -- 172.21.15.81:0/1062489396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bf40684b0 con 0x7f2bf40cb3b0 2024-05-08T23:30:51.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.305+0000 7f2be2ffd700 1 -- 172.21.15.81:0/1062489396 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7f2be80546b0 con 0x7f2bf40cb3b0 2024-05-08T23:30:51.308 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:30:51.308 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1475726755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1475726755},{"type":"v1","addr":"172.21.15.81:6835","nonce":1475726755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3535820540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3535820540},{"type":"v1","addr":"172.21.15.81:6837","nonce":3535820540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1531019955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1531019955},{"type":"v1","addr":"172.21.15.117:6833","nonce":1531019955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:30:38.665368+0000","modified":"2024-05-08T23:30:41.672722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/491911303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":491911303},{"type":"v1","addr":"172.21.15.117:6835","nonce":491911303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:30:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.305+0000 7f2bf89d2700 1 -- 172.21.15.81:0/1062489396 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2bd4066730 msgr2=0x7f2bd4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.305+0000 7f2bf89d2700 1 --2- 172.21.15.81:0/1062489396 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2bd4066730 0x7f2bd4068bf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2bec003fd0 tx=0x7f2bec003a40 comp rx=0 tx=0).stop 2024-05-08T23:30:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.305+0000 7f2bf89d2700 1 -- 172.21.15.81:0/1062489396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf40cb3b0 msgr2=0x7f2bf415d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.305+0000 7f2bf89d2700 1 --2- 172.21.15.81:0/1062489396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf40cb3b0 0x7f2bf415d580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2be800e880 tx=0x7f2be800eb90 comp rx=0 tx=0).stop 2024-05-08T23:30:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.305+0000 7f2bf89d2700 1 -- 172.21.15.81:0/1062489396 shutdown_connections 2024-05-08T23:30:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.305+0000 7f2bf89d2700 1 --2- 172.21.15.81:0/1062489396 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2bd4066730 0x7f2bd4068bf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.305+0000 7f2bf89d2700 1 --2- 172.21.15.81:0/1062489396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf4165240 0x7f2bf4145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.305+0000 7f2bf89d2700 1 --2- 172.21.15.81:0/1062489396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bf41628a0 0x7f2bf4145610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.305+0000 7f2bf89d2700 1 --2- 172.21.15.81:0/1062489396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bf40cb3b0 0x7f2bf415d580 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.305+0000 7f2bf89d2700 1 -- 172.21.15.81:0/1062489396 >> 172.21.15.81:0/1062489396 conn(0x7f2bf406a360 msgr2=0x7f2bf4164700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.305+0000 7f2bf89d2700 1 -- 172.21.15.81:0/1062489396 shutdown_connections 2024-05-08T23:30:51.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.305+0000 7f2bf89d2700 1 -- 172.21.15.81:0/1062489396 wait complete. 2024-05-08T23:30:51.312 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 7 2024-05-08T23:30:51.726 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:30:38.665368+0000', 'modified': '2024-05-08T23:30:41.672722+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4297}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4297': {'gid': 4297, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6835/491911303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 491911303}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 491911303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:30:51.726 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T23:30:51.726 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T23:30:51.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.841+0000 7fddc2e83700 1 Processor -- start 2024-05-08T23:30:51.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.841+0000 7fddc2e83700 1 -- start start 2024-05-08T23:30:51.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc2e83700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fddbc0ce990 0x7fddbc15f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc2e83700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddbc15f770 0x7fddbc0b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc2e83700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddbc15eda0 con 0x7fddbc15afa0 2024-05-08T23:30:51.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc2e83700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddbc15ea70 con 0x7fddbc0ce990 2024-05-08T23:30:51.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc2e83700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc15ba90 con 0x7fddbc15f770 2024-05-08T23:30:51.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc1680700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fddbc0ce990 0x7fddbc15f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc1680700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fddbc0ce990 0x7fddbc15f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51352/0 (socket says 172.21.15.81:51352) 2024-05-08T23:30:51.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc1680700 1 -- 172.21.15.81:0/463216853 learned_addr learned my addr 172.21.15.81:0/463216853 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:51.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc2682700 1 --2- 172.21.15.81:0/463216853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddbc15f770 0x7fddbc0b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc1680700 1 -- 172.21.15.81:0/463216853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddbc15f770 msgr2=0x7fddbc0b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:51.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc1680700 1 --2- 172.21.15.81:0/463216853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddbc15f770 0x7fddbc0b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc1680700 1 -- 172.21.15.81:0/463216853 >> v1:172.21.15.117:6789/0 conn(0x7fddbc15afa0 legacy=0x7fddbc15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:51.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc1680700 1 -- 172.21.15.81:0/463216853 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddbc15bd40 con 0x7fddbc0ce990 2024-05-08T23:30:51.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc1680700 1 --2- 172.21.15.81:0/463216853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc0ce990 0x7fddbc15f230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009910 tx=0x7fddb0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda0efd04b40672f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:51.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc0e7f700 1 -- 172.21.15.81:0/463216853 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0011830 con 0x7fddbc0ce990 2024-05-08T23:30:51.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc0e7f700 1 -- 172.21.15.81:0/463216853 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb0011990 con 0x7fddbc0ce990 2024-05-08T23:30:51.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc0e7f700 1 -- 172.21.15.81:0/463216853 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0011d40 con 0x7fddbc0ce990 2024-05-08T23:30:51.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc2e83700 1 -- 172.21.15.81:0/463216853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc0ce990 msgr2=0x7fddbc15f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:51.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc2e83700 1 --2- 172.21.15.81:0/463216853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc0ce990 0x7fddbc15f230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009910 tx=0x7fddb0009c20 comp rx=0 tx=0).stop 2024-05-08T23:30:51.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc2e83700 1 -- 172.21.15.81:0/463216853 shutdown_connections 2024-05-08T23:30:51.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc2e83700 1 --2- 172.21.15.81:0/463216853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddbc15f770 0x7fddbc0b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.852 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc2e83700 1 --2- 172.21.15.81:0/463216853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc0ce990 0x7fddbc15f230 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.852 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc2e83700 1 -- 172.21.15.81:0/463216853 >> 172.21.15.81:0/463216853 conn(0x7fddbc06c330 msgr2=0x7fddbc06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:51.852 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc2e83700 1 -- 172.21.15.81:0/463216853 shutdown_connections 2024-05-08T23:30:51.852 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.845+0000 7fddc2e83700 1 -- 172.21.15.81:0/463216853 wait complete. 2024-05-08T23:30:51.852 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2e83700 1 Processor -- start 2024-05-08T23:30:51.853 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2e83700 1 -- start start 2024-05-08T23:30:51.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2e83700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddbc0ce990 0x7fddbc152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2e83700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddbc15afa0 0x7fddbc147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2e83700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc15f770 0x7fddbc1e5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2e83700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc0adc00 con 0x7fddbc15f770 2024-05-08T23:30:51.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2e83700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc0ad2f0 con 0x7fddbc15afa0 2024-05-08T23:30:51.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2e83700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc147d60 con 0x7fddbc0ce990 2024-05-08T23:30:51.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2682700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc15f770 0x7fddbc1e5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2682700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc15f770 0x7fddbc1e5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51362/0 (socket says 172.21.15.81:51362) 2024-05-08T23:30:51.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2682700 1 -- 172.21.15.81:0/3606071152 learned_addr learned my addr 172.21.15.81:0/3606071152 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:51.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc1e81700 1 --2- 172.21.15.81:0/3606071152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddbc0ce990 0x7fddbc152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc1680700 1 --2- 172.21.15.81:0/3606071152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddbc15afa0 0x7fddbc147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2682700 1 -- 172.21.15.81:0/3606071152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddbc0ce990 msgr2=0x7fddbc152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:51.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2682700 1 --2- 172.21.15.81:0/3606071152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddbc0ce990 0x7fddbc152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2682700 1 -- 172.21.15.81:0/3606071152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddbc15afa0 msgr2=0x7fddbc147820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:51.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2682700 1 --2- 172.21.15.81:0/3606071152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddbc15afa0 0x7fddbc147820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2682700 1 -- 172.21.15.81:0/3606071152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb0009570 con 0x7fddbc15f770 2024-05-08T23:30:51.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2682700 1 --2- 172.21.15.81:0/3606071152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc15f770 0x7fddbc1e5520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fddb400ef90 tx=0x7fddb4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:51.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddae7fc700 1 -- 172.21.15.81:0/3606071152 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb4009b20 con 0x7fddbc15f770 2024-05-08T23:30:51.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddae7fc700 1 -- 172.21.15.81:0/3606071152 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb400f040 con 0x7fddbc15f770 2024-05-08T23:30:51.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.849+0000 7fddc2e83700 1 -- 172.21.15.81:0/3606071152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddbc0b29a0 con 0x7fddbc15f770 2024-05-08T23:30:51.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.853+0000 7fddae7fc700 1 -- 172.21.15.81:0/3606071152 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb400cc70 con 0x7fddbc15f770 2024-05-08T23:30:51.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.853+0000 7fddc2e83700 1 -- 172.21.15.81:0/3606071152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddbc0ad820 con 0x7fddbc15f770 2024-05-08T23:30:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.853+0000 7fddae7fc700 1 -- 172.21.15.81:0/3606071152 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fddb4014480 con 0x7fddbc15f770 2024-05-08T23:30:51.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.853+0000 7fddae7fc700 1 --2- 172.21.15.81:0/3606071152 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdda0066730 0x7fdda0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.853+0000 7fddc1e81700 1 --2- 172.21.15.81:0/3606071152 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdda0066730 0x7fdda0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.853+0000 7fddae7fc700 1 -- 172.21.15.81:0/3606071152 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddb4084c70 con 0x7fddbc15f770 2024-05-08T23:30:51.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc1e81700 1 --2- 172.21.15.81:0/3606071152 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdda0066730 0x7fdda0068bf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fddbc1f64f0 tx=0x7fddb8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:51.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc7394740 1 Processor -- start 2024-05-08T23:30:51.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc7394740 1 -- start start 2024-05-08T23:30:51.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc7394740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd0f770 0xd0fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc7394740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd065c0 0xd10090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc7394740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd082a0 0xd08680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc7394740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0xc36d80 con 0xd082a0 2024-05-08T23:30:51.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc7394740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0xc34510 con 0xd065c0 2024-05-08T23:30:51.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc7394740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0xc2fe00 con 0xd0f770 2024-05-08T23:30:51.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc2682700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd082a0 0xd08680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc2682700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd082a0 0xd08680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51370/0 (socket says 172.21.15.81:51370) 2024-05-08T23:30:51.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc2682700 1 -- 172.21.15.81:0/2611088181 learned_addr learned my addr 172.21.15.81:0/2611088181 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:51.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc1680700 1 --2- 172.21.15.81:0/2611088181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd065c0 0xd10090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc1e81700 1 --2- 172.21.15.81:0/2611088181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd0f770 0xd0fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc2682700 1 -- 172.21.15.81:0/2611088181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd0f770 msgr2=0xd0fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:51.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc2682700 1 --2- 172.21.15.81:0/2611088181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd0f770 0xd0fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc2682700 1 -- 172.21.15.81:0/2611088181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd065c0 msgr2=0xd10090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:51.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc2682700 1 --2- 172.21.15.81:0/2611088181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd065c0 0xd10090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc2682700 1 -- 172.21.15.81:0/2611088181 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xaa2960 con 0xd082a0 2024-05-08T23:30:51.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fddc2682700 1 --2- 172.21.15.81:0/2611088181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd082a0 0xd08680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fddb405da40 tx=0x7fddb4055b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3fa8771d9df2cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:51.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fdd9f7fe700 1 -- 172.21.15.81:0/2611088181 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb406d070 con 0xd082a0 2024-05-08T23:30:51.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fdd9f7fe700 1 -- 172.21.15.81:0/2611088181 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb405df70 con 0xd082a0 2024-05-08T23:30:51.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.857+0000 7fdd9f7fe700 1 -- 172.21.15.81:0/2611088181 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb4053f20 con 0xd082a0 2024-05-08T23:30:51.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 -- 172.21.15.81:0/2611088181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd082a0 msgr2=0xd08680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:51.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 --2- 172.21.15.81:0/2611088181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd082a0 0xd08680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fddb405da40 tx=0x7fddb4055b80 comp rx=0 tx=0).stop 2024-05-08T23:30:51.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 -- 172.21.15.81:0/2611088181 shutdown_connections 2024-05-08T23:30:51.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 --2- 172.21.15.81:0/2611088181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd082a0 0xd08680 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 --2- 172.21.15.81:0/2611088181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd065c0 0xd10090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 --2- 172.21.15.81:0/2611088181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd0f770 0xd0fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 -- 172.21.15.81:0/2611088181 >> 172.21.15.81:0/2611088181 conn(0xa574d0 msgr2=0xcf1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:51.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 -- 172.21.15.81:0/2611088181 shutdown_connections 2024-05-08T23:30:51.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 -- 172.21.15.81:0/2611088181 wait complete. 2024-05-08T23:30:51.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 -- start start 2024-05-08T23:30:51.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 Processor -- start 2024-05-08T23:30:51.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd065c0 0xcee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd082a0 0xcc96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd0f770 0xcc9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0xbed730 con 0xd065c0 2024-05-08T23:30:51.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0xbf35b0 con 0xd0f770 2024-05-08T23:30:51.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0xadbaa0 con 0xd082a0 2024-05-08T23:30:51.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 10 client.0 _ll_register_callbacks cb 0xc29610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:51.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc1e81700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd065c0 0xcee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc1e81700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd065c0 0xcee3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51376/0 (socket says 172.21.15.81:51376) 2024-05-08T23:30:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc1e81700 1 -- 172.21.15.81:0/2184704416 learned_addr learned my addr 172.21.15.81:0/2184704416 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc2682700 1 --2- 172.21.15.81:0/2184704416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd0f770 0xcc9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc1680700 1 --2- 172.21.15.81:0/2184704416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd082a0 0xcc96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc1e81700 1 -- 172.21.15.81:0/2184704416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd082a0 msgr2=0xcc96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc1e81700 1 --2- 172.21.15.81:0/2184704416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd082a0 0xcc96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc1e81700 1 -- 172.21.15.81:0/2184704416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd0f770 msgr2=0xcc9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc1e81700 1 --2- 172.21.15.81:0/2184704416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd0f770 0xcc9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc1e81700 1 -- 172.21.15.81:0/2184704416 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddbc0ad820 con 0xd065c0 2024-05-08T23:30:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc1680700 1 --2- 172.21.15.81:0/2184704416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd082a0 0xcc96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc1e81700 1 --2- 172.21.15.81:0/2184704416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd065c0 0xcee3e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fddb8015460 tx=0x7fddb8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da44498989620e9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fdd9cff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-08T23:30:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fdd9cff9700 1 -- 172.21.15.81:0/2184704416 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8015c90 con 0xd065c0 2024-05-08T23:30:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fdd9cff9700 1 -- 172.21.15.81:0/2184704416 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddb8015df0 con 0xd065c0 2024-05-08T23:30:51.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 -- 172.21.15.81:0/2184704416 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xa641e0 con 0xd065c0 2024-05-08T23:30:51.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fdd9cff9700 1 -- 172.21.15.81:0/2184704416 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddb8021420 con 0xd065c0 2024-05-08T23:30:51.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 10 client.4635 fetch_fsmap learned FSMap version 7 2024-05-08T23:30:51.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.861+0000 7fddc7394740 1 -- 172.21.15.81:0/2184704416 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0xbf6220 con 0xd065c0 2024-05-08T23:30:51.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fdd9cff9700 1 -- 172.21.15.81:0/2184704416 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xa641e0 con 0xd065c0 2024-05-08T23:30:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fdd9cff9700 1 -- 172.21.15.81:0/2184704416 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fddb8021580 con 0xd065c0 2024-05-08T23:30:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 10 client.4635 fetch_fsmap finished waiting for FSMap version 7 2024-05-08T23:30:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 10 client.4635 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:30:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 20 client.4635 populate_metadata read hostname 'smithi081' 2024-05-08T23:30:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 1 --2- 172.21.15.81:0/2184704416 >> [v2:172.21.15.117:6834/491911303,v1:172.21.15.117:6835/491911303] conn(0xd09a70 0xd230a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 4 client.4635 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:30:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 1 -- 172.21.15.81:0/2184704416 --> [v2:172.21.15.117:6834/491911303,v1:172.21.15.117:6835/491911303] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xaae460 con 0xd09a70 2024-05-08T23:30:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc1680700 1 --2- 172.21.15.81:0/2184704416 >> [v2:172.21.15.117:6834/491911303,v1:172.21.15.117:6835/491911303] conn(0xd09a70 0xd230a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fdd9cff9700 10 client.4635 ms_handle_connect on v2:172.21.15.117:6834/491911303 2024-05-08T23:30:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc1680700 1 --2- 172.21.15.81:0/2184704416 >> [v2:172.21.15.117:6834/491911303,v1:172.21.15.117:6835/491911303] conn(0xd09a70 0xd230a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0954d5ad1d18d server_cookie=e809f1f629b49342 in_seq=0 out_seq=0 2024-05-08T23:30:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fdd9cff9700 1 -- 172.21.15.81:0/2184704416 <== mds.0 v2:172.21.15.117:6834/491911303 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0xaae460 con 0xd09a70 2024-05-08T23:30:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fdd9cff9700 10 client.4635 handle_command_reply: tid=0 2024-05-08T23:30:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 1 client.4635 shutdown 2024-05-08T23:30:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 1 -- 172.21.15.81:0/2184704416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd065c0 msgr2=0xcee3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 1 --2- 172.21.15.81:0/2184704416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd065c0 0xcee3e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fddb8015460 tx=0x7fddb8007f90 comp rx=0 tx=0).stop 2024-05-08T23:30:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 1 -- 172.21.15.81:0/2184704416 shutdown_connections 2024-05-08T23:30:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 1 --2- 172.21.15.81:0/2184704416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd0f770 0xcc9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 1 --2- 172.21.15.81:0/2184704416 >> [v2:172.21.15.117:6834/491911303,v1:172.21.15.117:6835/491911303] conn(0xd09a70 0xd230a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 1 --2- 172.21.15.81:0/2184704416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd082a0 0xcc96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 1 --2- 172.21.15.81:0/2184704416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd065c0 0xcee3e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fdd9cff9700 0 client.4635 ms_handle_reset on v2:172.21.15.117:6834/491911303 2024-05-08T23:30:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 1 -- 172.21.15.81:0/2184704416 >> 172.21.15.81:0/2184704416 conn(0xa574d0 msgr2=0xcf0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 1 -- 172.21.15.81:0/2184704416 shutdown_connections 2024-05-08T23:30:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 1 -- 172.21.15.81:0/2184704416 wait complete. 2024-05-08T23:30:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.865+0000 7fddc7394740 20 client.4635 trim_cache size 0 max 16384 2024-05-08T23:30:51.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc7394740 1 Processor -- start 2024-05-08T23:30:51.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc7394740 1 -- start start 2024-05-08T23:30:51.894 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc7394740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd065c0 0xced7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.894 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc7394740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd082a0 0xd0e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc7394740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd0f770 0xd0eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc7394740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0xc2eb20 con 0xd082a0 2024-05-08T23:30:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc7394740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0xbce470 con 0xd0f770 2024-05-08T23:30:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc7394740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0xc34cf0 con 0xd065c0 2024-05-08T23:30:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc1680700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd082a0 0xd0e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc1680700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd082a0 0xd0e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51378/0 (socket says 172.21.15.81:51378) 2024-05-08T23:30:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc1680700 1 -- 172.21.15.81:0/1902189349 learned_addr learned my addr 172.21.15.81:0/1902189349 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc1e81700 1 --2- 172.21.15.81:0/1902189349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd065c0 0xced7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc2682700 1 --2- 172.21.15.81:0/1902189349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd0f770 0xd0eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc1680700 1 -- 172.21.15.81:0/1902189349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd065c0 msgr2=0xced7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc1680700 1 --2- 172.21.15.81:0/1902189349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd065c0 0xced7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc1680700 1 -- 172.21.15.81:0/1902189349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd0f770 msgr2=0xd0eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc1680700 1 --2- 172.21.15.81:0/1902189349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd0f770 0xd0eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc1680700 1 -- 172.21.15.81:0/1902189349 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xcc8350 con 0xd082a0 2024-05-08T23:30:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fddc1680700 1 --2- 172.21.15.81:0/1902189349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd082a0 0xd0e640 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fddb0013000 tx=0x7fddb0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368cccccd180d583 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fdd9ffff700 1 -- 172.21.15.81:0/1902189349 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0020040 con 0xd082a0 2024-05-08T23:30:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fdd9ffff700 1 -- 172.21.15.81:0/1902189349 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb0021040 con 0xd082a0 2024-05-08T23:30:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.889+0000 7fdd9ffff700 1 -- 172.21.15.81:0/1902189349 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0027660 con 0xd082a0 2024-05-08T23:30:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.893+0000 7fddc7394740 1 -- 172.21.15.81:0/1902189349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd082a0 msgr2=0xd0e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.893+0000 7fddc7394740 1 --2- 172.21.15.81:0/1902189349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd082a0 0xd0e640 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fddb0013000 tx=0x7fddb0010000 comp rx=0 tx=0).stop 2024-05-08T23:30:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.893+0000 7fddc7394740 1 -- 172.21.15.81:0/1902189349 shutdown_connections 2024-05-08T23:30:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.893+0000 7fddc7394740 1 --2- 172.21.15.81:0/1902189349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd0f770 0xd0eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.893+0000 7fddc7394740 1 --2- 172.21.15.81:0/1902189349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd082a0 0xd0e640 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.893+0000 7fddc7394740 1 --2- 172.21.15.81:0/1902189349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd065c0 0xced7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.893+0000 7fddc7394740 1 -- 172.21.15.81:0/1902189349 >> 172.21.15.81:0/1902189349 conn(0xa574d0 msgr2=0xd07aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.893+0000 7fddc7394740 1 -- 172.21.15.81:0/1902189349 shutdown_connections 2024-05-08T23:30:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.893+0000 7fddc7394740 1 -- 172.21.15.81:0/1902189349 wait complete. 2024-05-08T23:30:51.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.893+0000 7fddc7394740 1 -- start start 2024-05-08T23:30:51.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.893+0000 7fddc7394740 1 Processor -- start 2024-05-08T23:30:51.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd065c0 0xd2ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd082a0 0xd2dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd0f770 0xd70bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0xd2d300 con 0xd0f770 2024-05-08T23:30:51.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0xd2e130 con 0xd065c0 2024-05-08T23:30:51.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0xcf1ae0 con 0xd082a0 2024-05-08T23:30:51.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 10 client.0 _ll_register_callbacks cb 0xc2e9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:51.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc2682700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd0f770 0xd70bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc2682700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd0f770 0xd70bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51390/0 (socket says 172.21.15.81:51390) 2024-05-08T23:30:51.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc2682700 1 -- 172.21.15.81:0/3517295287 learned_addr learned my addr 172.21.15.81:0/3517295287 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:51.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc1e81700 1 --2- 172.21.15.81:0/3517295287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd065c0 0xd2ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc1680700 1 --2- 172.21.15.81:0/3517295287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd082a0 0xd2dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc2682700 1 -- 172.21.15.81:0/3517295287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd082a0 msgr2=0xd2dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:51.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc2682700 1 --2- 172.21.15.81:0/3517295287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd082a0 0xd2dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc2682700 1 -- 172.21.15.81:0/3517295287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd065c0 msgr2=0xd2ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:51.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc2682700 1 --2- 172.21.15.81:0/3517295287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd065c0 0xd2ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc2682700 1 -- 172.21.15.81:0/3517295287 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddbc15bd40 con 0xd0f770 2024-05-08T23:30:51.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc2682700 1 --2- 172.21.15.81:0/3517295287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd0f770 0xd70bb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fddb4059b70 tx=0x7fddb401f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97244f60451fb9f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:51.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fdd9cff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-08T23:30:51.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fdd9cff9700 1 -- 172.21.15.81:0/3517295287 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb406d040 con 0xd0f770 2024-05-08T23:30:51.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fdd9cff9700 1 -- 172.21.15.81:0/3517295287 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddb4055de0 con 0xd0f770 2024-05-08T23:30:51.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 1 -- 172.21.15.81:0/3517295287 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fddb405df70 con 0xd0f770 2024-05-08T23:30:51.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fdd9cff9700 1 -- 172.21.15.81:0/3517295287 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddb40554e0 con 0xd0f770 2024-05-08T23:30:51.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 10 client.4641 fetch_fsmap learned FSMap version 7 2024-05-08T23:30:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 1 -- 172.21.15.81:0/3517295287 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0xbdc450 con 0xd0f770 2024-05-08T23:30:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fdd9cff9700 1 -- 172.21.15.81:0/3517295287 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb405df70 con 0xd0f770 2024-05-08T23:30:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fdd9cff9700 1 -- 172.21.15.81:0/3517295287 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fddb4055070 con 0xd0f770 2024-05-08T23:30:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 10 client.4641 fetch_fsmap finished waiting for FSMap version 7 2024-05-08T23:30:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 10 client.4641 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:30:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 20 client.4641 populate_metadata read hostname 'smithi081' 2024-05-08T23:30:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 1 --2- 172.21.15.81:0/3517295287 >> [v2:172.21.15.117:6834/491911303,v1:172.21.15.117:6835/491911303] conn(0xd23f10 0xd242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 4 client.4641 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:30:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc7394740 1 -- 172.21.15.81:0/3517295287 --> [v2:172.21.15.117:6834/491911303,v1:172.21.15.117:6835/491911303] -- command(tid 0: {"prefix": "status"}) v1 -- 0xad31e0 con 0xd23f10 2024-05-08T23:30:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc1e81700 1 --2- 172.21.15.81:0/3517295287 >> [v2:172.21.15.117:6834/491911303,v1:172.21.15.117:6835/491911303] conn(0xd23f10 0xd242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fdd9cff9700 10 client.4641 ms_handle_connect on v2:172.21.15.117:6834/491911303 2024-05-08T23:30:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.897+0000 7fddc1e81700 1 --2- 172.21.15.81:0/3517295287 >> [v2:172.21.15.117:6834/491911303,v1:172.21.15.117:6835/491911303] conn(0xd23f10 0xd242f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90abf4b14fedefc0 server_cookie=5920053925ef5124 in_seq=0 out_seq=0 2024-05-08T23:30:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fdd9cff9700 1 -- 172.21.15.81:0/3517295287 <== mds.0 v2:172.21.15.117:6834/491911303 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0xad31e0 con 0xd23f10 2024-05-08T23:30:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fdd9cff9700 10 client.4641 handle_command_reply: tid=0 2024-05-08T23:30:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc7394740 1 client.4641 shutdown 2024-05-08T23:30:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc7394740 1 -- 172.21.15.81:0/3517295287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd0f770 msgr2=0xd70bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc7394740 1 --2- 172.21.15.81:0/3517295287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd0f770 0xd70bb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fddb4059b70 tx=0x7fddb401f070 comp rx=0 tx=0).stop 2024-05-08T23:30:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc7394740 1 -- 172.21.15.81:0/3517295287 shutdown_connections 2024-05-08T23:30:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc7394740 1 --2- 172.21.15.81:0/3517295287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0xd0f770 0xd70bb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc7394740 1 --2- 172.21.15.81:0/3517295287 >> [v2:172.21.15.117:6834/491911303,v1:172.21.15.117:6835/491911303] conn(0xd23f10 0xd242f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc7394740 1 --2- 172.21.15.81:0/3517295287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0xd082a0 0xd2dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc7394740 1 --2- 172.21.15.81:0/3517295287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0xd065c0 0xd2ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fdd9cff9700 0 client.4641 ms_handle_reset on v2:172.21.15.117:6834/491911303 2024-05-08T23:30:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc7394740 1 -- 172.21.15.81:0/3517295287 >> 172.21.15.81:0/3517295287 conn(0xa574d0 msgr2=0xd0eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc7394740 1 -- 172.21.15.81:0/3517295287 shutdown_connections 2024-05-08T23:30:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc7394740 1 -- 172.21.15.81:0/3517295287 wait complete. 2024-05-08T23:30:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc7394740 20 client.4641 trim_cache size 0 max 16384 2024-05-08T23:30:51.907 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:30:51.907 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_fsid": "dee3de5e-bc15-439a-9578-b5583aefc890", 2024-05-08T23:30:51.907 INFO:teuthology.orchestra.run.smithi081.stdout: "whoami": 0, 2024-05-08T23:30:51.907 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 4297, 2024-05-08T23:30:51.907 INFO:teuthology.orchestra.run.smithi081.stdout: "want_state": "up:active", 2024-05-08T23:30:51.907 INFO:teuthology.orchestra.run.smithi081.stdout: "state": "up:active", 2024-05-08T23:30:51.907 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs", 2024-05-08T23:30:51.907 INFO:teuthology.orchestra.run.smithi081.stdout: "rank_uptime": 13.234652719, 2024-05-08T23:30:51.907 INFO:teuthology.orchestra.run.smithi081.stdout: "mdsmap_epoch": 7, 2024-05-08T23:30:51.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch": 22, 2024-05-08T23:30:51.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch_barrier": 22, 2024-05-08T23:30:51.907 INFO:teuthology.orchestra.run.smithi081.stdout: "uptime": 19.974087819000001 2024-05-08T23:30:51.908 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:30:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc2e83700 1 -- 172.21.15.81:0/3606071152 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdda0066730 msgr2=0x7fdda0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc2e83700 1 --2- 172.21.15.81:0/3606071152 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdda0066730 0x7fdda0068bf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fddbc1f64f0 tx=0x7fddb8009040 comp rx=0 tx=0).stop 2024-05-08T23:30:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc2e83700 1 -- 172.21.15.81:0/3606071152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc15f770 msgr2=0x7fddbc1e5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc2e83700 1 --2- 172.21.15.81:0/3606071152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc15f770 0x7fddbc1e5520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fddb400ef90 tx=0x7fddb4009f60 comp rx=0 tx=0).stop 2024-05-08T23:30:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc2e83700 1 -- 172.21.15.81:0/3606071152 shutdown_connections 2024-05-08T23:30:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc2e83700 1 --2- 172.21.15.81:0/3606071152 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdda0066730 0x7fdda0068bf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc2e83700 1 --2- 172.21.15.81:0/3606071152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc15f770 0x7fddbc1e5520 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc2e83700 1 --2- 172.21.15.81:0/3606071152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddbc15afa0 0x7fddbc147820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc2e83700 1 --2- 172.21.15.81:0/3606071152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddbc0ce990 0x7fddbc152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc2e83700 1 -- 172.21.15.81:0/3606071152 >> 172.21.15.81:0/3606071152 conn(0x7fddbc06c330 msgr2=0x7fddbc0aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc2e83700 1 -- 172.21.15.81:0/3606071152 shutdown_connections 2024-05-08T23:30:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:51.901+0000 7fddc2e83700 1 -- 172.21.15.81:0/3606071152 wait complete. 2024-05-08T23:30:51.921 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T23:30:51.931 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T23:30:51.931 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T23:30:51.931 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T23:30:51.931 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-08T23:30:52.034 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-08T23:30:52.035 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-08T23:30:52.035 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-08T23:30:52.035 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T23:30:52.035 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:30:52.035 INFO:tasks.ceph_fuse:client.2 config is {} 2024-05-08T23:30:52.035 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T23:30:52.035 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-08T23:30:52.035 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-08T23:30:52.035 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T23:30:52.035 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T23:30:52.036 INFO:tasks.ceph_fuse:client.3 config is {} 2024-05-08T23:30:52.036 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T23:30:52.036 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-08T23:30:52.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:52.036 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-05-08T23:30:52.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:52.045 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link delete ceph-brx 2024-05-08T23:30:52.107 INFO:teuthology.orchestra.run.smithi081.stderr:Cannot find device "ceph-brx" 2024-05-08T23:30:52.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:30:52.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:52.110 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-05-08T23:30:52.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:52.118 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-05-08T23:30:52.137 INFO:teuthology.orchestra.run.smithi117.stderr:Cannot find device "ceph-brx" 2024-05-08T23:30:52.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:30:52.139 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-08T23:30:52.139 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:30:52.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.249+0000 7fe87d14c700 1 Processor -- start 2024-05-08T23:30:52.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.249+0000 7fe87d14c700 1 -- start start 2024-05-08T23:30:52.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe87d14c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe87806ba80 0x7fe87815b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:52.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe87d14c700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87815f2c0 con 0x7fe8780cce10 2024-05-08T23:30:52.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe87d14c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8780cddb0 con 0x7fe87815b280 2024-05-08T23:30:52.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe87d14c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8780cd7f0 con 0x7fe87806ba80 2024-05-08T23:30:52.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe87659c700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe8780cce10 0x7fe87806b370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38790/0 (socket says 172.21.15.81:38790) 2024-05-08T23:30:52.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe87659c700 1 -- 172.21.15.81:0/2797853689 learned_addr learned my addr 172.21.15.81:0/2797853689 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:52.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe87759e700 1 --2- 172.21.15.81:0/2797853689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe87806ba80 0x7fe87815b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:52.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe875d9b700 1 -- 172.21.15.81:0/2797853689 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296475169 0 0) 0x7fe87815f2c0 con 0x7fe8780cce10 2024-05-08T23:30:52.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe875d9b700 1 -- 172.21.15.81:0/2797853689 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8600035d0 con 0x7fe8780cce10 2024-05-08T23:30:52.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe875d9b700 1 -- 172.21.15.81:0/2797853689 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 451632146 0 0) 0x7fe8780cddb0 con 0x7fe87815b280 2024-05-08T23:30:52.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe875d9b700 1 -- 172.21.15.81:0/2797853689 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe87815f2c0 con 0x7fe87815b280 2024-05-08T23:30:52.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe875d9b700 1 -- 172.21.15.81:0/2797853689 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3049951208 0 0) 0x7fe8600035d0 con 0x7fe8780cce10 2024-05-08T23:30:52.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe87759e700 1 -- 172.21.15.81:0/2797853689 >> v1:172.21.15.117:6789/0 conn(0x7fe87815b280 legacy=0x7fe87815f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:52.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe87759e700 1 -- 172.21.15.81:0/2797853689 >> v1:172.21.15.81:6789/0 conn(0x7fe8780cce10 legacy=0x7fe87806b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:52.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe87759e700 1 -- 172.21.15.81:0/2797853689 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8780cdad0 con 0x7fe87806ba80 2024-05-08T23:30:52.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe87759e700 1 --2- 172.21.15.81:0/2797853689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe87806ba80 0x7fe87815b660 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8700098b0 tx=0x7fe870009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2118d660c6e1fea4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:52.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe875d9b700 1 -- 172.21.15.81:0/2797853689 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c001670 con 0x7fe87806ba80 2024-05-08T23:30:52.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe875d9b700 1 -- 172.21.15.81:0/2797853689 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe870004030 con 0x7fe87806ba80 2024-05-08T23:30:52.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.253+0000 7fe875d9b700 1 -- 172.21.15.81:0/2797853689 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8700043e0 con 0x7fe87806ba80 2024-05-08T23:30:52.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.257+0000 7fe87d14c700 1 -- 172.21.15.81:0/2797853689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe87806ba80 msgr2=0x7fe87815b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:52.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.257+0000 7fe87d14c700 1 --2- 172.21.15.81:0/2797853689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe87806ba80 0x7fe87815b660 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8700098b0 tx=0x7fe870009bc0 comp rx=0 tx=0).stop 2024-05-08T23:30:52.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.257+0000 7fe87d14c700 1 -- 172.21.15.81:0/2797853689 shutdown_connections 2024-05-08T23:30:52.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.257+0000 7fe87d14c700 1 --2- 172.21.15.81:0/2797853689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe87806ba80 0x7fe87815b660 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:52.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.257+0000 7fe87d14c700 1 -- 172.21.15.81:0/2797853689 >> 172.21.15.81:0/2797853689 conn(0x7fe8780b1c00 msgr2=0x7fe878070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:52.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.257+0000 7fe87d14c700 1 -- 172.21.15.81:0/2797853689 shutdown_connections 2024-05-08T23:30:52.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.257+0000 7fe87d14c700 1 -- 172.21.15.81:0/2797853689 wait complete. 2024-05-08T23:30:52.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.257+0000 7fe87d14c700 1 Processor -- start 2024-05-08T23:30:52.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.257+0000 7fe87d14c700 1 -- start start 2024-05-08T23:30:52.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87d14c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe87806ba80 0x7fe87814c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:52.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87d14c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8780cce10 0x7fe8780b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:52.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87d14c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe87815b280 0x7fe8780afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:52.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87d14c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87815c590 con 0x7fe87815b280 2024-05-08T23:30:52.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87d14c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8781f6b80 con 0x7fe87806ba80 2024-05-08T23:30:52.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87d14c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8781539b0 con 0x7fe8780cce10 2024-05-08T23:30:52.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe87815b280 0x7fe8780afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:52.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe87815b280 0x7fe8780afdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51406/0 (socket says 172.21.15.81:51406) 2024-05-08T23:30:52.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87759e700 1 -- 172.21.15.81:0/4004179396 learned_addr learned my addr 172.21.15.81:0/4004179396 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:52.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe876d9d700 1 --2- 172.21.15.81:0/4004179396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe87806ba80 0x7fe87814c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:52.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87659c700 1 --2- 172.21.15.81:0/4004179396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8780cce10 0x7fe8780b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:52.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87759e700 1 -- 172.21.15.81:0/4004179396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8780cce10 msgr2=0x7fe8780b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:52.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87759e700 1 --2- 172.21.15.81:0/4004179396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8780cce10 0x7fe8780b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:52.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87759e700 1 -- 172.21.15.81:0/4004179396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe87806ba80 msgr2=0x7fe87814c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:52.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87759e700 1 --2- 172.21.15.81:0/4004179396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe87806ba80 0x7fe87814c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:52.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87759e700 1 -- 172.21.15.81:0/4004179396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8700094b0 con 0x7fe87815b280 2024-05-08T23:30:52.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87759e700 1 --2- 172.21.15.81:0/4004179396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe87815b280 0x7fe8780afdb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe870004000 tx=0x7fe870004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:52.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe85f7fe700 1 -- 172.21.15.81:0/4004179396 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870006ea0 con 0x7fe87815b280 2024-05-08T23:30:52.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe85f7fe700 1 -- 172.21.15.81:0/4004179396 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87001f070 con 0x7fe87815b280 2024-05-08T23:30:52.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87d14c700 1 -- 172.21.15.81:0/4004179396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8781f6500 con 0x7fe87815b280 2024-05-08T23:30:52.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe85f7fe700 1 -- 172.21.15.81:0/4004179396 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8700045b0 con 0x7fe87815b280 2024-05-08T23:30:52.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87d14c700 1 -- 172.21.15.81:0/4004179396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8781f67a0 con 0x7fe87815b280 2024-05-08T23:30:52.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.261+0000 7fe87d14c700 1 -- 172.21.15.81:0/4004179396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8780cd260 con 0x7fe87815b280 2024-05-08T23:30:52.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.265+0000 7fe85f7fe700 1 -- 172.21.15.81:0/4004179396 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe870021030 con 0x7fe87815b280 2024-05-08T23:30:52.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.265+0000 7fe85f7fe700 1 --2- 172.21.15.81:0/4004179396 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe864066730 0x7fe864068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:52.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.265+0000 7fe85f7fe700 1 -- 172.21.15.81:0/4004179396 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe870085360 con 0x7fe87815b280 2024-05-08T23:30:52.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.265+0000 7fe876d9d700 1 --2- 172.21.15.81:0/4004179396 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe864066730 0x7fe864068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:52.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.265+0000 7fe876d9d700 1 --2- 172.21.15.81:0/4004179396 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe864066730 0x7fe864068bf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe868001560 tx=0x7fe868009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:52.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.273+0000 7fe85f7fe700 1 -- 172.21.15.81:0/4004179396 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe8700526c0 con 0x7fe87815b280 2024-05-08T23:30:52.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.473+0000 7fe87d14c700 1 -- 172.21.15.81:0/4004179396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe8781f6cc0 con 0x7fe87815b280 2024-05-08T23:30:52.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.473+0000 7fe85f7fe700 1 -- 172.21.15.81:0/4004179396 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe870055ce0 con 0x7fe87815b280 2024-05-08T23:30:52.478 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:30:52.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.477+0000 7fe87d14c700 1 -- 172.21.15.81:0/4004179396 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe864066730 msgr2=0x7fe864068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:52.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.477+0000 7fe87d14c700 1 --2- 172.21.15.81:0/4004179396 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe864066730 0x7fe864068bf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe868001560 tx=0x7fe868009040 comp rx=0 tx=0).stop 2024-05-08T23:30:52.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.477+0000 7fe87d14c700 1 -- 172.21.15.81:0/4004179396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe87815b280 msgr2=0x7fe8780afdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:52.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.477+0000 7fe87d14c700 1 --2- 172.21.15.81:0/4004179396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe87815b280 0x7fe8780afdb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe870004000 tx=0x7fe870004770 comp rx=0 tx=0).stop 2024-05-08T23:30:52.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.477+0000 7fe87d14c700 1 -- 172.21.15.81:0/4004179396 shutdown_connections 2024-05-08T23:30:52.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.477+0000 7fe87d14c700 1 --2- 172.21.15.81:0/4004179396 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe864066730 0x7fe864068bf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:52.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.477+0000 7fe87d14c700 1 --2- 172.21.15.81:0/4004179396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe87815b280 0x7fe8780afdb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:52.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.477+0000 7fe87d14c700 1 --2- 172.21.15.81:0/4004179396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8780cce10 0x7fe8780b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:52.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.477+0000 7fe87d14c700 1 --2- 172.21.15.81:0/4004179396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe87806ba80 0x7fe87814c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:52.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.477+0000 7fe87d14c700 1 -- 172.21.15.81:0/4004179396 >> 172.21.15.81:0/4004179396 conn(0x7fe8780b1c00 msgr2=0x7fe87806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:52.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.477+0000 7fe87d14c700 1 -- 172.21.15.81:0/4004179396 shutdown_connections 2024-05-08T23:30:52.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:52.477+0000 7fe87d14c700 1 -- 172.21.15.81:0/4004179396 wait complete. 2024-05-08T23:30:52.498 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:30:52.498 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:30:52.499 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2024-05-08T23:30:52.499 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:30:52.499 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:30:52.499 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:30:52.499 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:30:52.499 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:30:52.499 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:30:52.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:52.499 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2024-05-08T23:30:52.505 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:30:52.505 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:30:52.506 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:30:52.506 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:52.506 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-05-08T23:30:52.506 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:52.506 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:30:52.506 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:52.506 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:30:52.506 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:52.506 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:30:52.506 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:52.506 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:30:52.506 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:52.506 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T23:30:52.507 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42563sec preferred_lft 42563sec 2024-05-08T23:30:52.507 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2024-05-08T23:30:52.507 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:52.507 INFO:teuthology.orchestra.run.smithi081.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T23:30:52.507 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 02:42:22:62:7a:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:52.507 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T23:30:52.507 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:52.507 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T23:30:52.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:52.508 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-05-08T23:30:52.508 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-05-08T23:30:52.508 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link add name ceph-brx type bridge 2024-05-08T23:30:52.508 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip addr flush dev ceph-brx 2024-05-08T23:30:52.508 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set ceph-brx up 2024-05-08T23:30:52.508 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T23:30:52.508 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-05-08T23:30:52.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:52.593 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:30:52.605 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-05-08T23:30:52.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:52.607 DEBUG:teuthology.orchestra.run.smithi081:> route 2024-05-08T23:30:52.638 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2024-05-08T23:30:52.638 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:30:52.638 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T23:30:52.638 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T23:30:52.638 INFO:teuthology.orchestra.run.smithi081.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T23:30:52.638 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T23:30:52.638 INFO:teuthology.orchestra.run.smithi081.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T23:30:52.638 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:30:52.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:52.639 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-05-08T23:30:52.639 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-05-08T23:30:52.639 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T23:30:52.639 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T23:30:52.639 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T23:30:52.639 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-05-08T23:30:52.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:52.715 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-05-08T23:30:52.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:52.723 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list-id 2024-05-08T23:30:52.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:52.729 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-05-08T23:30:52.729 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-05-08T23:30:52.729 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:30:52.729 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T23:30:52.729 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-05-08T23:30:52.803 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T23:30:52.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:52.804 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-05-08T23:30:52.804 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-05-08T23:30:52.804 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T23:30:52.804 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T23:30:52.804 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T23:30:52.805 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T23:30:52.805 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T23:30:52.805 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-05-08T23:30:52.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:52.871 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-05-08T23:30:52.871 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-05-08T23:30:52.871 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2024-05-08T23:30:52.871 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T23:30:52.871 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-05-08T23:30:52.899 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T23:30:52.899 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:30:52.899 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:30:52.950 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:30:52.950 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:30:52.950 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:30:52.956 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:30:52.970 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:30:52.971 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:30:53.027 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:30:53.028 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:30:53.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:53.029 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:30:53.036 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T23:30:53.036 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T23:30:53.038 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:30:53.053 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:30:53.053 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:30:53.108 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:30:53.109 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:30:53.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:53.109 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:30:53.123 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2024-05-08T23:30:53.117+0000 7eff7e1dd640 -1 init, newargv = 0x55ef503939c0 newargc=15 2024-05-08T23:30:53.123 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[33197]: starting ceph client 2024-05-08T23:30:53.131 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[33197]: starting fuse 2024-05-08T23:30:54.217 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:30:54.235 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:30:54.236 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:30:54.295 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:30:54.296 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:30:54.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:54.296 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:30:54.303 INFO:teuthology.orchestra.run.smithi081.stdout:52 2024-05-08T23:30:54.304 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T23:30:54.304 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:30:54.304 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2024-05-08T23:30:54.304 DEBUG:teuthology.orchestra.run.smithi081:> import re 2024-05-08T23:30:54.304 DEBUG:teuthology.orchestra.run.smithi081:> import os 2024-05-08T23:30:54.304 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2024-05-08T23:30:54.304 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:30:54.304 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2024-05-08T23:30:54.304 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T23:30:54.305 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2024-05-08T23:30:54.305 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T23:30:54.305 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:30:54.305 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2024-05-08T23:30:54.305 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2024-05-08T23:30:54.305 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2024-05-08T23:30:54.305 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2024-05-08T23:30:54.305 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:30:54.305 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2024-05-08T23:30:54.305 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T23:30:54.305 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T23:30:54.305 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T23:30:54.305 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2024-05-08T23:30:54.305 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2024-05-08T23:30:54.306 DEBUG:teuthology.orchestra.run.smithi081:> return f 2024-05-08T23:30:54.306 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T23:30:54.306 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:30:54.306 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2024-05-08T23:30:54.306 DEBUG:teuthology.orchestra.run.smithi081:> ' 2024-05-08T23:30:54.385 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.33197.asok 2024-05-08T23:30:54.568 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33197.asok 2024-05-08T23:30:54.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:54.569 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33197.asok status 2024-05-08T23:30:54.720 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:30:54.720 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2024-05-08T23:30:54.720 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T23:30:54.720 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:30:54.720 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2024-05-08T23:30:54.720 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2024-05-08T23:30:54.721 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T23:30:54.721 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "33197", 2024-05-08T23:30:54.721 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2024-05-08T23:30:54.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:30:54.721 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2024-05-08T23:30:54.721 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2024-05-08T23:30:54.721 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 4656, 2024-05-08T23:30:54.721 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2024-05-08T23:30:54.721 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2024-05-08T23:30:54.721 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2024-05-08T23:30:54.721 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 4656 2024-05-08T23:30:54.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:30:54.721 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:30:54.721 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:30:54.722 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:30:54.722 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3352204415 2024-05-08T23:30:54.722 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:30:54.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:30:54.722 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:30:54.722 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:30:54.722 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:30:54.722 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3352204415 2024-05-08T23:30:54.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:30:54.722 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.4656 192.168.0.1:0/3352204415", 2024-05-08T23:30:54.722 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/3352204415", 2024-05-08T23:30:54.722 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2024-05-08T23:30:54.722 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 7, 2024-05-08T23:30:54.722 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 22, 2024-05-08T23:30:54.723 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2024-05-08T23:30:54.723 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2024-05-08T23:30:54.723 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2024-05-08T23:30:54.723 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:30:54.730 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:30:54.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.833+0000 7f80c602d700 1 Processor -- start 2024-05-08T23:30:54.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.833+0000 7f80c602d700 1 -- start start 2024-05-08T23:30:54.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.833+0000 7f80c602d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f80c0159a80 0x7f80c0159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:54.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.833+0000 7f80c602d700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c00b3590 con 0x7f80c01596a0 2024-05-08T23:30:54.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.833+0000 7f80c602d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c00b5020 con 0x7f80c00709a0 2024-05-08T23:30:54.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.833+0000 7f80c602d700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c00b24f0 con 0x7f80c0159a80 2024-05-08T23:30:54.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.833+0000 7f80c502b700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f80c01596a0 0x7f80c00b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38798/0 (socket says 172.21.15.81:38798) 2024-05-08T23:30:54.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.833+0000 7f80c502b700 1 -- 172.21.15.81:0/2828777093 learned_addr learned my addr 172.21.15.81:0/2828777093 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:54.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.833+0000 7f80c482a700 1 --2- 172.21.15.81:0/2828777093 >> v2:172.21.15.117:3300/0 conn(0x7f80c0159a80 0x7f80c0159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:54.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80b7fff700 1 -- 172.21.15.81:0/2828777093 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876818666 0 0) 0x7f80c00b3590 con 0x7f80c01596a0 2024-05-08T23:30:54.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80b7fff700 1 -- 172.21.15.81:0/2828777093 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80a80035d0 con 0x7f80c01596a0 2024-05-08T23:30:54.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80c482a700 1 -- 172.21.15.81:0/2828777093 >> v1:172.21.15.117:6789/0 conn(0x7f80c00709a0 legacy=0x7f80c0070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:54.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80c482a700 1 -- 172.21.15.81:0/2828777093 >> v1:172.21.15.81:6789/0 conn(0x7f80c01596a0 legacy=0x7f80c00b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:54.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80c482a700 1 -- 172.21.15.81:0/2828777093 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c00b27d0 con 0x7f80c0159a80 2024-05-08T23:30:54.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80c482a700 1 --2- 172.21.15.81:0/2828777093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80c0159a80 0x7f80c0159ef0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00aea0 tx=0x7f80bc00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dc9e766cdc7ef2a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:54.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80b7fff700 1 -- 172.21.15.81:0/2828777093 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00f070 con 0x7f80c0159a80 2024-05-08T23:30:54.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80b7fff700 1 -- 172.21.15.81:0/2828777093 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80bc00c910 con 0x7f80c0159a80 2024-05-08T23:30:54.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80b7fff700 1 -- 172.21.15.81:0/2828777093 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00cc80 con 0x7f80c0159a80 2024-05-08T23:30:54.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80c602d700 1 -- 172.21.15.81:0/2828777093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80c0159a80 msgr2=0x7f80c0159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:54.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80c602d700 1 --2- 172.21.15.81:0/2828777093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80c0159a80 0x7f80c0159ef0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00aea0 tx=0x7f80bc00c330 comp rx=0 tx=0).stop 2024-05-08T23:30:54.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80c602d700 1 -- 172.21.15.81:0/2828777093 shutdown_connections 2024-05-08T23:30:54.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80c602d700 1 --2- 172.21.15.81:0/2828777093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80c0159a80 0x7f80c0159ef0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:54.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80c602d700 1 -- 172.21.15.81:0/2828777093 >> 172.21.15.81:0/2828777093 conn(0x7f80c006c330 msgr2=0x7f80c006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:54.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80c602d700 1 -- 172.21.15.81:0/2828777093 shutdown_connections 2024-05-08T23:30:54.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80c602d700 1 -- 172.21.15.81:0/2828777093 wait complete. 2024-05-08T23:30:54.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80c602d700 1 Processor -- start 2024-05-08T23:30:54.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.837+0000 7f80c602d700 1 -- start start 2024-05-08T23:30:54.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c602d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80c00709a0 0x7f80c014f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c602d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80c01596a0 0x7f80c00add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c602d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80c0159a80 0x7f80c0156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c602d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c0155a20 con 0x7f80c0159a80 2024-05-08T23:30:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c602d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c014fd50 con 0x7f80c00709a0 2024-05-08T23:30:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c602d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c0155cc0 con 0x7f80c01596a0 2024-05-08T23:30:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c582c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80c0159a80 0x7f80c0156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c582c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80c0159a80 0x7f80c0156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51412/0 (socket says 172.21.15.81:51412) 2024-05-08T23:30:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c582c700 1 -- 172.21.15.81:0/1469972564 learned_addr learned my addr 172.21.15.81:0/1469972564 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c482a700 1 --2- 172.21.15.81:0/1469972564 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80c01596a0 0x7f80c00add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c502b700 1 --2- 172.21.15.81:0/1469972564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80c00709a0 0x7f80c014f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c582c700 1 -- 172.21.15.81:0/1469972564 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80c01596a0 msgr2=0x7f80c00add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c582c700 1 --2- 172.21.15.81:0/1469972564 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80c01596a0 0x7f80c00add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c582c700 1 -- 172.21.15.81:0/1469972564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80c00709a0 msgr2=0x7f80c014f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c582c700 1 --2- 172.21.15.81:0/1469972564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80c00709a0 0x7f80c014f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c582c700 1 -- 172.21.15.81:0/1469972564 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc00ab80 con 0x7f80c0159a80 2024-05-08T23:30:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c582c700 1 --2- 172.21.15.81:0/1469972564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80c0159a80 0x7f80c0156d60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f80b800c430 tx=0x7f80b800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80b5ffb700 1 -- 172.21.15.81:0/1469972564 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b80052d0 con 0x7f80c0159a80 2024-05-08T23:30:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80b5ffb700 1 -- 172.21.15.81:0/1469972564 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b8005430 con 0x7f80c0159a80 2024-05-08T23:30:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c602d700 1 -- 172.21.15.81:0/1469972564 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c0155fa0 con 0x7f80c0159a80 2024-05-08T23:30:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80b5ffb700 1 -- 172.21.15.81:0/1469972564 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b80055a0 con 0x7f80c0159a80 2024-05-08T23:30:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c602d700 1 -- 172.21.15.81:0/1469972564 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c0156560 con 0x7f80c0159a80 2024-05-08T23:30:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.841+0000 7f80c602d700 1 -- 172.21.15.81:0/1469972564 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c00ce120 con 0x7f80c0159a80 2024-05-08T23:30:54.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.845+0000 7f80b5ffb700 1 -- 172.21.15.81:0/1469972564 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f80b80038e0 con 0x7f80c0159a80 2024-05-08T23:30:54.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.845+0000 7f80b5ffb700 1 --2- 172.21.15.81:0/1469972564 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80ac066730 0x7f80ac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:54.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.845+0000 7f80b5ffb700 1 -- 172.21.15.81:0/1469972564 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80b8083490 con 0x7f80c0159a80 2024-05-08T23:30:54.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.845+0000 7f80c502b700 1 --2- 172.21.15.81:0/1469972564 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80ac066730 0x7f80ac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:54.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.845+0000 7f80c502b700 1 --2- 172.21.15.81:0/1469972564 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80ac066730 0x7f80ac068bf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f80b00011e0 tx=0x7f80b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:54.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:54.849+0000 7f80b5ffb700 1 -- 172.21.15.81:0/1469972564 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f80b8050800 con 0x7f80c0159a80 2024-05-08T23:30:55.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:55.057+0000 7f80c602d700 1 -- 172.21.15.81:0/1469972564 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f80c00684b0 con 0x7f80c0159a80 2024-05-08T23:30:55.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:55.057+0000 7f80b5ffb700 1 -- 172.21.15.81:0/1469972564 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f80b8053e20 con 0x7f80c0159a80 2024-05-08T23:30:55.062 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:30:55.064 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:55.061+0000 7f80c602d700 1 -- 172.21.15.81:0/1469972564 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80ac066730 msgr2=0x7f80ac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:55.064 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:55.061+0000 7f80c602d700 1 --2- 172.21.15.81:0/1469972564 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80ac066730 0x7f80ac068bf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f80b00011e0 tx=0x7f80b0009040 comp rx=0 tx=0).stop 2024-05-08T23:30:55.064 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:55.061+0000 7f80c602d700 1 -- 172.21.15.81:0/1469972564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80c0159a80 msgr2=0x7f80c0156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:55.064 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:55.061+0000 7f80c602d700 1 --2- 172.21.15.81:0/1469972564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80c0159a80 0x7f80c0156d60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f80b800c430 tx=0x7f80b800c740 comp rx=0 tx=0).stop 2024-05-08T23:30:55.065 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:55.061+0000 7f80c602d700 1 -- 172.21.15.81:0/1469972564 shutdown_connections 2024-05-08T23:30:55.065 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:55.061+0000 7f80c602d700 1 --2- 172.21.15.81:0/1469972564 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80ac066730 0x7f80ac068bf0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:55.065 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:55.061+0000 7f80c602d700 1 --2- 172.21.15.81:0/1469972564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80c0159a80 0x7f80c0156d60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:55.065 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:55.061+0000 7f80c602d700 1 --2- 172.21.15.81:0/1469972564 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80c01596a0 0x7f80c00add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:55.065 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:55.061+0000 7f80c602d700 1 --2- 172.21.15.81:0/1469972564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80c00709a0 0x7f80c014f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:55.065 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:55.061+0000 7f80c602d700 1 -- 172.21.15.81:0/1469972564 >> 172.21.15.81:0/1469972564 conn(0x7f80c006c330 msgr2=0x7f80c0071e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:55.065 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:55.061+0000 7f80c602d700 1 -- 172.21.15.81:0/1469972564 shutdown_connections 2024-05-08T23:30:55.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:55.061+0000 7f80c602d700 1 -- 172.21.15.81:0/1469972564 wait complete. 2024-05-08T23:30:55.077 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:30:55.077 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:30:55.077 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2024-05-08T23:30:55.078 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-08T23:30:55.078 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-08T23:30:55.078 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:30:55.078 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:30:55.078 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:30:55.078 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-08T23:30:55.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:55.078 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2024-05-08T23:30:55.083 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:30:55.084 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:30:55.084 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:30:55.084 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:55.084 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-05-08T23:30:55.084 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:55.084 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:30:55.084 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:55.084 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:30:55.084 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:55.084 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:30:55.084 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:55.084 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:30:55.084 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:55.085 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T23:30:55.085 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42560sec preferred_lft 42560sec 2024-05-08T23:30:55.085 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2024-05-08T23:30:55.085 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:55.085 INFO:teuthology.orchestra.run.smithi081.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T23:30:55.085 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 02:42:22:62:7a:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:55.085 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T23:30:55.085 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:55.085 INFO:teuthology.orchestra.run.smithi081.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:30:55.085 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:2b:10:6b:08:e3 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:55.085 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:30:55.085 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:55.085 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::48fc:c7ff:fe41:6d94/64 scope link 2024-05-08T23:30:55.086 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:55.086 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:30:55.086 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:2b:10:6b:08:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:30:55.086 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::702b:10ff:fe6b:8e3/64 scope link 2024-05-08T23:30:55.086 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:55.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:55.086 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:30:55.100 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-05-08T23:30:55.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:55.103 DEBUG:teuthology.orchestra.run.smithi081:> route 2024-05-08T23:30:55.152 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2024-05-08T23:30:55.152 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:30:55.152 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T23:30:55.152 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T23:30:55.152 INFO:teuthology.orchestra.run.smithi081.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T23:30:55.152 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T23:30:55.152 INFO:teuthology.orchestra.run.smithi081.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T23:30:55.152 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:30:55.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:55.153 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-05-08T23:30:55.153 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-05-08T23:30:55.153 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T23:30:55.153 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T23:30:55.153 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T23:30:55.153 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-05-08T23:30:55.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:55.193 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-05-08T23:30:55.199 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:30:55.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:55.200 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list-id 2024-05-08T23:30:55.205 INFO:teuthology.orchestra.run.smithi081.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-08T23:30:55.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:55.206 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-05-08T23:30:55.206 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-05-08T23:30:55.206 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:30:55.206 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-08T23:30:55.206 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-05-08T23:30:55.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:55.279 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-08T23:30:55.291 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:30:55.291 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:30:55.291 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:30:55.291 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:55.292 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-05-08T23:30:55.292 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:55.292 INFO:teuthology.orchestra.run.smithi081.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:30:55.292 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether d2:50:24:7f:23:29 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T23:30:55.292 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T23:30:55.292 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:55.292 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::d050:24ff:fe7f:2329/64 scope link 2024-05-08T23:30:55.292 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:55.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:55.293 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-08T23:30:55.306 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:30:55.307 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:30:55.307 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:30:55.307 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:55.307 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-05-08T23:30:55.307 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:55.307 INFO:teuthology.orchestra.run.smithi081.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:30:55.307 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether d2:50:24:7f:23:29 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T23:30:55.307 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T23:30:55.307 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:55.307 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::d050:24ff:fe7f:2329/64 scope link 2024-05-08T23:30:55.307 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:55.308 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-08T23:30:55.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:55.309 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-05-08T23:30:55.309 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-05-08T23:30:55.309 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-08T23:30:55.309 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-08T23:30:55.309 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-08T23:30:55.309 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-08T23:30:55.309 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-08T23:30:55.309 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-05-08T23:30:55.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:55.365 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-05-08T23:30:55.365 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-05-08T23:30:55.365 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.1 up 2024-05-08T23:30:55.365 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set dev brx.1 master ceph-brx 2024-05-08T23:30:55.365 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-05-08T23:30:55.392 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-05-08T23:30:55.393 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:30:55.393 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-08T23:30:55.442 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-08T23:30:55.443 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:30:55.443 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-08T23:30:55.449 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:30:55.466 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:30:55.466 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:30:55.480 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:30:55.480 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:30:55.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:55.481 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:30:55.530 INFO:teuthology.orchestra.run.smithi081.stdout:52 2024-05-08T23:30:55.530 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2024-05-08T23:30:55.530 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-05-08T23:30:55.533 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:30:55.551 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:30:55.551 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:30:55.560 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:30:55.561 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:30:55.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:55.562 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:30:55.615 INFO:teuthology.orchestra.run.smithi081.stdout:52 2024-05-08T23:30:55.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi081.stderr:2024-05-08T23:30:55.625+0000 7f823830e640 -1 init, newargv = 0x56323e46abe0 newargc=15 2024-05-08T23:30:55.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi081.stderr:ceph-fuse[33340]: starting ceph client 2024-05-08T23:30:55.640 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi081.stderr:ceph-fuse[33340]: starting fuse 2024-05-08T23:30:56.716 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:30:56.735 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:30:56.735 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:30:56.795 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:30:56.796 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:30:56.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:56.797 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:30:56.804 INFO:teuthology.orchestra.run.smithi081.stdout:52 2024-05-08T23:30:56.804 INFO:teuthology.orchestra.run.smithi081.stdout:53 2024-05-08T23:30:56.804 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2024-05-08T23:30:56.804 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:30:56.804 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2024-05-08T23:30:56.804 DEBUG:teuthology.orchestra.run.smithi081:> import re 2024-05-08T23:30:56.804 DEBUG:teuthology.orchestra.run.smithi081:> import os 2024-05-08T23:30:56.804 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2024-05-08T23:30:56.804 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:30:56.804 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2024-05-08T23:30:56.805 DEBUG:teuthology.orchestra.run.smithi081:> return f 2024-05-08T23:30:56.806 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T23:30:56.806 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:30:56.806 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.2")) 2024-05-08T23:30:56.806 DEBUG:teuthology.orchestra.run.smithi081:> ' 2024-05-08T23:30:56.887 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.2.33340.asok 2024-05-08T23:30:57.070 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.33340.asok 2024-05-08T23:30:57.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:57.071 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.33340.asok status 2024-05-08T23:30:57.176 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:30:57.177 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2024-05-08T23:30:57.177 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T23:30:57.177 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:30:57.177 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "2", 2024-05-08T23:30:57.177 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2024-05-08T23:30:57.177 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-05-08T23:30:57.177 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "33340", 2024-05-08T23:30:57.177 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2024-05-08T23:30:57.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:30:57.177 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2024-05-08T23:30:57.177 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2024-05-08T23:30:57.177 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 4668, 2024-05-08T23:30:57.178 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2024-05-08T23:30:57.178 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2024-05-08T23:30:57.178 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2024-05-08T23:30:57.178 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 4668 2024-05-08T23:30:57.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:30:57.178 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:30:57.178 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:30:57.178 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.2:0", 2024-05-08T23:30:57.178 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3842498988 2024-05-08T23:30:57.178 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:30:57.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:30:57.178 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:30:57.178 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:30:57.178 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.2:0", 2024-05-08T23:30:57.178 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3842498988 2024-05-08T23:30:57.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:30:57.179 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.4668 192.168.0.2:0/3842498988", 2024-05-08T23:30:57.179 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.2:0/3842498988", 2024-05-08T23:30:57.179 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2024-05-08T23:30:57.179 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 7, 2024-05-08T23:30:57.179 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 22, 2024-05-08T23:30:57.179 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2024-05-08T23:30:57.179 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2024-05-08T23:30:57.179 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2024-05-08T23:30:57.179 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:30:57.186 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:30:57.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.297+0000 7f65094ba700 1 Processor -- start 2024-05-08T23:30:57.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.297+0000 7f65094ba700 1 -- start start 2024-05-08T23:30:57.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f65094ba700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f65040b0540 0x7f650406b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:57.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f65094ba700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65040cc240 con 0x7f65040cfd30 2024-05-08T23:30:57.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f65094ba700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65040cb630 con 0x7f650409a180 2024-05-08T23:30:57.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f65094ba700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65040cb910 con 0x7f65040b0540 2024-05-08T23:30:57.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f6502ffd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f65040b0540 0x7f650406b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:57.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f65027fc700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f65040cfd30 0x7f6504099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44554/0 (socket says 172.21.15.81:44554) 2024-05-08T23:30:57.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f65027fc700 1 -- 172.21.15.81:0/4237917845 learned_addr learned my addr 172.21.15.81:0/4237917845 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:57.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f6502ffd700 1 -- 172.21.15.81:0/4237917845 >> v1:172.21.15.117:6789/0 conn(0x7f650409a180 legacy=0x7f650415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:57.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f6502ffd700 1 -- 172.21.15.81:0/4237917845 >> v1:172.21.15.81:6789/0 conn(0x7f65040cfd30 legacy=0x7f6504099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:57.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f6501ffb700 1 -- 172.21.15.81:0/4237917845 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806692791 0 0) 0x7f65040cc240 con 0x7f65040cfd30 2024-05-08T23:30:57.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f6502ffd700 1 -- 172.21.15.81:0/4237917845 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65040cbbf0 con 0x7f65040b0540 2024-05-08T23:30:57.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f6502ffd700 1 --2- 172.21.15.81:0/4237917845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65040b0540 0x7f650406b470 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009850 tx=0x7f64f8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fc8bf0609de4d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:57.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f6501ffb700 1 -- 172.21.15.81:0/4237917845 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8011830 con 0x7f65040b0540 2024-05-08T23:30:57.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f6501ffb700 1 -- 172.21.15.81:0/4237917845 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f8011990 con 0x7f65040b0540 2024-05-08T23:30:57.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f6501ffb700 1 -- 172.21.15.81:0/4237917845 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8011d20 con 0x7f65040b0540 2024-05-08T23:30:57.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f65094ba700 1 -- 172.21.15.81:0/4237917845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65040b0540 msgr2=0x7f650406b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:57.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f65094ba700 1 --2- 172.21.15.81:0/4237917845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65040b0540 0x7f650406b470 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009850 tx=0x7f64f8009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:57.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f65094ba700 1 -- 172.21.15.81:0/4237917845 shutdown_connections 2024-05-08T23:30:57.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f65094ba700 1 --2- 172.21.15.81:0/4237917845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65040b0540 0x7f650406b470 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:57.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f65094ba700 1 -- 172.21.15.81:0/4237917845 >> 172.21.15.81:0/4237917845 conn(0x7f65040b2ad0 msgr2=0x7f65040b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:57.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f65094ba700 1 -- 172.21.15.81:0/4237917845 shutdown_connections 2024-05-08T23:30:57.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.301+0000 7f65094ba700 1 -- 172.21.15.81:0/4237917845 wait complete. 2024-05-08T23:30:57.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f65094ba700 1 Processor -- start 2024-05-08T23:30:57.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f65094ba700 1 -- start start 2024-05-08T23:30:57.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f65094ba700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f650409a180 0x7f65040c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:57.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f65094ba700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65040b0540 0x7f65040cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:57.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f65094ba700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65040cfd30 0x7f65040c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:57.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f65094ba700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65040c9aa0 con 0x7f650409a180 2024-05-08T23:30:57.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f65094ba700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65040c6200 con 0x7f65040b0540 2024-05-08T23:30:57.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f65094ba700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65040c6450 con 0x7f65040cfd30 2024-05-08T23:30:57.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f6502ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f650409a180 0x7f65040c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:57.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f6502ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f650409a180 0x7f65040c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38880/0 (socket says 172.21.15.81:38880) 2024-05-08T23:30:57.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f6502ffd700 1 -- 172.21.15.81:0/897795554 learned_addr learned my addr 172.21.15.81:0/897795554 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:57.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f65027fc700 1 --2- 172.21.15.81:0/897795554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65040b0540 0x7f65040cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:57.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f65037fe700 1 --2- 172.21.15.81:0/897795554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65040cfd30 0x7f65040c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:57.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f6502ffd700 1 -- 172.21.15.81:0/897795554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65040cfd30 msgr2=0x7f65040c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:57.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f6502ffd700 1 --2- 172.21.15.81:0/897795554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65040cfd30 0x7f65040c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:57.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f6502ffd700 1 -- 172.21.15.81:0/897795554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65040b0540 msgr2=0x7f65040cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:57.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f6502ffd700 1 --2- 172.21.15.81:0/897795554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65040b0540 0x7f65040cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:57.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f6502ffd700 1 -- 172.21.15.81:0/897795554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f80094b0 con 0x7f650409a180 2024-05-08T23:30:57.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f65037fe700 1 --2- 172.21.15.81:0/897795554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65040cfd30 0x7f65040c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:57.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f6502ffd700 1 --2- 172.21.15.81:0/897795554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f650409a180 0x7f65040c74f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f64f8000c00 tx=0x7f64f8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:57.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.305+0000 7f64eb7fe700 1 -- 172.21.15.81:0/897795554 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f80045b0 con 0x7f650409a180 2024-05-08T23:30:57.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.309+0000 7f64eb7fe700 1 -- 172.21.15.81:0/897795554 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f801f070 con 0x7f650409a180 2024-05-08T23:30:57.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.309+0000 7f65094ba700 1 -- 172.21.15.81:0/897795554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65040c6730 con 0x7f650409a180 2024-05-08T23:30:57.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.309+0000 7f64eb7fe700 1 -- 172.21.15.81:0/897795554 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f80038e0 con 0x7f650409a180 2024-05-08T23:30:57.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.309+0000 7f65094ba700 1 -- 172.21.15.81:0/897795554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65040c6cf0 con 0x7f650409a180 2024-05-08T23:30:57.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.309+0000 7f65094ba700 1 -- 172.21.15.81:0/897795554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65040672a0 con 0x7f650409a180 2024-05-08T23:30:57.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.309+0000 7f64eb7fe700 1 -- 172.21.15.81:0/897795554 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f64f8021030 con 0x7f650409a180 2024-05-08T23:30:57.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.309+0000 7f64eb7fe700 1 --2- 172.21.15.81:0/897795554 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f64f0066730 0x7f64f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:57.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.309+0000 7f65027fc700 1 --2- 172.21.15.81:0/897795554 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f64f0066730 0x7f64f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:57.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.309+0000 7f64eb7fe700 1 -- 172.21.15.81:0/897795554 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64f8084910 con 0x7f650409a180 2024-05-08T23:30:57.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.309+0000 7f65027fc700 1 --2- 172.21.15.81:0/897795554 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f64f0066730 0x7f64f0068bf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f65040ca270 tx=0x7f64f4007bb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:57.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.317+0000 7f64eb7fe700 1 -- 172.21.15.81:0/897795554 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f64f8051c80 con 0x7f650409a180 2024-05-08T23:30:57.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.529+0000 7f65094ba700 1 -- 172.21.15.81:0/897795554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f65040cb3a0 con 0x7f650409a180 2024-05-08T23:30:57.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.529+0000 7f64eb7fe700 1 -- 172.21.15.81:0/897795554 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f64f8015060 con 0x7f650409a180 2024-05-08T23:30:57.533 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:30:57.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.533+0000 7f65094ba700 1 -- 172.21.15.81:0/897795554 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f64f0066730 msgr2=0x7f64f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:57.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.533+0000 7f65094ba700 1 --2- 172.21.15.81:0/897795554 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f64f0066730 0x7f64f0068bf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f65040ca270 tx=0x7f64f4007bb0 comp rx=0 tx=0).stop 2024-05-08T23:30:57.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.533+0000 7f65094ba700 1 -- 172.21.15.81:0/897795554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f650409a180 msgr2=0x7f65040c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:57.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.533+0000 7f65094ba700 1 --2- 172.21.15.81:0/897795554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f650409a180 0x7f65040c74f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f64f8000c00 tx=0x7f64f8004770 comp rx=0 tx=0).stop 2024-05-08T23:30:57.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.533+0000 7f65094ba700 1 -- 172.21.15.81:0/897795554 shutdown_connections 2024-05-08T23:30:57.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.533+0000 7f65094ba700 1 --2- 172.21.15.81:0/897795554 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f64f0066730 0x7f64f0068bf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:57.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.533+0000 7f65094ba700 1 --2- 172.21.15.81:0/897795554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65040cfd30 0x7f65040c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:57.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.533+0000 7f65094ba700 1 --2- 172.21.15.81:0/897795554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65040b0540 0x7f65040cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:57.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.533+0000 7f65094ba700 1 --2- 172.21.15.81:0/897795554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f650409a180 0x7f65040c74f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:57.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.533+0000 7f65094ba700 1 -- 172.21.15.81:0/897795554 >> 172.21.15.81:0/897795554 conn(0x7f65040b2ad0 msgr2=0x7f650406b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:57.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.533+0000 7f65094ba700 1 -- 172.21.15.81:0/897795554 shutdown_connections 2024-05-08T23:30:57.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:57.533+0000 7f65094ba700 1 -- 172.21.15.81:0/897795554 wait complete. 2024-05-08T23:30:57.548 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:30:57.548 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:30:57.548 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi117.front.sepia.ceph.com 2024-05-08T23:30:57.548 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-08T23:30:57.548 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T23:30:57.548 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:30:57.548 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:30:57.548 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:30:57.549 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T23:30:57.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:57.549 DEBUG:teuthology.orchestra.run.smithi117:> ip addr 2024-05-08T23:30:57.557 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:30:57.557 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:30:57.557 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:30:57.557 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:57.557 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-05-08T23:30:57.558 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:57.558 INFO:teuthology.orchestra.run.smithi117.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:30:57.558 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8a brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:57.558 INFO:teuthology.orchestra.run.smithi117.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:30:57.558 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8b brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:57.558 INFO:teuthology.orchestra.run.smithi117.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:30:57.558 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:62 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:57.558 INFO:teuthology.orchestra.run.smithi117.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:30:57.558 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:63 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:57.558 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.21.15.117/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T23:30:57.558 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft 42551sec preferred_lft 42551sec 2024-05-08T23:30:57.558 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-05-08T23:30:57.558 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:57.558 INFO:teuthology.orchestra.run.smithi117.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T23:30:57.559 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 02:42:94:b7:2a:7e brd ff:ff:ff:ff:ff:ff 2024-05-08T23:30:57.559 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T23:30:57.559 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:30:57.559 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T23:30:57.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:57.559 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-05-08T23:30:57.559 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-05-08T23:30:57.559 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add name ceph-brx type bridge 2024-05-08T23:30:57.559 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr flush dev ceph-brx 2024-05-08T23:30:57.559 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set ceph-brx up 2024-05-08T23:30:57.559 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T23:30:57.560 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-05-08T23:30:57.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:57.603 DEBUG:teuthology.orchestra.run.smithi117:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:30:57.660 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-05-08T23:30:57.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:57.663 DEBUG:teuthology.orchestra.run.smithi117:> route 2024-05-08T23:30:57.699 INFO:teuthology.orchestra.run.smithi117.stdout:Kernel IP routing table 2024-05-08T23:30:57.699 INFO:teuthology.orchestra.run.smithi117.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:30:57.699 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T23:30:57.699 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T23:30:57.699 INFO:teuthology.orchestra.run.smithi117.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T23:30:57.699 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T23:30:57.699 INFO:teuthology.orchestra.run.smithi117.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T23:30:57.700 INFO:teuthology.orchestra.run.smithi117.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:30:57.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:57.700 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-05-08T23:30:57.700 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-05-08T23:30:57.700 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T23:30:57.700 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T23:30:57.700 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T23:30:57.700 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-05-08T23:30:57.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:57.741 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-05-08T23:30:57.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:57.747 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list-id 2024-05-08T23:30:57.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:57.797 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-05-08T23:30:57.797 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-05-08T23:30:57.797 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T23:30:57.797 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-08T23:30:57.797 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-05-08T23:30:57.826 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-08T23:30:57.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:57.827 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-05-08T23:30:57.827 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-05-08T23:30:57.827 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-08T23:30:57.827 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T23:30:57.827 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-08T23:30:57.827 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-08T23:30:57.827 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-08T23:30:57.827 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-05-08T23:30:57.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:57.908 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-05-08T23:30:57.908 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-05-08T23:30:57.908 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set brx.0 up 2024-05-08T23:30:57.908 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T23:30:57.908 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-05-08T23:30:57.932 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-08T23:30:57.932 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:30:57.932 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-08T23:30:57.982 INFO:teuthology.orchestra.run.smithi117.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-08T23:30:57.982 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:30:57.982 DEBUG:teuthology.orchestra.run.smithi117:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-08T23:30:57.989 DEBUG:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2024-05-08T23:30:58.007 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:30:58.007 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:30:58.065 INFO:teuthology.orchestra.run.smithi117.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:30:58.066 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:30:58.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:58.067 DEBUG:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2024-05-08T23:30:58.074 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T23:30:58.074 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-08T23:30:58.076 DEBUG:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2024-05-08T23:30:58.092 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:30:58.092 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:30:58.148 INFO:teuthology.orchestra.run.smithi117.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:30:58.149 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:30:58.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:58.150 DEBUG:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2024-05-08T23:30:58.167 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi117.stderr:2024-05-08T23:30:58.166+0000 7f7bc9517640 -1 init, newargv = 0x55eb14d5fa80 newargc=15 2024-05-08T23:30:58.168 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi117.stderr:ceph-fuse[31546]: starting ceph client 2024-05-08T23:30:58.176 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi117.stderr:ceph-fuse[31546]: starting fuse 2024-05-08T23:30:59.257 DEBUG:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2024-05-08T23:30:59.274 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:30:59.274 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:30:59.290 INFO:teuthology.orchestra.run.smithi117.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:30:59.290 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:30:59.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:59.291 DEBUG:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2024-05-08T23:30:59.298 INFO:teuthology.orchestra.run.smithi117.stdout:52 2024-05-08T23:30:59.299 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T23:30:59.299 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:30:59.299 DEBUG:teuthology.orchestra.run.smithi117:> import glob 2024-05-08T23:30:59.299 DEBUG:teuthology.orchestra.run.smithi117:> import re 2024-05-08T23:30:59.299 DEBUG:teuthology.orchestra.run.smithi117:> import os 2024-05-08T23:30:59.299 DEBUG:teuthology.orchestra.run.smithi117:> import subprocess 2024-05-08T23:30:59.299 DEBUG:teuthology.orchestra.run.smithi117:> 2024-05-08T23:30:59.299 DEBUG:teuthology.orchestra.run.smithi117:> def _find_admin_socket(client_name): 2024-05-08T23:30:59.299 DEBUG:teuthology.orchestra.run.smithi117:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-08T23:30:59.299 DEBUG:teuthology.orchestra.run.smithi117:> files = glob.glob(asok_path) 2024-05-08T23:30:59.299 DEBUG:teuthology.orchestra.run.smithi117:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-08T23:30:59.299 DEBUG:teuthology.orchestra.run.smithi117:> 2024-05-08T23:30:59.299 DEBUG:teuthology.orchestra.run.smithi117:> # Given a non-glob path, it better be there 2024-05-08T23:30:59.300 DEBUG:teuthology.orchestra.run.smithi117:> if "*" not in asok_path: 2024-05-08T23:30:59.300 DEBUG:teuthology.orchestra.run.smithi117:> assert(len(files) == 1) 2024-05-08T23:30:59.300 DEBUG:teuthology.orchestra.run.smithi117:> return files[0] 2024-05-08T23:30:59.300 DEBUG:teuthology.orchestra.run.smithi117:> 2024-05-08T23:30:59.300 DEBUG:teuthology.orchestra.run.smithi117:> for f in files: 2024-05-08T23:30:59.300 DEBUG:teuthology.orchestra.run.smithi117:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T23:30:59.300 DEBUG:teuthology.orchestra.run.smithi117:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T23:30:59.300 DEBUG:teuthology.orchestra.run.smithi117:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T23:30:59.300 DEBUG:teuthology.orchestra.run.smithi117:> contents = proc_f.read() 2024-05-08T23:30:59.300 DEBUG:teuthology.orchestra.run.smithi117:> if mountpoint in contents: 2024-05-08T23:30:59.300 DEBUG:teuthology.orchestra.run.smithi117:> return f 2024-05-08T23:30:59.300 DEBUG:teuthology.orchestra.run.smithi117:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T23:30:59.300 DEBUG:teuthology.orchestra.run.smithi117:> 2024-05-08T23:30:59.300 DEBUG:teuthology.orchestra.run.smithi117:> print(_find_admin_socket("client.1")) 2024-05-08T23:30:59.300 DEBUG:teuthology.orchestra.run.smithi117:> ' 2024-05-08T23:30:59.383 INFO:teuthology.orchestra.run.smithi117.stdout:/var/run/ceph/ceph-client.1.31546.asok 2024-05-08T23:30:59.566 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.31546.asok 2024-05-08T23:30:59.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:30:59.567 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.31546.asok status 2024-05-08T23:30:59.672 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-05-08T23:30:59.672 INFO:teuthology.orchestra.run.smithi117.stdout: "metadata": { 2024-05-08T23:30:59.672 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T23:30:59.672 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:30:59.672 INFO:teuthology.orchestra.run.smithi117.stdout: "entity_id": "1", 2024-05-08T23:30:59.672 INFO:teuthology.orchestra.run.smithi117.stdout: "hostname": "smithi117", 2024-05-08T23:30:59.672 INFO:teuthology.orchestra.run.smithi117.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-08T23:30:59.672 INFO:teuthology.orchestra.run.smithi117.stdout: "pid": "31546", 2024-05-08T23:30:59.673 INFO:teuthology.orchestra.run.smithi117.stdout: "root": "/" 2024-05-08T23:30:59.673 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-05-08T23:30:59.673 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_count": 0, 2024-05-08T23:30:59.673 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_pinned_count": 0, 2024-05-08T23:30:59.673 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 14468, 2024-05-08T23:30:59.673 INFO:teuthology.orchestra.run.smithi117.stdout: "inst": { 2024-05-08T23:30:59.673 INFO:teuthology.orchestra.run.smithi117.stdout: "name": { 2024-05-08T23:30:59.673 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client", 2024-05-08T23:30:59.673 INFO:teuthology.orchestra.run.smithi117.stdout: "num": 14468 2024-05-08T23:30:59.673 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-05-08T23:30:59.673 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2024-05-08T23:30:59.673 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2024-05-08T23:30:59.673 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:30:59.673 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 1324154865 2024-05-08T23:30:59.673 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-05-08T23:30:59.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-05-08T23:30:59.674 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2024-05-08T23:30:59.674 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2024-05-08T23:30:59.674 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:30:59.674 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 1324154865 2024-05-08T23:30:59.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-05-08T23:30:59.674 INFO:teuthology.orchestra.run.smithi117.stdout: "inst_str": "client.14468 192.168.0.1:0/1324154865", 2024-05-08T23:30:59.674 INFO:teuthology.orchestra.run.smithi117.stdout: "addr_str": "192.168.0.1:0/1324154865", 2024-05-08T23:30:59.674 INFO:teuthology.orchestra.run.smithi117.stdout: "inode_count": 1, 2024-05-08T23:30:59.674 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_epoch": 7, 2024-05-08T23:30:59.674 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch": 22, 2024-05-08T23:30:59.674 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch_barrier": 0, 2024-05-08T23:30:59.674 INFO:teuthology.orchestra.run.smithi117.stdout: "blocklisted": false, 2024-05-08T23:30:59.674 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs" 2024-05-08T23:30:59.675 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-05-08T23:30:59.681 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:30:59.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0ff1441700 1 Processor -- start 2024-05-08T23:30:59.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0ff1441700 1 -- start start 2024-05-08T23:30:59.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0ff1441700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0fec1478c0 0x7f0fec165240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:59.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0ff1441700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fec144d40 0x7f0fec1451b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:59.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0ff1441700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fec147d20 con 0x7f0fec0cb3b0 2024-05-08T23:30:59.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0ff1441700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fec161c50 con 0x7f0fec1478c0 2024-05-08T23:30:59.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0ff1441700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec165780 con 0x7f0fec144d40 2024-05-08T23:30:59.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0feaffd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f0fec0cb3b0 0x7f0fec162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44562/0 (socket says 172.21.15.81:44562) 2024-05-08T23:30:59.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0fe3fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0fec1478c0 0x7f0fec165240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:59.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0feaffd700 1 -- 172.21.15.81:0/3981844447 learned_addr learned my addr 172.21.15.81:0/3981844447 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:30:59.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0feb7fe700 1 --2- 172.21.15.81:0/3981844447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fec144d40 0x7f0fec1451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:59.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0fe3fff700 1 -- 172.21.15.81:0/3981844447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fec144d40 msgr2=0x7f0fec1451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:59.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0fe3fff700 1 --2- 172.21.15.81:0/3981844447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fec144d40 0x7f0fec1451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:59.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0fe3fff700 1 -- 172.21.15.81:0/3981844447 >> v1:172.21.15.81:6789/0 conn(0x7f0fec0cb3b0 legacy=0x7f0fec162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:59.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0fe3fff700 1 -- 172.21.15.81:0/3981844447 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec165970 con 0x7f0fec1478c0 2024-05-08T23:30:59.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0fe3fff700 1 --2- 172.21.15.81:0/3981844447 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fec1478c0 0x7f0fec165240 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009850 tx=0x7f0fd4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7ec1338f82fa4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:59.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0fea7fc700 1 -- 172.21.15.81:0/3981844447 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4011830 con 0x7f0fec1478c0 2024-05-08T23:30:59.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0fea7fc700 1 -- 172.21.15.81:0/3981844447 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd4011990 con 0x7f0fec1478c0 2024-05-08T23:30:59.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0fea7fc700 1 -- 172.21.15.81:0/3981844447 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4011d20 con 0x7f0fec1478c0 2024-05-08T23:30:59.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0ff1441700 1 -- 172.21.15.81:0/3981844447 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fec1478c0 msgr2=0x7f0fec165240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:59.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0ff1441700 1 --2- 172.21.15.81:0/3981844447 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fec1478c0 0x7f0fec165240 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009850 tx=0x7f0fd4009b60 comp rx=0 tx=0).stop 2024-05-08T23:30:59.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0ff1441700 1 -- 172.21.15.81:0/3981844447 shutdown_connections 2024-05-08T23:30:59.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0ff1441700 1 --2- 172.21.15.81:0/3981844447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fec144d40 0x7f0fec1451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:59.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0ff1441700 1 --2- 172.21.15.81:0/3981844447 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fec1478c0 0x7f0fec165240 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:59.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0ff1441700 1 -- 172.21.15.81:0/3981844447 >> 172.21.15.81:0/3981844447 conn(0x7f0fec06a360 msgr2=0x7f0fec06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:59.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0ff1441700 1 -- 172.21.15.81:0/3981844447 shutdown_connections 2024-05-08T23:30:59.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.801+0000 7f0ff1441700 1 -- 172.21.15.81:0/3981844447 wait complete. 2024-05-08T23:30:59.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0ff1441700 1 Processor -- start 2024-05-08T23:30:59.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0ff1441700 1 -- start start 2024-05-08T23:30:59.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0ff1441700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fec0cb3b0 0x7f0fec133bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:59.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0ff1441700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fec144d40 0x7f0fec15d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:59.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0ff1441700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fec1478c0 0x7f0fec15d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:59.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0ff1441700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec14b200 con 0x7f0fec1478c0 2024-05-08T23:30:59.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0ff1441700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec134110 con 0x7f0fec144d40 2024-05-08T23:30:59.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0ff1441700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec148620 con 0x7f0fec0cb3b0 2024-05-08T23:30:59.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0feb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fec1478c0 0x7f0fec15d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:59.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0feb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fec1478c0 0x7f0fec15d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38896/0 (socket says 172.21.15.81:38896) 2024-05-08T23:30:59.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0feb7fe700 1 -- 172.21.15.81:0/3954952136 learned_addr learned my addr 172.21.15.81:0/3954952136 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:30:59.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0fe3fff700 1 --2- 172.21.15.81:0/3954952136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fec144d40 0x7f0fec15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:59.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0feaffd700 1 --2- 172.21.15.81:0/3954952136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fec0cb3b0 0x7f0fec133bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:59.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0feb7fe700 1 -- 172.21.15.81:0/3954952136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fec0cb3b0 msgr2=0x7f0fec133bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:59.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0feb7fe700 1 --2- 172.21.15.81:0/3954952136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fec0cb3b0 0x7f0fec133bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:59.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0feb7fe700 1 -- 172.21.15.81:0/3954952136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fec144d40 msgr2=0x7f0fec15d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:59.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0feb7fe700 1 --2- 172.21.15.81:0/3954952136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fec144d40 0x7f0fec15d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:59.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0feb7fe700 1 -- 172.21.15.81:0/3954952136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd40094b0 con 0x7f0fec1478c0 2024-05-08T23:30:59.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0feb7fe700 1 --2- 172.21.15.81:0/3954952136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fec1478c0 0x7f0fec15d8a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4007e40 tx=0x7f0fe400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:59.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0fe37fe700 1 -- 172.21.15.81:0/3954952136 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe400eea0 con 0x7f0fec1478c0 2024-05-08T23:30:59.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0fe37fe700 1 -- 172.21.15.81:0/3954952136 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe4014420 con 0x7f0fec1478c0 2024-05-08T23:30:59.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0fe37fe700 1 -- 172.21.15.81:0/3954952136 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40147b0 con 0x7f0fec1478c0 2024-05-08T23:30:59.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0ff1441700 1 -- 172.21.15.81:0/3954952136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fec14af20 con 0x7f0fec1478c0 2024-05-08T23:30:59.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.805+0000 7f0ff1441700 1 -- 172.21.15.81:0/3954952136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fec148340 con 0x7f0fec1478c0 2024-05-08T23:30:59.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.809+0000 7f0ff1441700 1 -- 172.21.15.81:0/3954952136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fec0684b0 con 0x7f0fec1478c0 2024-05-08T23:30:59.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.809+0000 7f0fe37fe700 1 -- 172.21.15.81:0/3954952136 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0fe4014910 con 0x7f0fec1478c0 2024-05-08T23:30:59.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.809+0000 7f0fe37fe700 1 --2- 172.21.15.81:0/3954952136 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0fcc066760 0x7f0fcc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:59.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.809+0000 7f0fe37fe700 1 -- 172.21.15.81:0/3954952136 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fe40849f0 con 0x7f0fec1478c0 2024-05-08T23:30:59.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.809+0000 7f0feaffd700 1 --2- 172.21.15.81:0/3954952136 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0fcc066760 0x7f0fcc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:59.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.809+0000 7f0feaffd700 1 --2- 172.21.15.81:0/3954952136 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0fcc066760 0x7f0fcc068c20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc005930 tx=0x7f0fdc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:59.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:30:59.817+0000 7f0fe37fe700 1 -- 172.21.15.81:0/3954952136 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0fe4051cd0 con 0x7f0fec1478c0 2024-05-08T23:31:00.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:00.025+0000 7f0ff1441700 1 -- 172.21.15.81:0/3954952136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0fec15de70 con 0x7f0fec1478c0 2024-05-08T23:31:00.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:00.025+0000 7f0fe37fe700 1 -- 172.21.15.81:0/3954952136 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0fe40552f0 con 0x7f0fec1478c0 2024-05-08T23:31:00.034 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:31:00.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:00.029+0000 7f0ff1441700 1 -- 172.21.15.81:0/3954952136 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0fcc066760 msgr2=0x7f0fcc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:00.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:00.029+0000 7f0ff1441700 1 --2- 172.21.15.81:0/3954952136 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0fcc066760 0x7f0fcc068c20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc005930 tx=0x7f0fdc009040 comp rx=0 tx=0).stop 2024-05-08T23:31:00.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:00.029+0000 7f0ff1441700 1 -- 172.21.15.81:0/3954952136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fec1478c0 msgr2=0x7f0fec15d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:00.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:00.029+0000 7f0ff1441700 1 --2- 172.21.15.81:0/3954952136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fec1478c0 0x7f0fec15d8a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4007e40 tx=0x7f0fe400e3c0 comp rx=0 tx=0).stop 2024-05-08T23:31:00.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:00.029+0000 7f0ff1441700 1 -- 172.21.15.81:0/3954952136 shutdown_connections 2024-05-08T23:31:00.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:00.029+0000 7f0ff1441700 1 --2- 172.21.15.81:0/3954952136 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0fcc066760 0x7f0fcc068c20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:00.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:00.029+0000 7f0ff1441700 1 --2- 172.21.15.81:0/3954952136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fec1478c0 0x7f0fec15d8a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:00.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:00.029+0000 7f0ff1441700 1 --2- 172.21.15.81:0/3954952136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fec144d40 0x7f0fec15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:00.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:00.029+0000 7f0ff1441700 1 --2- 172.21.15.81:0/3954952136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fec0cb3b0 0x7f0fec133bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:00.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:00.029+0000 7f0ff1441700 1 -- 172.21.15.81:0/3954952136 >> 172.21.15.81:0/3954952136 conn(0x7f0fec06a360 msgr2=0x7f0fec146c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:00.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:00.029+0000 7f0ff1441700 1 -- 172.21.15.81:0/3954952136 shutdown_connections 2024-05-08T23:31:00.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:00.029+0000 7f0ff1441700 1 -- 172.21.15.81:0/3954952136 wait complete. 2024-05-08T23:31:00.046 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:31:00.046 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:31:00.046 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi117.front.sepia.ceph.com 2024-05-08T23:31:00.046 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-08T23:31:00.046 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-08T23:31:00.046 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:31:00.046 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:31:00.046 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:31:00.046 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-08T23:31:00.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:00.047 DEBUG:teuthology.orchestra.run.smithi117:> ip addr 2024-05-08T23:31:00.055 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:31:00.055 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:31:00.055 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:31:00.055 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:00.055 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-05-08T23:31:00.055 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:00.055 INFO:teuthology.orchestra.run.smithi117.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:31:00.055 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8a brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:00.055 INFO:teuthology.orchestra.run.smithi117.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:31:00.055 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8b brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:00.055 INFO:teuthology.orchestra.run.smithi117.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:31:00.055 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:62 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:00.056 INFO:teuthology.orchestra.run.smithi117.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:31:00.056 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:63 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:00.056 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.21.15.117/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T23:31:00.056 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft 42548sec preferred_lft 42548sec 2024-05-08T23:31:00.056 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-05-08T23:31:00.056 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:00.056 INFO:teuthology.orchestra.run.smithi117.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T23:31:00.056 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 02:42:94:b7:2a:7e brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:00.056 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T23:31:00.056 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:00.056 INFO:teuthology.orchestra.run.smithi117.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:31:00.056 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether d6:47:5d:f7:f1:2e brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:00.056 INFO:teuthology.orchestra.run.smithi117.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:31:00.056 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:00.057 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::fc16:27ff:fe0b:c0c8/64 scope link 2024-05-08T23:31:00.057 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:00.057 INFO:teuthology.orchestra.run.smithi117.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:31:00.057 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether d6:47:5d:f7:f1:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T23:31:00.057 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::d447:5dff:fef7:f12e/64 scope link 2024-05-08T23:31:00.057 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:00.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:00.057 DEBUG:teuthology.orchestra.run.smithi117:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:31:00.071 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-05-08T23:31:00.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:00.073 DEBUG:teuthology.orchestra.run.smithi117:> route 2024-05-08T23:31:00.120 INFO:teuthology.orchestra.run.smithi117.stdout:Kernel IP routing table 2024-05-08T23:31:00.120 INFO:teuthology.orchestra.run.smithi117.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:31:00.120 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T23:31:00.121 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T23:31:00.121 INFO:teuthology.orchestra.run.smithi117.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T23:31:00.121 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T23:31:00.121 INFO:teuthology.orchestra.run.smithi117.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T23:31:00.121 INFO:teuthology.orchestra.run.smithi117.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:31:00.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:00.121 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-05-08T23:31:00.121 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-05-08T23:31:00.121 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T23:31:00.121 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T23:31:00.121 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T23:31:00.122 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-05-08T23:31:00.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:00.157 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-05-08T23:31:00.163 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T23:31:00.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:00.163 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list-id 2024-05-08T23:31:00.212 INFO:teuthology.orchestra.run.smithi117.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-08T23:31:00.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:00.212 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-05-08T23:31:00.212 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-05-08T23:31:00.212 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T23:31:00.212 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-08T23:31:00.212 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-05-08T23:31:00.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:00.240 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-08T23:31:00.252 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:31:00.252 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:31:00.252 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:31:00.252 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:00.253 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-05-08T23:31:00.253 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:00.253 INFO:teuthology.orchestra.run.smithi117.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:31:00.253 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 92:78:77:b6:9c:f1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T23:31:00.253 INFO:teuthology.orchestra.run.smithi117.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T23:31:00.253 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:00.253 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::9078:77ff:feb6:9cf1/64 scope link 2024-05-08T23:31:00.253 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:00.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:00.254 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-08T23:31:00.314 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:31:00.314 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:31:00.314 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:31:00.314 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:00.314 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-05-08T23:31:00.314 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:00.315 INFO:teuthology.orchestra.run.smithi117.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:31:00.315 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 92:78:77:b6:9c:f1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T23:31:00.315 INFO:teuthology.orchestra.run.smithi117.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T23:31:00.315 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:00.315 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::9078:77ff:feb6:9cf1/64 scope link 2024-05-08T23:31:00.315 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:00.316 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-08T23:31:00.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:00.316 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-05-08T23:31:00.316 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-05-08T23:31:00.316 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-08T23:31:00.316 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-08T23:31:00.316 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-08T23:31:00.316 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-08T23:31:00.317 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-08T23:31:00.317 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-05-08T23:31:00.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:00.369 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-05-08T23:31:00.369 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-05-08T23:31:00.369 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set brx.1 up 2024-05-08T23:31:00.369 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set dev brx.1 master ceph-brx 2024-05-08T23:31:00.369 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-05-08T23:31:00.392 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-05-08T23:31:00.392 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:31:00.392 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-08T23:31:00.443 INFO:teuthology.orchestra.run.smithi117.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-08T23:31:00.443 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:31:00.443 DEBUG:teuthology.orchestra.run.smithi117:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-08T23:31:00.449 DEBUG:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2024-05-08T23:31:00.510 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:31:00.510 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:31:00.519 INFO:teuthology.orchestra.run.smithi117.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:31:00.520 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:31:00.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:00.520 DEBUG:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2024-05-08T23:31:00.525 INFO:teuthology.orchestra.run.smithi117.stdout:52 2024-05-08T23:31:00.525 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2024-05-08T23:31:00.525 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-05-08T23:31:00.527 DEBUG:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2024-05-08T23:31:00.543 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:31:00.543 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:31:00.604 INFO:teuthology.orchestra.run.smithi117.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:31:00.605 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:31:00.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:00.606 DEBUG:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2024-05-08T23:31:00.609 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi117.stderr:2024-05-08T23:31:00.606+0000 7f355eb0c640 -1 init, newargv = 0x55a906f3bce0 newargc=15 2024-05-08T23:31:00.609 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi117.stderr:ceph-fuse[31660]: starting ceph client 2024-05-08T23:31:00.613 INFO:teuthology.orchestra.run.smithi117.stdout:52 2024-05-08T23:31:00.616 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi117.stderr:ceph-fuse[31660]: starting fuse 2024-05-08T23:31:01.716 DEBUG:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2024-05-08T23:31:01.734 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:31:01.734 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:31:01.793 INFO:teuthology.orchestra.run.smithi117.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:31:01.794 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:31:01.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:01.795 DEBUG:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2024-05-08T23:31:01.800 INFO:teuthology.orchestra.run.smithi117.stdout:52 2024-05-08T23:31:01.800 INFO:teuthology.orchestra.run.smithi117.stdout:53 2024-05-08T23:31:01.800 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2024-05-08T23:31:01.800 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:31:01.800 DEBUG:teuthology.orchestra.run.smithi117:> import glob 2024-05-08T23:31:01.800 DEBUG:teuthology.orchestra.run.smithi117:> import re 2024-05-08T23:31:01.801 DEBUG:teuthology.orchestra.run.smithi117:> import os 2024-05-08T23:31:01.801 DEBUG:teuthology.orchestra.run.smithi117:> import subprocess 2024-05-08T23:31:01.801 DEBUG:teuthology.orchestra.run.smithi117:> 2024-05-08T23:31:01.801 DEBUG:teuthology.orchestra.run.smithi117:> def _find_admin_socket(client_name): 2024-05-08T23:31:01.801 DEBUG:teuthology.orchestra.run.smithi117:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-05-08T23:31:01.801 DEBUG:teuthology.orchestra.run.smithi117:> files = glob.glob(asok_path) 2024-05-08T23:31:01.801 DEBUG:teuthology.orchestra.run.smithi117:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-05-08T23:31:01.801 DEBUG:teuthology.orchestra.run.smithi117:> 2024-05-08T23:31:01.801 DEBUG:teuthology.orchestra.run.smithi117:> # Given a non-glob path, it better be there 2024-05-08T23:31:01.801 DEBUG:teuthology.orchestra.run.smithi117:> if "*" not in asok_path: 2024-05-08T23:31:01.801 DEBUG:teuthology.orchestra.run.smithi117:> assert(len(files) == 1) 2024-05-08T23:31:01.801 DEBUG:teuthology.orchestra.run.smithi117:> return files[0] 2024-05-08T23:31:01.801 DEBUG:teuthology.orchestra.run.smithi117:> 2024-05-08T23:31:01.801 DEBUG:teuthology.orchestra.run.smithi117:> for f in files: 2024-05-08T23:31:01.801 DEBUG:teuthology.orchestra.run.smithi117:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T23:31:01.802 DEBUG:teuthology.orchestra.run.smithi117:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T23:31:01.802 DEBUG:teuthology.orchestra.run.smithi117:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T23:31:01.802 DEBUG:teuthology.orchestra.run.smithi117:> contents = proc_f.read() 2024-05-08T23:31:01.802 DEBUG:teuthology.orchestra.run.smithi117:> if mountpoint in contents: 2024-05-08T23:31:01.802 DEBUG:teuthology.orchestra.run.smithi117:> return f 2024-05-08T23:31:01.802 DEBUG:teuthology.orchestra.run.smithi117:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T23:31:01.802 DEBUG:teuthology.orchestra.run.smithi117:> 2024-05-08T23:31:01.802 DEBUG:teuthology.orchestra.run.smithi117:> print(_find_admin_socket("client.3")) 2024-05-08T23:31:01.802 DEBUG:teuthology.orchestra.run.smithi117:> ' 2024-05-08T23:31:01.885 INFO:teuthology.orchestra.run.smithi117.stdout:/var/run/ceph/ceph-client.3.31660.asok 2024-05-08T23:31:02.075 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.31660.asok 2024-05-08T23:31:02.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:02.075 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.31660.asok status 2024-05-08T23:31:02.220 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-05-08T23:31:02.220 INFO:teuthology.orchestra.run.smithi117.stdout: "metadata": { 2024-05-08T23:31:02.220 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T23:31:02.221 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:31:02.221 INFO:teuthology.orchestra.run.smithi117.stdout: "entity_id": "3", 2024-05-08T23:31:02.221 INFO:teuthology.orchestra.run.smithi117.stdout: "hostname": "smithi117", 2024-05-08T23:31:02.221 INFO:teuthology.orchestra.run.smithi117.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-05-08T23:31:02.221 INFO:teuthology.orchestra.run.smithi117.stdout: "pid": "31660", 2024-05-08T23:31:02.221 INFO:teuthology.orchestra.run.smithi117.stdout: "root": "/" 2024-05-08T23:31:02.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-05-08T23:31:02.221 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_count": 0, 2024-05-08T23:31:02.221 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_pinned_count": 0, 2024-05-08T23:31:02.221 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 4738, 2024-05-08T23:31:02.221 INFO:teuthology.orchestra.run.smithi117.stdout: "inst": { 2024-05-08T23:31:02.221 INFO:teuthology.orchestra.run.smithi117.stdout: "name": { 2024-05-08T23:31:02.221 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client", 2024-05-08T23:31:02.222 INFO:teuthology.orchestra.run.smithi117.stdout: "num": 4738 2024-05-08T23:31:02.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-05-08T23:31:02.222 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2024-05-08T23:31:02.222 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2024-05-08T23:31:02.222 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "192.168.0.2:0", 2024-05-08T23:31:02.222 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 4132039454 2024-05-08T23:31:02.222 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-05-08T23:31:02.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-05-08T23:31:02.222 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2024-05-08T23:31:02.222 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2024-05-08T23:31:02.222 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "192.168.0.2:0", 2024-05-08T23:31:02.222 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 4132039454 2024-05-08T23:31:02.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-05-08T23:31:02.222 INFO:teuthology.orchestra.run.smithi117.stdout: "inst_str": "client.4738 192.168.0.2:0/4132039454", 2024-05-08T23:31:02.223 INFO:teuthology.orchestra.run.smithi117.stdout: "addr_str": "192.168.0.2:0/4132039454", 2024-05-08T23:31:02.223 INFO:teuthology.orchestra.run.smithi117.stdout: "inode_count": 1, 2024-05-08T23:31:02.223 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_epoch": 7, 2024-05-08T23:31:02.223 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch": 22, 2024-05-08T23:31:02.223 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch_barrier": 0, 2024-05-08T23:31:02.223 INFO:teuthology.orchestra.run.smithi117.stdout: "blocklisted": false, 2024-05-08T23:31:02.223 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs" 2024-05-08T23:31:02.223 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-05-08T23:31:02.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:02.230 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2024-05-08T23:31:02.230 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T23:31:02.238 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2024-05-08T23:31:02.239 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T23:31:02.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:02.239 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:31:02.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:02.298 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2024-05-08T23:31:02.298 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.2 2024-05-08T23:31:02.307 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2024-05-08T23:31:02.307 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-05-08T23:31:02.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:02.307 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-08T23:31:02.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:02.326 DEBUG:teuthology.orchestra.run.smithi117:> stat --file-system '--printf=%T 2024-05-08T23:31:02.326 DEBUG:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-08T23:31:02.333 INFO:teuthology.orchestra.run.smithi117.stdout:fuseblk 2024-05-08T23:31:02.334 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-08T23:31:02.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:02.334 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-08T23:31:02.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:02.350 DEBUG:teuthology.orchestra.run.smithi117:> stat --file-system '--printf=%T 2024-05-08T23:31:02.350 DEBUG:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.3 2024-05-08T23:31:02.401 INFO:teuthology.orchestra.run.smithi117.stdout:fuseblk 2024-05-08T23:31:02.401 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-05-08T23:31:02.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:02.401 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-08T23:31:02.418 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-08T23:31:02.461 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:02.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6cb096700 1 Processor -- start 2024-05-08T23:31:02.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6cb096700 1 -- start start 2024-05-08T23:31:02.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6cb096700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6c40ceb40 0x7ff6c40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:02.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6cb096700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff6c40d0b90 0x7ff6c40d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:02.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6cb096700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6c40cef80 con 0x7ff6c415e6d0 2024-05-08T23:31:02.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6cb096700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c40cdd00 con 0x7ff6c40d0b90 2024-05-08T23:31:02.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6cb096700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c415eb20 con 0x7ff6c40ceb40 2024-05-08T23:31:02.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6ca895700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff6c40d0b90 0x7ff6c40d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:02.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6ca895700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff6c40d0b90 0x7ff6c40d1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38902/0 (socket says 172.21.15.81:38902) 2024-05-08T23:31:02.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6ca895700 1 -- 172.21.15.81:0/4290541244 learned_addr learned my addr 172.21.15.81:0/4290541244 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:02.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6c9893700 1 --2- 172.21.15.81:0/4290541244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6c40ceb40 0x7ff6c40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:02.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6ca895700 1 -- 172.21.15.81:0/4290541244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6c40ceb40 msgr2=0x7ff6c40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:02.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6ca895700 1 --2- 172.21.15.81:0/4290541244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6c40ceb40 0x7ff6c40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:02.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6ca895700 1 -- 172.21.15.81:0/4290541244 >> v1:172.21.15.117:6789/0 conn(0x7ff6c415e6d0 legacy=0x7ff6c415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:02.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6ca895700 1 -- 172.21.15.81:0/4290541244 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c415f9f0 con 0x7ff6c40d0b90 2024-05-08T23:31:02.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6c9893700 1 --2- 172.21.15.81:0/4290541244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6c40ceb40 0x7ff6c40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:02.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6ca895700 1 --2- 172.21.15.81:0/4290541244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c40d0b90 0x7ff6c40d1020 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009910 tx=0x7ff6b4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb760e4c79750b46 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:02.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6c9092700 1 -- 172.21.15.81:0/4290541244 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b400ba60 con 0x7ff6c40d0b90 2024-05-08T23:31:02.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6c9092700 1 -- 172.21.15.81:0/4290541244 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b400bbc0 con 0x7ff6c40d0b90 2024-05-08T23:31:02.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.577+0000 7ff6c9092700 1 -- 172.21.15.81:0/4290541244 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4009e60 con 0x7ff6c40d0b90 2024-05-08T23:31:02.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.581+0000 7ff6cb096700 1 -- 172.21.15.81:0/4290541244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c40d0b90 msgr2=0x7ff6c40d1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:02.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.581+0000 7ff6cb096700 1 --2- 172.21.15.81:0/4290541244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c40d0b90 0x7ff6c40d1020 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009910 tx=0x7ff6b4009c20 comp rx=0 tx=0).stop 2024-05-08T23:31:02.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.581+0000 7ff6cb096700 1 -- 172.21.15.81:0/4290541244 shutdown_connections 2024-05-08T23:31:02.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.581+0000 7ff6cb096700 1 --2- 172.21.15.81:0/4290541244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c40d0b90 0x7ff6c40d1020 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:02.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.581+0000 7ff6cb096700 1 --2- 172.21.15.81:0/4290541244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6c40ceb40 0x7ff6c40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:02.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.581+0000 7ff6cb096700 1 -- 172.21.15.81:0/4290541244 >> 172.21.15.81:0/4290541244 conn(0x7ff6c4099a70 msgr2=0x7ff6c4099e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:02.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.581+0000 7ff6cb096700 1 -- 172.21.15.81:0/4290541244 shutdown_connections 2024-05-08T23:31:02.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.581+0000 7ff6cb096700 1 -- 172.21.15.81:0/4290541244 wait complete. 2024-05-08T23:31:02.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.581+0000 7ff6cb096700 1 Processor -- start 2024-05-08T23:31:02.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.581+0000 7ff6cb096700 1 -- start start 2024-05-08T23:31:02.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6cb096700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6c40ceb40 0x7ff6c40c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:02.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6cb096700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6c40d0b90 0x7ff6c40c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:02.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6cb096700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c415e6d0 0x7ff6c40af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:02.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6cb096700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c40c6070 con 0x7ff6c415e6d0 2024-05-08T23:31:02.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6cb096700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c40c6350 con 0x7ff6c40d0b90 2024-05-08T23:31:02.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6cb096700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c40c4950 con 0x7ff6c40ceb40 2024-05-08T23:31:02.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6ca895700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c415e6d0 0x7ff6c40af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:02.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6ca895700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c415e6d0 0x7ff6c40af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38908/0 (socket says 172.21.15.81:38908) 2024-05-08T23:31:02.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6ca895700 1 -- 172.21.15.81:0/1521303748 learned_addr learned my addr 172.21.15.81:0/1521303748 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:02.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6ca094700 1 --2- 172.21.15.81:0/1521303748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6c40ceb40 0x7ff6c40c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:02.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6c9893700 1 --2- 172.21.15.81:0/1521303748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6c40d0b90 0x7ff6c40c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:02.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6ca895700 1 -- 172.21.15.81:0/1521303748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6c40ceb40 msgr2=0x7ff6c40c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:02.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6ca895700 1 --2- 172.21.15.81:0/1521303748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6c40ceb40 0x7ff6c40c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:02.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6ca895700 1 -- 172.21.15.81:0/1521303748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6c40d0b90 msgr2=0x7ff6c40c5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:02.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6ca895700 1 --2- 172.21.15.81:0/1521303748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6c40d0b90 0x7ff6c40c5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:02.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6ca895700 1 -- 172.21.15.81:0/1521303748 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b4009570 con 0x7ff6c415e6d0 2024-05-08T23:31:02.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6ca895700 1 --2- 172.21.15.81:0/1521303748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c415e6d0 0x7ff6c40af1b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009c60 tx=0x7ff6b4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:02.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6c2ffd700 1 -- 172.21.15.81:0/1521303748 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4004250 con 0x7ff6c415e6d0 2024-05-08T23:31:02.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6c2ffd700 1 -- 172.21.15.81:0/1521303748 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b401f070 con 0x7ff6c415e6d0 2024-05-08T23:31:02.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6cb096700 1 -- 172.21.15.81:0/1521303748 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c40c4c30 con 0x7ff6c415e6d0 2024-05-08T23:31:02.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6c2ffd700 1 -- 172.21.15.81:0/1521303748 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b40045d0 con 0x7ff6c415e6d0 2024-05-08T23:31:02.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.585+0000 7ff6cb096700 1 -- 172.21.15.81:0/1521303748 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c40c6630 con 0x7ff6c415e6d0 2024-05-08T23:31:02.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.589+0000 7ff6cb096700 1 -- 172.21.15.81:0/1521303748 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c40c4e40 con 0x7ff6c415e6d0 2024-05-08T23:31:02.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.589+0000 7ff6c2ffd700 1 -- 172.21.15.81:0/1521303748 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff6b4021030 con 0x7ff6c415e6d0 2024-05-08T23:31:02.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.589+0000 7ff6c2ffd700 1 --2- 172.21.15.81:0/1521303748 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff6a4066730 0x7ff6a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:02.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.589+0000 7ff6ca094700 1 --2- 172.21.15.81:0/1521303748 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff6a4066730 0x7ff6a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:02.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.589+0000 7ff6c2ffd700 1 -- 172.21.15.81:0/1521303748 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6b40852d0 con 0x7ff6c415e6d0 2024-05-08T23:31:02.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.589+0000 7ff6ca094700 1 --2- 172.21.15.81:0/1521303748 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff6a4066730 0x7ff6a4068bf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40c90f0 tx=0x7ff6bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:02.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.597+0000 7ff6c2ffd700 1 -- 172.21.15.81:0/1521303748 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff6b40526d0 con 0x7ff6c415e6d0 2024-05-08T23:31:02.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.773+0000 7ff6cb096700 1 -- 172.21.15.81:0/1521303748 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6c40684b0 con 0x7ff6c415e6d0 2024-05-08T23:31:02.781 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.777+0000 7ff6c2ffd700 1 -- 172.21.15.81:0/1521303748 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7ff6b4055cf0 con 0x7ff6c415e6d0 2024-05-08T23:31:02.782 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:02.782 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:02.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.781+0000 7ff6cb096700 1 -- 172.21.15.81:0/1521303748 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff6a4066730 msgr2=0x7ff6a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:02.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.781+0000 7ff6cb096700 1 --2- 172.21.15.81:0/1521303748 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff6a4066730 0x7ff6a4068bf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40c90f0 tx=0x7ff6bc009040 comp rx=0 tx=0).stop 2024-05-08T23:31:02.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.781+0000 7ff6cb096700 1 -- 172.21.15.81:0/1521303748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c415e6d0 msgr2=0x7ff6c40af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:02.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.781+0000 7ff6cb096700 1 --2- 172.21.15.81:0/1521303748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c415e6d0 0x7ff6c40af1b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009c60 tx=0x7ff6b4004770 comp rx=0 tx=0).stop 2024-05-08T23:31:02.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.781+0000 7ff6cb096700 1 -- 172.21.15.81:0/1521303748 shutdown_connections 2024-05-08T23:31:02.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.781+0000 7ff6cb096700 1 --2- 172.21.15.81:0/1521303748 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff6a4066730 0x7ff6a4068bf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:02.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.781+0000 7ff6cb096700 1 --2- 172.21.15.81:0/1521303748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c415e6d0 0x7ff6c40af1b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:02.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.781+0000 7ff6cb096700 1 --2- 172.21.15.81:0/1521303748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6c40d0b90 0x7ff6c40c5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:02.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.781+0000 7ff6cb096700 1 --2- 172.21.15.81:0/1521303748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6c40ceb40 0x7ff6c40c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:02.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.781+0000 7ff6cb096700 1 -- 172.21.15.81:0/1521303748 >> 172.21.15.81:0/1521303748 conn(0x7ff6c4099a70 msgr2=0x7ff6c40b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:02.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.781+0000 7ff6cb096700 1 -- 172.21.15.81:0/1521303748 shutdown_connections 2024-05-08T23:31:02.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.781+0000 7ff6cb096700 1 -- 172.21.15.81:0/1521303748 wait complete. 2024-05-08T23:31:02.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:02.797 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:02.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.909+0000 7f95a759e700 1 Processor -- start 2024-05-08T23:31:02.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.909+0000 7f95a759e700 1 -- start start 2024-05-08T23:31:02.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a759e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f95a815d180 0x7f95a80cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:02.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a815d560 0x7f95a80d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:02.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a759e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a80ce200 con 0x7f95a80d2f60 2024-05-08T23:31:02.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a759e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a80ce480 con 0x7f95a815d180 2024-05-08T23:31:02.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80ce760 con 0x7f95a815d560 2024-05-08T23:31:02.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a659c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f95a815d180 0x7f95a80cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:02.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a5d9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a815d560 0x7f95a80d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:02.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a6d9d700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f95a80d2f60 0x7f95a80cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:55294/0 (socket says 172.21.15.81:55294) 2024-05-08T23:31:02.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a5d9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a815d560 0x7f95a80d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:43234/0 (socket says 172.21.15.81:43234) 2024-05-08T23:31:02.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a659c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f95a815d180 0x7f95a80cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:53850/0 (socket says 172.21.15.81:53850) 2024-05-08T23:31:02.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a6d9d700 1 -- 172.21.15.81:0/520537693 learned_addr learned my addr 172.21.15.81:0/520537693 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:02.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a5d9b700 1 -- 172.21.15.81:0/520537693 >> v2:172.21.15.117:3300/0 conn(0x7f95a815d180 msgr2=0x7f95a80cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:02.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a5d9b700 1 --2- 172.21.15.81:0/520537693 >> v2:172.21.15.117:3300/0 conn(0x7f95a815d180 0x7f95a80cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:02.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a5d9b700 1 -- 172.21.15.81:0/520537693 >> v1:172.21.15.117:6789/0 conn(0x7f95a80d2f60 legacy=0x7f95a80cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:02.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a5d9b700 1 -- 172.21.15.81:0/520537693 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a80cea40 con 0x7f95a815d560 2024-05-08T23:31:02.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a5d9b700 1 --2- 172.21.15.81:0/520537693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a815d560 0x7f95a80d1bf0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9598009850 tx=0x7f9598009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37e683c41e101f4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:02.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a559a700 1 -- 172.21.15.81:0/520537693 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598011830 con 0x7f95a815d560 2024-05-08T23:31:02.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a559a700 1 -- 172.21.15.81:0/520537693 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9598011990 con 0x7f95a815d560 2024-05-08T23:31:02.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a559a700 1 -- 172.21.15.81:0/520537693 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598011d40 con 0x7f95a815d560 2024-05-08T23:31:02.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a759e700 1 -- 172.21.15.81:0/520537693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a815d560 msgr2=0x7f95a80d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:02.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a759e700 1 --2- 172.21.15.81:0/520537693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a815d560 0x7f95a80d1bf0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9598009850 tx=0x7f9598009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:02.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a759e700 1 -- 172.21.15.81:0/520537693 shutdown_connections 2024-05-08T23:31:02.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a759e700 1 --2- 172.21.15.81:0/520537693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a815d560 0x7f95a80d1bf0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:02.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a759e700 1 --2- 172.21.15.81:0/520537693 >> v2:172.21.15.117:3300/0 conn(0x7f95a815d180 0x7f95a80cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:02.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a759e700 1 -- 172.21.15.81:0/520537693 >> 172.21.15.81:0/520537693 conn(0x7f95a808c530 msgr2=0x7f95a809c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:02.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a759e700 1 -- 172.21.15.81:0/520537693 shutdown_connections 2024-05-08T23:31:02.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.913+0000 7f95a759e700 1 -- 172.21.15.81:0/520537693 wait complete. 2024-05-08T23:31:02.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a759e700 1 Processor -- start 2024-05-08T23:31:02.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a759e700 1 -- start start 2024-05-08T23:31:02.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a80d2f60 0x7f95a81f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:02.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a815d180 0x7f95a8148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:02.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95a815d560 0x7f95a81491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:02.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a759e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80c9070 con 0x7f95a815d560 2024-05-08T23:31:02.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a759e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a814cf30 con 0x7f95a815d180 2024-05-08T23:31:02.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80b7da0 con 0x7f95a80d2f60 2024-05-08T23:31:02.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a6d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95a815d560 0x7f95a81491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:02.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a6d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95a815d560 0x7f95a81491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38924/0 (socket says 172.21.15.81:38924) 2024-05-08T23:31:02.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a6d9d700 1 -- 172.21.15.81:0/1552015743 learned_addr learned my addr 172.21.15.81:0/1552015743 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:02.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a659c700 1 --2- 172.21.15.81:0/1552015743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a80d2f60 0x7f95a81f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:02.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a5d9b700 1 --2- 172.21.15.81:0/1552015743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a815d180 0x7f95a8148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:02.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a6d9d700 1 -- 172.21.15.81:0/1552015743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a80d2f60 msgr2=0x7f95a81f20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:02.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a6d9d700 1 --2- 172.21.15.81:0/1552015743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a80d2f60 0x7f95a81f20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:02.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a6d9d700 1 -- 172.21.15.81:0/1552015743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a815d180 msgr2=0x7f95a8148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:02.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a6d9d700 1 --2- 172.21.15.81:0/1552015743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a815d180 0x7f95a8148c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:02.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a6d9d700 1 -- 172.21.15.81:0/1552015743 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95980094b0 con 0x7f95a815d560 2024-05-08T23:31:02.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f95a6d9d700 1 --2- 172.21.15.81:0/1552015743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95a815d560 0x7f95a81491c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f95a000ca70 tx=0x7f95a000caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:02.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.917+0000 7f9596ffd700 1 -- 172.21.15.81:0/1552015743 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0007480 con 0x7f95a815d560 2024-05-08T23:31:02.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.921+0000 7f9596ffd700 1 -- 172.21.15.81:0/1552015743 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a00075e0 con 0x7f95a815d560 2024-05-08T23:31:02.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.921+0000 7f95a759e700 1 -- 172.21.15.81:0/1552015743 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a80b8080 con 0x7f95a815d560 2024-05-08T23:31:02.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.921+0000 7f9596ffd700 1 -- 172.21.15.81:0/1552015743 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0007950 con 0x7f95a815d560 2024-05-08T23:31:02.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.921+0000 7f95a759e700 1 -- 172.21.15.81:0/1552015743 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a80b8640 con 0x7f95a815d560 2024-05-08T23:31:02.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.921+0000 7f95a759e700 1 -- 172.21.15.81:0/1552015743 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a80672a0 con 0x7f95a815d560 2024-05-08T23:31:02.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.921+0000 7f9596ffd700 1 -- 172.21.15.81:0/1552015743 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f95a001b3b0 con 0x7f95a815d560 2024-05-08T23:31:02.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.921+0000 7f9596ffd700 1 --2- 172.21.15.81:0/1552015743 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9588066730 0x7f9588068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:02.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.925+0000 7f95a659c700 1 --2- 172.21.15.81:0/1552015743 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9588066730 0x7f9588068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:02.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.925+0000 7f9596ffd700 1 -- 172.21.15.81:0/1552015743 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95a00838f0 con 0x7f95a815d560 2024-05-08T23:31:02.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.925+0000 7f95a659c700 1 --2- 172.21.15.81:0/1552015743 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9588066730 0x7f9588068bf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f959c009bf0 tx=0x7f959c009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:02.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:02.933+0000 7f9596ffd700 1 -- 172.21.15.81:0/1552015743 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f95a0050c70 con 0x7f95a815d560 2024-05-08T23:31:03.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.097+0000 7f95a759e700 1 -- 172.21.15.81:0/1552015743 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95a80d05d0 con 0x7f95a815d560 2024-05-08T23:31:03.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.097+0000 7f9596ffd700 1 -- 172.21.15.81:0/1552015743 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95a0017020 con 0x7f95a815d560 2024-05-08T23:31:03.103 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:03.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.097+0000 7f95a759e700 1 -- 172.21.15.81:0/1552015743 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9588066730 msgr2=0x7f9588068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.097+0000 7f95a759e700 1 --2- 172.21.15.81:0/1552015743 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9588066730 0x7f9588068bf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f959c009bf0 tx=0x7f959c009240 comp rx=0 tx=0).stop 2024-05-08T23:31:03.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.097+0000 7f95a759e700 1 -- 172.21.15.81:0/1552015743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95a815d560 msgr2=0x7f95a81491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.097+0000 7f95a759e700 1 --2- 172.21.15.81:0/1552015743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95a815d560 0x7f95a81491c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f95a000ca70 tx=0x7f95a000caa0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.097+0000 7f95a759e700 1 -- 172.21.15.81:0/1552015743 shutdown_connections 2024-05-08T23:31:03.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.097+0000 7f95a759e700 1 --2- 172.21.15.81:0/1552015743 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9588066730 0x7f9588068bf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.097+0000 7f95a759e700 1 --2- 172.21.15.81:0/1552015743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95a815d560 0x7f95a81491c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.097+0000 7f95a759e700 1 --2- 172.21.15.81:0/1552015743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a815d180 0x7f95a8148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.097+0000 7f95a759e700 1 --2- 172.21.15.81:0/1552015743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95a80d2f60 0x7f95a81f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.097+0000 7f95a759e700 1 -- 172.21.15.81:0/1552015743 >> 172.21.15.81:0/1552015743 conn(0x7f95a808c530 msgr2=0x7f95a80b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:03.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.097+0000 7f95a759e700 1 -- 172.21.15.81:0/1552015743 shutdown_connections 2024-05-08T23:31:03.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.101+0000 7f95a759e700 1 -- 172.21.15.81:0/1552015743 wait complete. 2024-05-08T23:31:03.120 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:03.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bb59e700 1 Processor -- start 2024-05-08T23:31:03.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bb59e700 1 -- start start 2024-05-08T23:31:03.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bb59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49bc143cd0 0x7f49bc1440b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bb59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f49bc06b020 0x7f49bc06a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bb59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f49bc0ad280 0x7f49bc06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bb59e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49bc1445f0 con 0x7f49bc0ad280 2024-05-08T23:31:03.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bb59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49bc0af840 con 0x7f49bc06b020 2024-05-08T23:31:03.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bb59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc0af310 con 0x7f49bc143cd0 2024-05-08T23:31:03.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bad9d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f49bc0ad280 0x7f49bc06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bad9d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f49bc0ad280 0x7f49bc06a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38928/0 (socket says 172.21.15.81:38928) 2024-05-08T23:31:03.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bad9d700 1 -- 172.21.15.81:0/1719012140 learned_addr learned my addr 172.21.15.81:0/1719012140 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:03.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49b9d9b700 1 --2- 172.21.15.81:0/1719012140 >> v2:172.21.15.117:3300/0 conn(0x7f49bc06b020 0x7f49bc06a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49ba59c700 1 --2- 172.21.15.81:0/1719012140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49bc143cd0 0x7f49bc1440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bad9d700 1 -- 172.21.15.81:0/1719012140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49bc143cd0 msgr2=0x7f49bc1440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:03.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bad9d700 1 --2- 172.21.15.81:0/1719012140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49bc143cd0 0x7f49bc1440b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bad9d700 1 -- 172.21.15.81:0/1719012140 >> v2:172.21.15.117:3300/0 conn(0x7f49bc06b020 msgr2=0x7f49bc06a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:03.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bad9d700 1 --2- 172.21.15.81:0/1719012140 >> v2:172.21.15.117:3300/0 conn(0x7f49bc06b020 0x7f49bc06a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49bad9d700 1 -- 172.21.15.81:0/1719012140 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc0af560 con 0x7f49bc0ad280 2024-05-08T23:31:03.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.229+0000 7f49b9d9b700 1 --2- 172.21.15.81:0/1719012140 >> v2:172.21.15.117:3300/0 conn(0x7f49bc06b020 0x7f49bc06a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:03.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.233+0000 7f49bad9d700 1 --2- 172.21.15.81:0/1719012140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49bc0ad280 0x7f49bc06a8a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0b5230 tx=0x7f49b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0f22403907dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:03.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.233+0000 7f49b959a700 1 -- 172.21.15.81:0/1719012140 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b400e070 con 0x7f49bc0ad280 2024-05-08T23:31:03.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.233+0000 7f49b959a700 1 -- 172.21.15.81:0/1719012140 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b400b9c0 con 0x7f49bc0ad280 2024-05-08T23:31:03.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.233+0000 7f49b959a700 1 -- 172.21.15.81:0/1719012140 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b400bd70 con 0x7f49bc0ad280 2024-05-08T23:31:03.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.233+0000 7f49bb59e700 1 -- 172.21.15.81:0/1719012140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49bc0ad280 msgr2=0x7f49bc06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.233+0000 7f49bb59e700 1 --2- 172.21.15.81:0/1719012140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49bc0ad280 0x7f49bc06a8a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0b5230 tx=0x7f49b400b330 comp rx=0 tx=0).stop 2024-05-08T23:31:03.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.233+0000 7f49bb59e700 1 -- 172.21.15.81:0/1719012140 shutdown_connections 2024-05-08T23:31:03.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.233+0000 7f49bb59e700 1 --2- 172.21.15.81:0/1719012140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49bc0ad280 0x7f49bc06a8a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.233+0000 7f49bb59e700 1 --2- 172.21.15.81:0/1719012140 >> v2:172.21.15.117:3300/0 conn(0x7f49bc06b020 0x7f49bc06a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.233+0000 7f49bb59e700 1 --2- 172.21.15.81:0/1719012140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49bc143cd0 0x7f49bc1440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.233+0000 7f49bb59e700 1 -- 172.21.15.81:0/1719012140 >> 172.21.15.81:0/1719012140 conn(0x7f49bc0b1a60 msgr2=0x7f49bc0b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:03.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.233+0000 7f49bb59e700 1 -- 172.21.15.81:0/1719012140 shutdown_connections 2024-05-08T23:31:03.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.233+0000 7f49bb59e700 1 -- 172.21.15.81:0/1719012140 wait complete. 2024-05-08T23:31:03.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.233+0000 7f49bb59e700 1 Processor -- start 2024-05-08T23:31:03.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.233+0000 7f49bb59e700 1 -- start start 2024-05-08T23:31:03.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bb59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49bc06b020 0x7f49bc09c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bb59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49bc0ad280 0x7f49bc099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bb59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49bc143cd0 0x7f49bc132e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bb59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc131dc0 con 0x7f49bc143cd0 2024-05-08T23:31:03.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bb59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc1320a0 con 0x7f49bc0ad280 2024-05-08T23:31:03.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bb59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc132380 con 0x7f49bc06b020 2024-05-08T23:31:03.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49bc143cd0 0x7f49bc132e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49b9d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49bc0ad280 0x7f49bc099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49ba59c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49bc06b020 0x7f49bc09c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49ba59c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49bc06b020 0x7f49bc09c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:43260/0 (socket says 172.21.15.81:43260) 2024-05-08T23:31:03.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49b9d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49bc0ad280 0x7f49bc099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:53880/0 (socket says 172.21.15.81:53880) 2024-05-08T23:31:03.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49b9d9b700 1 -- 172.21.15.81:0/217809963 learned_addr learned my addr 172.21.15.81:0/217809963 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:03.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49bc143cd0 0x7f49bc132e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38936/0 (socket says 172.21.15.81:38936) 2024-05-08T23:31:03.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bad9d700 1 -- 172.21.15.81:0/217809963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49bc06b020 msgr2=0x7f49bc09c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bad9d700 1 --2- 172.21.15.81:0/217809963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49bc06b020 0x7f49bc09c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bad9d700 1 -- 172.21.15.81:0/217809963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49bc0ad280 msgr2=0x7f49bc099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bad9d700 1 --2- 172.21.15.81:0/217809963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49bc0ad280 0x7f49bc099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bad9d700 1 -- 172.21.15.81:0/217809963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b4009b30 con 0x7f49bc143cd0 2024-05-08T23:31:03.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49ba59c700 1 --2- 172.21.15.81:0/217809963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49bc06b020 0x7f49bc09c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:03.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49b9d9b700 1 --2- 172.21.15.81:0/217809963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49bc0ad280 0x7f49bc099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:03.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bad9d700 1 --2- 172.21.15.81:0/217809963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49bc143cd0 0x7f49bc132e40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f49b4015000 tx=0x7f49b4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:03.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49aaffd700 1 -- 172.21.15.81:0/217809963 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b40099b0 con 0x7f49bc143cd0 2024-05-08T23:31:03.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49aaffd700 1 -- 172.21.15.81:0/217809963 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b4004250 con 0x7f49bc143cd0 2024-05-08T23:31:03.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bb59e700 1 -- 172.21.15.81:0/217809963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49bc132660 con 0x7f49bc143cd0 2024-05-08T23:31:03.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49aaffd700 1 -- 172.21.15.81:0/217809963 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4004600 con 0x7f49bc143cd0 2024-05-08T23:31:03.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bb59e700 1 -- 172.21.15.81:0/217809963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc09cd20 con 0x7f49bc143cd0 2024-05-08T23:31:03.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.237+0000 7f49bb59e700 1 -- 172.21.15.81:0/217809963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49bc09ceb0 con 0x7f49bc143cd0 2024-05-08T23:31:03.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.241+0000 7f49aaffd700 1 -- 172.21.15.81:0/217809963 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f49b4004760 con 0x7f49bc143cd0 2024-05-08T23:31:03.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.241+0000 7f49aaffd700 1 --2- 172.21.15.81:0/217809963 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f499c066730 0x7f499c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.241+0000 7f49aaffd700 1 -- 172.21.15.81:0/217809963 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49b4085570 con 0x7f49bc143cd0 2024-05-08T23:31:03.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.241+0000 7f49ba59c700 1 --2- 172.21.15.81:0/217809963 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f499c066730 0x7f499c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.241+0000 7f49ba59c700 1 --2- 172.21.15.81:0/217809963 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f499c066730 0x7f499c068bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f49ac007dd0 tx=0x7f49ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:03.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.249+0000 7f49aaffd700 1 -- 172.21.15.81:0/217809963 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f49b4052900 con 0x7f49bc143cd0 2024-05-08T23:31:03.422 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.417+0000 7f49bb59e700 1 -- 172.21.15.81:0/217809963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49bc0672a0 con 0x7f49bc143cd0 2024-05-08T23:31:03.422 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.417+0000 7f49aaffd700 1 -- 172.21.15.81:0/217809963 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49b4055f20 con 0x7f49bc143cd0 2024-05-08T23:31:03.422 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:03.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.421+0000 7f49bb59e700 1 -- 172.21.15.81:0/217809963 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f499c066730 msgr2=0x7f499c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.421+0000 7f49bb59e700 1 --2- 172.21.15.81:0/217809963 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f499c066730 0x7f499c068bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f49ac007dd0 tx=0x7f49ac008040 comp rx=0 tx=0).stop 2024-05-08T23:31:03.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.421+0000 7f49bb59e700 1 -- 172.21.15.81:0/217809963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49bc143cd0 msgr2=0x7f49bc132e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.421+0000 7f49bb59e700 1 --2- 172.21.15.81:0/217809963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49bc143cd0 0x7f49bc132e40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f49b4015000 tx=0x7f49b4004af0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.421+0000 7f49bb59e700 1 -- 172.21.15.81:0/217809963 shutdown_connections 2024-05-08T23:31:03.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.421+0000 7f49bb59e700 1 --2- 172.21.15.81:0/217809963 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f499c066730 0x7f499c068bf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.421+0000 7f49bb59e700 1 --2- 172.21.15.81:0/217809963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49bc143cd0 0x7f49bc132e40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.421+0000 7f49bb59e700 1 --2- 172.21.15.81:0/217809963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49bc0ad280 0x7f49bc099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.421+0000 7f49bb59e700 1 --2- 172.21.15.81:0/217809963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49bc06b020 0x7f49bc09c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.421+0000 7f49bb59e700 1 -- 172.21.15.81:0/217809963 >> 172.21.15.81:0/217809963 conn(0x7f49bc0b1a60 msgr2=0x7f49bc0b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:03.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.421+0000 7f49bb59e700 1 -- 172.21.15.81:0/217809963 shutdown_connections 2024-05-08T23:31:03.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.421+0000 7f49bb59e700 1 -- 172.21.15.81:0/217809963 wait complete. 2024-05-08T23:31:03.440 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:03.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.557+0000 7f591b59e700 1 Processor -- start 2024-05-08T23:31:03.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.557+0000 7f591b59e700 1 -- start start 2024-05-08T23:31:03.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.557+0000 7f591b59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f591c160920 0x7f591c0ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.557+0000 7f591b59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c0d0050 0x7f591c0b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.557+0000 7f591b59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591c0d3d40 con 0x7f591c0d1da0 2024-05-08T23:31:03.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.557+0000 7f591b59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591c0d0d00 con 0x7f591c160920 2024-05-08T23:31:03.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.557+0000 7f591b59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c0d0fe0 con 0x7f591c0d0050 2024-05-08T23:31:03.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f5919d9b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f591c0d1da0 0x7f591c0cf940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:55306/0 (socket says 172.21.15.81:55306) 2024-05-08T23:31:03.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591a59c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f591c160920 0x7f591c0ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f5919d9b700 1 -- 172.21.15.81:0/4156516683 learned_addr learned my addr 172.21.15.81:0/4156516683 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:03.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591ad9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c0d0050 0x7f591c0b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591959a700 1 -- 172.21.15.81:0/4156516683 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166612997 0 0) 0x7f591c0d3d40 con 0x7f591c0d1da0 2024-05-08T23:31:03.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591959a700 1 -- 172.21.15.81:0/4156516683 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59040035d0 con 0x7f591c0d1da0 2024-05-08T23:31:03.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591ad9d700 1 -- 172.21.15.81:0/4156516683 >> v2:172.21.15.117:3300/0 conn(0x7f591c160920 msgr2=0x7f591c0ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:03.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591ad9d700 1 --2- 172.21.15.81:0/4156516683 >> v2:172.21.15.117:3300/0 conn(0x7f591c160920 0x7f591c0ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591ad9d700 1 -- 172.21.15.81:0/4156516683 >> v1:172.21.15.117:6789/0 conn(0x7f591c0d1da0 legacy=0x7f591c0cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591ad9d700 1 -- 172.21.15.81:0/4156516683 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c0d12c0 con 0x7f591c0d0050 2024-05-08T23:31:03.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591a59c700 1 --2- 172.21.15.81:0/4156516683 >> v2:172.21.15.117:3300/0 conn(0x7f591c160920 0x7f591c0ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:03.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591ad9d700 1 --2- 172.21.15.81:0/4156516683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c0d0050 0x7f591c0b5e50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5914009d90 tx=0x7f591400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b521429c60d6af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:03.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591959a700 1 -- 172.21.15.81:0/4156516683 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591400e070 con 0x7f591c0d0050 2024-05-08T23:31:03.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591959a700 1 -- 172.21.15.81:0/4156516683 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591400bb20 con 0x7f591c0d0050 2024-05-08T23:31:03.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591959a700 1 -- 172.21.15.81:0/4156516683 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914012430 con 0x7f591c0d0050 2024-05-08T23:31:03.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591b59e700 1 -- 172.21.15.81:0/4156516683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c0d0050 msgr2=0x7f591c0b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591b59e700 1 --2- 172.21.15.81:0/4156516683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c0d0050 0x7f591c0b5e50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5914009d90 tx=0x7f591400b330 comp rx=0 tx=0).stop 2024-05-08T23:31:03.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591b59e700 1 -- 172.21.15.81:0/4156516683 shutdown_connections 2024-05-08T23:31:03.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591b59e700 1 --2- 172.21.15.81:0/4156516683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c0d0050 0x7f591c0b5e50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591b59e700 1 --2- 172.21.15.81:0/4156516683 >> v2:172.21.15.117:3300/0 conn(0x7f591c160920 0x7f591c0ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591b59e700 1 -- 172.21.15.81:0/4156516683 >> 172.21.15.81:0/4156516683 conn(0x7f591c09c340 msgr2=0x7f591c0d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:03.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591b59e700 1 -- 172.21.15.81:0/4156516683 shutdown_connections 2024-05-08T23:31:03.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.561+0000 7f591b59e700 1 -- 172.21.15.81:0/4156516683 wait complete. 2024-05-08T23:31:03.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f591b59e700 1 Processor -- start 2024-05-08T23:31:03.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f591b59e700 1 -- start start 2024-05-08T23:31:03.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f591b59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c0d0050 0x7f591c1f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f591b59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f591c0d1da0 0x7f591c14f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f591b59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f591c160920 0x7f591c0b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f591b59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c150f10 con 0x7f591c0d1da0 2024-05-08T23:31:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f591b59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c151330 con 0x7f591c160920 2024-05-08T23:31:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f591b59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c1f2a70 con 0x7f591c0d0050 2024-05-08T23:31:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f5919d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f591c0d1da0 0x7f591c14f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f5919d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f591c0d1da0 0x7f591c14f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38952/0 (socket says 172.21.15.81:38952) 2024-05-08T23:31:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f5919d9b700 1 -- 172.21.15.81:0/2481604713 learned_addr learned my addr 172.21.15.81:0/2481604713 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f591a59c700 1 --2- 172.21.15.81:0/2481604713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c0d0050 0x7f591c1f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f591ad9d700 1 --2- 172.21.15.81:0/2481604713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f591c160920 0x7f591c0b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f5919d9b700 1 -- 172.21.15.81:0/2481604713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c0d0050 msgr2=0x7f591c1f2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f5919d9b700 1 --2- 172.21.15.81:0/2481604713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c0d0050 0x7f591c1f2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f5919d9b700 1 -- 172.21.15.81:0/2481604713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f591c160920 msgr2=0x7f591c0b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f5919d9b700 1 --2- 172.21.15.81:0/2481604713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f591c160920 0x7f591c0b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f5919d9b700 1 -- 172.21.15.81:0/2481604713 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59100094b0 con 0x7f591c0d1da0 2024-05-08T23:31:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f5919d9b700 1 --2- 172.21.15.81:0/2481604713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f591c0d1da0 0x7f591c14f880 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f590c00c4d0 tx=0x7f590c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f590affd700 1 -- 172.21.15.81:0/2481604713 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c005d90 con 0x7f591c0d1da0 2024-05-08T23:31:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f590affd700 1 -- 172.21.15.81:0/2481604713 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c0070d0 con 0x7f591c0d1da0 2024-05-08T23:31:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f591b59e700 1 -- 172.21.15.81:0/2481604713 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5914009a70 con 0x7f591c0d1da0 2024-05-08T23:31:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f590affd700 1 -- 172.21.15.81:0/2481604713 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c007480 con 0x7f591c0d1da0 2024-05-08T23:31:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.565+0000 7f591b59e700 1 -- 172.21.15.81:0/2481604713 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c160e80 con 0x7f591c0d1da0 2024-05-08T23:31:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.569+0000 7f591b59e700 1 -- 172.21.15.81:0/2481604713 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591c151050 con 0x7f591c0d1da0 2024-05-08T23:31:03.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.569+0000 7f590affd700 1 -- 172.21.15.81:0/2481604713 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f590c0075e0 con 0x7f591c0d1da0 2024-05-08T23:31:03.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.569+0000 7f590affd700 1 --2- 172.21.15.81:0/2481604713 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f58fc066760 0x7f58fc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.569+0000 7f591a59c700 1 --2- 172.21.15.81:0/2481604713 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f58fc066760 0x7f58fc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.569+0000 7f590affd700 1 -- 172.21.15.81:0/2481604713 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f590c083860 con 0x7f591c0d1da0 2024-05-08T23:31:03.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.573+0000 7f591a59c700 1 --2- 172.21.15.81:0/2481604713 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f58fc066760 0x7f58fc068c20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5910009c70 tx=0x7f5910008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:03.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.581+0000 7f590affd700 1 -- 172.21.15.81:0/2481604713 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f591c151050 con 0x7f591c0d1da0 2024-05-08T23:31:03.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.745+0000 7f591b59e700 1 -- 172.21.15.81:0/2481604713 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f591c151050 con 0x7f591c0d1da0 2024-05-08T23:31:03.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.749+0000 7f590affd700 1 -- 172.21.15.81:0/2481604713 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f591c151050 con 0x7f591c0d1da0 2024-05-08T23:31:03.752 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:03.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.749+0000 7f591b59e700 1 -- 172.21.15.81:0/2481604713 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f58fc066760 msgr2=0x7f58fc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.749+0000 7f591b59e700 1 --2- 172.21.15.81:0/2481604713 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f58fc066760 0x7f58fc068c20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5910009c70 tx=0x7f5910008040 comp rx=0 tx=0).stop 2024-05-08T23:31:03.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.749+0000 7f591b59e700 1 -- 172.21.15.81:0/2481604713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f591c0d1da0 msgr2=0x7f591c14f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.749+0000 7f591b59e700 1 --2- 172.21.15.81:0/2481604713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f591c0d1da0 0x7f591c14f880 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f590c00c4d0 tx=0x7f590c00c890 comp rx=0 tx=0).stop 2024-05-08T23:31:03.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.749+0000 7f591b59e700 1 -- 172.21.15.81:0/2481604713 shutdown_connections 2024-05-08T23:31:03.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.749+0000 7f591b59e700 1 --2- 172.21.15.81:0/2481604713 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f58fc066760 0x7f58fc068c20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.749+0000 7f591b59e700 1 --2- 172.21.15.81:0/2481604713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f591c160920 0x7f591c0b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.749+0000 7f591b59e700 1 --2- 172.21.15.81:0/2481604713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f591c0d1da0 0x7f591c14f880 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.749+0000 7f591b59e700 1 --2- 172.21.15.81:0/2481604713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c0d0050 0x7f591c1f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.749+0000 7f591b59e700 1 -- 172.21.15.81:0/2481604713 >> 172.21.15.81:0/2481604713 conn(0x7f591c09c340 msgr2=0x7f591c15df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:03.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.749+0000 7f591b59e700 1 -- 172.21.15.81:0/2481604713 shutdown_connections 2024-05-08T23:31:03.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.749+0000 7f591b59e700 1 -- 172.21.15.81:0/2481604713 wait complete. 2024-05-08T23:31:03.767 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:03.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.877+0000 7f58053b5700 1 Processor -- start 2024-05-08T23:31:03.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.877+0000 7f58053b5700 1 -- start start 2024-05-08T23:31:03.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.877+0000 7f58053b5700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58000b3e40 0x7f580006b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.877+0000 7f58053b5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f580006b8b0 0x7f58000cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.877+0000 7f58053b5700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580015d1b0 con 0x7f5800153e40 2024-05-08T23:31:03.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.877+0000 7f58053b5700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58000ccf50 con 0x7f580006b8b0 2024-05-08T23:31:03.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.877+0000 7f57fffff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5800153e40 0x7f58000b4340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44566/0 (socket says 172.21.15.81:44566) 2024-05-08T23:31:03.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.877+0000 7f58053b5700 1 -- 172.21.15.81:0/3643974341 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580006bd40 con 0x7f58000b3e40 2024-05-08T23:31:03.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.877+0000 7f57fffff700 1 -- 172.21.15.81:0/3643974341 learned_addr learned my addr 172.21.15.81:0/3643974341 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:03.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.877+0000 7f5804bb4700 1 --2- 172.21.15.81:0/3643974341 >> v2:172.21.15.81:3300/0 conn(0x7f580006b8b0 0x7f58000cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.877+0000 7f57ff7fe700 1 --2- 172.21.15.81:0/3643974341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58000b3e40 0x7f580006b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.877+0000 7f57feffd700 1 -- 172.21.15.81:0/3643974341 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414156599 0 0) 0x7f580015d1b0 con 0x7f5800153e40 2024-05-08T23:31:03.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.877+0000 7f57feffd700 1 -- 172.21.15.81:0/3643974341 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e80035d0 con 0x7f5800153e40 2024-05-08T23:31:03.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f5804bb4700 1 -- 172.21.15.81:0/3643974341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58000b3e40 msgr2=0x7f580006b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:03.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f5804bb4700 1 --2- 172.21.15.81:0/3643974341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58000b3e40 0x7f580006b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f5804bb4700 1 -- 172.21.15.81:0/3643974341 >> v1:172.21.15.81:6789/0 conn(0x7f5800153e40 legacy=0x7f58000b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f5804bb4700 1 -- 172.21.15.81:0/3643974341 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580015d450 con 0x7f580006b8b0 2024-05-08T23:31:03.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f57ff7fe700 1 --2- 172.21.15.81:0/3643974341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58000b3e40 0x7f580006b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:03.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f5804bb4700 1 --2- 172.21.15.81:0/3643974341 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f580006b8b0 0x7f58000cd750 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009d90 tx=0x7f57f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8eba83a4b37ff68 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:03.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f57feffd700 1 -- 172.21.15.81:0/3643974341 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f800e070 con 0x7f580006b8b0 2024-05-08T23:31:03.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f57feffd700 1 -- 172.21.15.81:0/3643974341 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f800bb20 con 0x7f580006b8b0 2024-05-08T23:31:03.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f57feffd700 1 -- 172.21.15.81:0/3643974341 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f800be90 con 0x7f580006b8b0 2024-05-08T23:31:03.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f58053b5700 1 -- 172.21.15.81:0/3643974341 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f580006b8b0 msgr2=0x7f58000cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f58053b5700 1 --2- 172.21.15.81:0/3643974341 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f580006b8b0 0x7f58000cd750 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009d90 tx=0x7f57f800b330 comp rx=0 tx=0).stop 2024-05-08T23:31:03.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f58053b5700 1 -- 172.21.15.81:0/3643974341 shutdown_connections 2024-05-08T23:31:03.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f58053b5700 1 --2- 172.21.15.81:0/3643974341 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f580006b8b0 0x7f58000cd750 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f58053b5700 1 --2- 172.21.15.81:0/3643974341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58000b3e40 0x7f580006b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f58053b5700 1 -- 172.21.15.81:0/3643974341 >> 172.21.15.81:0/3643974341 conn(0x7f5800070960 msgr2=0x7f5800070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:03.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f58053b5700 1 -- 172.21.15.81:0/3643974341 shutdown_connections 2024-05-08T23:31:03.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.881+0000 7f58053b5700 1 -- 172.21.15.81:0/3643974341 wait complete. 2024-05-08T23:31:03.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f58053b5700 1 Processor -- start 2024-05-08T23:31:03.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f58053b5700 1 -- start start 2024-05-08T23:31:03.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f58053b5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f580006b8b0 0x7f58000a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f58053b5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58000b3e40 0x7f58000b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f58053b5700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5800153e40 0x7f5800158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f58053b5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580015a740 con 0x7f58000b3e40 2024-05-08T23:31:03.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f58053b5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58000a8890 con 0x7f580006b8b0 2024-05-08T23:31:03.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f58053b5700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5800149730 con 0x7f5800153e40 2024-05-08T23:31:03.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f57ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58000b3e40 0x7f58000b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f57ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58000b3e40 0x7f58000b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38968/0 (socket says 172.21.15.81:38968) 2024-05-08T23:31:03.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f57ff7fe700 1 -- 172.21.15.81:0/3581270534 learned_addr learned my addr 172.21.15.81:0/3581270534 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:03.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f57fffff700 1 --2- 172.21.15.81:0/3581270534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f580006b8b0 0x7f58000a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f5804bb4700 1 --2- 172.21.15.81:0/3581270534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5800153e40 0x7f5800158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f57ff7fe700 1 -- 172.21.15.81:0/3581270534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5800153e40 msgr2=0x7f5800158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f57ff7fe700 1 --2- 172.21.15.81:0/3581270534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5800153e40 0x7f5800158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f57ff7fe700 1 -- 172.21.15.81:0/3581270534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f580006b8b0 msgr2=0x7f58000a8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:03.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f57ff7fe700 1 --2- 172.21.15.81:0/3581270534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f580006b8b0 0x7f58000a8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:03.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f57ff7fe700 1 -- 172.21.15.81:0/3581270534 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f8009a70 con 0x7f58000b3e40 2024-05-08T23:31:03.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f57ff7fe700 1 --2- 172.21.15.81:0/3581270534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58000b3e40 0x7f58000b2670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f57f400e760 tx=0x7f57f400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:03.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f57fcff9700 1 -- 172.21.15.81:0/3581270534 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4004230 con 0x7f58000b3e40 2024-05-08T23:31:03.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f57fcff9700 1 -- 172.21.15.81:0/3581270534 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f40043b0 con 0x7f58000b3e40 2024-05-08T23:31:03.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f58053b5700 1 -- 172.21.15.81:0/3581270534 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5800149a10 con 0x7f58000b3e40 2024-05-08T23:31:03.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f57fcff9700 1 -- 172.21.15.81:0/3581270534 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f40036c0 con 0x7f58000b3e40 2024-05-08T23:31:03.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.885+0000 7f58053b5700 1 -- 172.21.15.81:0/3581270534 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5800149fd0 con 0x7f58000b3e40 2024-05-08T23:31:03.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.889+0000 7f58053b5700 1 -- 172.21.15.81:0/3581270534 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5800157000 con 0x7f58000b3e40 2024-05-08T23:31:03.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.889+0000 7f57fcff9700 1 -- 172.21.15.81:0/3581270534 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f57f4003820 con 0x7f58000b3e40 2024-05-08T23:31:03.894 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.889+0000 7f57fcff9700 1 --2- 172.21.15.81:0/3581270534 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f57ec066730 0x7f57ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:03.894 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.889+0000 7f57fcff9700 1 -- 172.21.15.81:0/3581270534 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57f4083c80 con 0x7f58000b3e40 2024-05-08T23:31:03.894 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.889+0000 7f57fffff700 1 --2- 172.21.15.81:0/3581270534 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f57ec066730 0x7f57ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:03.894 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.889+0000 7f57fffff700 1 --2- 172.21.15.81:0/3581270534 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f57ec066730 0x7f57ec068bf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f57f00011e0 tx=0x7f57f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:03.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:03.897+0000 7f57fcff9700 1 -- 172.21.15.81:0/3581270534 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f57f4050ff0 con 0x7f58000b3e40 2024-05-08T23:31:04.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.069+0000 7f58053b5700 1 -- 172.21.15.81:0/3581270534 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58000672a0 con 0x7f58000b3e40 2024-05-08T23:31:04.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.069+0000 7f57fcff9700 1 -- 172.21.15.81:0/3581270534 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f57f4020090 con 0x7f58000b3e40 2024-05-08T23:31:04.075 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:04.075 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:04.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.073+0000 7f58053b5700 1 -- 172.21.15.81:0/3581270534 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f57ec066730 msgr2=0x7f57ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.073+0000 7f58053b5700 1 --2- 172.21.15.81:0/3581270534 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f57ec066730 0x7f57ec068bf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f57f00011e0 tx=0x7f57f0009040 comp rx=0 tx=0).stop 2024-05-08T23:31:04.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.073+0000 7f58053b5700 1 -- 172.21.15.81:0/3581270534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58000b3e40 msgr2=0x7f58000b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.073+0000 7f58053b5700 1 --2- 172.21.15.81:0/3581270534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58000b3e40 0x7f58000b2670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f57f400e760 tx=0x7f57f400eb20 comp rx=0 tx=0).stop 2024-05-08T23:31:04.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.073+0000 7f58053b5700 1 -- 172.21.15.81:0/3581270534 shutdown_connections 2024-05-08T23:31:04.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.073+0000 7f58053b5700 1 --2- 172.21.15.81:0/3581270534 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f57ec066730 0x7f57ec068bf0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.073+0000 7f58053b5700 1 --2- 172.21.15.81:0/3581270534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5800153e40 0x7f5800158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.073+0000 7f58053b5700 1 --2- 172.21.15.81:0/3581270534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58000b3e40 0x7f58000b2670 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.073+0000 7f58053b5700 1 --2- 172.21.15.81:0/3581270534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f580006b8b0 0x7f58000a8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.073+0000 7f58053b5700 1 -- 172.21.15.81:0/3581270534 >> 172.21.15.81:0/3581270534 conn(0x7f5800070960 msgr2=0x7f58001568f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:04.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.073+0000 7f58053b5700 1 -- 172.21.15.81:0/3581270534 shutdown_connections 2024-05-08T23:31:04.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.073+0000 7f58053b5700 1 -- 172.21.15.81:0/3581270534 wait complete. 2024-05-08T23:31:04.093 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:04.093 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:04.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.205+0000 7f1bd73f9700 1 Processor -- start 2024-05-08T23:31:04.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.205+0000 7f1bd73f9700 1 -- start start 2024-05-08T23:31:04.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.205+0000 7f1bd73f9700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1bc80507d0 0x7f1bc80dfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.205+0000 7f1bd73f9700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc80e0600 con 0x7f1bc804ff30 2024-05-08T23:31:04.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.205+0000 7f1bd73f9700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc80e08e0 con 0x7f1bc80e0160 2024-05-08T23:31:04.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.205+0000 7f1bd73f9700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc80d6b70 con 0x7f1bc80507d0 2024-05-08T23:31:04.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.205+0000 7f1bd6bf8700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f1bc804ff30 0x7f1bc80335c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44574/0 (socket says 172.21.15.81:44574) 2024-05-08T23:31:04.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.205+0000 7f1bd6bf8700 1 -- 172.21.15.81:0/1246649841 learned_addr learned my addr 172.21.15.81:0/1246649841 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:04.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.205+0000 7f1bd63f7700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1bc80507d0 0x7f1bc80dfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.205+0000 7f1bd63f7700 1 -- 172.21.15.81:0/1246649841 >> v1:172.21.15.117:6789/0 conn(0x7f1bc80e0160 legacy=0x7f1bc80067d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.205+0000 7f1bd63f7700 1 -- 172.21.15.81:0/1246649841 >> v1:172.21.15.81:6789/0 conn(0x7f1bc804ff30 legacy=0x7f1bc80335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.205+0000 7f1bd53f5700 1 -- 172.21.15.81:0/1246649841 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290376124 0 0) 0x7f1bc80e0600 con 0x7f1bc804ff30 2024-05-08T23:31:04.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.205+0000 7f1bd63f7700 1 -- 172.21.15.81:0/1246649841 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc80528e0 con 0x7f1bc80507d0 2024-05-08T23:31:04.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.209+0000 7f1bd63f7700 1 --2- 172.21.15.81:0/1246649841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bc80507d0 0x7f1bc80dfc20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0097d0 tx=0x7f1bcc009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16d20b7614cc2da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:04.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.209+0000 7f1bd53f5700 1 -- 172.21.15.81:0/1246649841 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc004230 con 0x7f1bc80507d0 2024-05-08T23:31:04.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.209+0000 7f1bd53f5700 1 -- 172.21.15.81:0/1246649841 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc004390 con 0x7f1bc80507d0 2024-05-08T23:31:04.214 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.209+0000 7f1bd53f5700 1 -- 172.21.15.81:0/1246649841 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0036c0 con 0x7f1bc80507d0 2024-05-08T23:31:04.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.209+0000 7f1bd73f9700 1 -- 172.21.15.81:0/1246649841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bc80507d0 msgr2=0x7f1bc80dfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.209+0000 7f1bd73f9700 1 --2- 172.21.15.81:0/1246649841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bc80507d0 0x7f1bc80dfc20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0097d0 tx=0x7f1bcc009ae0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.209+0000 7f1bd73f9700 1 -- 172.21.15.81:0/1246649841 shutdown_connections 2024-05-08T23:31:04.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.209+0000 7f1bd73f9700 1 --2- 172.21.15.81:0/1246649841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bc80507d0 0x7f1bc80dfc20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.209+0000 7f1bd73f9700 1 -- 172.21.15.81:0/1246649841 >> 172.21.15.81:0/1246649841 conn(0x7f1bc8054b70 msgr2=0x7f1bc8054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:04.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.209+0000 7f1bd73f9700 1 -- 172.21.15.81:0/1246649841 shutdown_connections 2024-05-08T23:31:04.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.209+0000 7f1bd73f9700 1 -- 172.21.15.81:0/1246649841 wait complete. 2024-05-08T23:31:04.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd73f9700 1 Processor -- start 2024-05-08T23:31:04.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd73f9700 1 -- start start 2024-05-08T23:31:04.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd73f9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bc804ff30 0x7f1bc80cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd73f9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bc80507d0 0x7f1bc817cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd73f9700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1bc80e0160 0x7f1bc80cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd73f9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc80e0bc0 con 0x7f1bc804ff30 2024-05-08T23:31:04.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd73f9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc817c810 con 0x7f1bc80507d0 2024-05-08T23:31:04.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd73f9700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc80cb060 con 0x7f1bc80e0160 2024-05-08T23:31:04.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd63f7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bc804ff30 0x7f1bc80cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd63f7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bc804ff30 0x7f1bc80cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38992/0 (socket says 172.21.15.81:38992) 2024-05-08T23:31:04.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd63f7700 1 -- 172.21.15.81:0/514451206 learned_addr learned my addr 172.21.15.81:0/514451206 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:04.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd5bf6700 1 --2- 172.21.15.81:0/514451206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bc80507d0 0x7f1bc817cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd6bf8700 1 --2- 172.21.15.81:0/514451206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1bc80e0160 0x7f1bc80cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd63f7700 1 -- 172.21.15.81:0/514451206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1bc80e0160 msgr2=0x7f1bc80cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd63f7700 1 --2- 172.21.15.81:0/514451206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1bc80e0160 0x7f1bc80cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd63f7700 1 -- 172.21.15.81:0/514451206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bc80507d0 msgr2=0x7f1bc817cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd63f7700 1 --2- 172.21.15.81:0/514451206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bc80507d0 0x7f1bc817cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd63f7700 1 -- 172.21.15.81:0/514451206 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc0094b0 con 0x7f1bc804ff30 2024-05-08T23:31:04.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd63f7700 1 --2- 172.21.15.81:0/514451206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bc804ff30 0x7f1bc80cab20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0090b0 tx=0x7f1bcc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:04.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bc2ffd700 1 -- 172.21.15.81:0/514451206 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc01e070 con 0x7f1bc804ff30 2024-05-08T23:31:04.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bc2ffd700 1 -- 172.21.15.81:0/514451206 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc004580 con 0x7f1bc804ff30 2024-05-08T23:31:04.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bd73f9700 1 -- 172.21.15.81:0/514451206 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc817c450 con 0x7f1bc804ff30 2024-05-08T23:31:04.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.213+0000 7f1bc2ffd700 1 -- 172.21.15.81:0/514451206 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc010a50 con 0x7f1bc804ff30 2024-05-08T23:31:04.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.217+0000 7f1bd73f9700 1 -- 172.21.15.81:0/514451206 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc80ce0e0 con 0x7f1bc804ff30 2024-05-08T23:31:04.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.217+0000 7f1bd73f9700 1 -- 172.21.15.81:0/514451206 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc8005440 con 0x7f1bc804ff30 2024-05-08T23:31:04.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.217+0000 7f1bc2ffd700 1 -- 172.21.15.81:0/514451206 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1bcc010bb0 con 0x7f1bc804ff30 2024-05-08T23:31:04.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.217+0000 7f1bc2ffd700 1 --2- 172.21.15.81:0/514451206 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1bb4066730 0x7f1bb4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.217+0000 7f1bd5bf6700 1 --2- 172.21.15.81:0/514451206 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1bb4066730 0x7f1bb4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.217+0000 7f1bc2ffd700 1 -- 172.21.15.81:0/514451206 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bcc085370 con 0x7f1bc804ff30 2024-05-08T23:31:04.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.217+0000 7f1bd5bf6700 1 --2- 172.21.15.81:0/514451206 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1bb4066730 0x7f1bb4068bf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1bc803abc0 tx=0x7f1bc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:04.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.225+0000 7f1bc2ffd700 1 -- 172.21.15.81:0/514451206 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1bcc052890 con 0x7f1bc804ff30 2024-05-08T23:31:04.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.401+0000 7f1bd73f9700 1 -- 172.21.15.81:0/514451206 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1bc8007520 con 0x7f1bc804ff30 2024-05-08T23:31:04.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.401+0000 7f1bc2ffd700 1 -- 172.21.15.81:0/514451206 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1bcc010e60 con 0x7f1bc804ff30 2024-05-08T23:31:04.406 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:04.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.405+0000 7f1bd73f9700 1 -- 172.21.15.81:0/514451206 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1bb4066730 msgr2=0x7f1bb4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.405+0000 7f1bd73f9700 1 --2- 172.21.15.81:0/514451206 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1bb4066730 0x7f1bb4068bf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1bc803abc0 tx=0x7f1bc4009040 comp rx=0 tx=0).stop 2024-05-08T23:31:04.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.405+0000 7f1bd73f9700 1 -- 172.21.15.81:0/514451206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bc804ff30 msgr2=0x7f1bc80cab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.405+0000 7f1bd73f9700 1 --2- 172.21.15.81:0/514451206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bc804ff30 0x7f1bc80cab20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0090b0 tx=0x7f1bcc004770 comp rx=0 tx=0).stop 2024-05-08T23:31:04.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.405+0000 7f1bd73f9700 1 -- 172.21.15.81:0/514451206 shutdown_connections 2024-05-08T23:31:04.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.405+0000 7f1bd73f9700 1 --2- 172.21.15.81:0/514451206 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1bb4066730 0x7f1bb4068bf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.405+0000 7f1bd73f9700 1 --2- 172.21.15.81:0/514451206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1bc80e0160 0x7f1bc80cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.405+0000 7f1bd73f9700 1 --2- 172.21.15.81:0/514451206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bc80507d0 0x7f1bc817cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.405+0000 7f1bd73f9700 1 --2- 172.21.15.81:0/514451206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bc804ff30 0x7f1bc80cab20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.405+0000 7f1bd73f9700 1 -- 172.21.15.81:0/514451206 >> 172.21.15.81:0/514451206 conn(0x7f1bc8054b70 msgr2=0x7f1bc8049c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:04.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.405+0000 7f1bd73f9700 1 -- 172.21.15.81:0/514451206 shutdown_connections 2024-05-08T23:31:04.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.405+0000 7f1bd73f9700 1 -- 172.21.15.81:0/514451206 wait complete. 2024-05-08T23:31:04.421 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:04.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.533+0000 7fbef759e700 1 Processor -- start 2024-05-08T23:31:04.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.533+0000 7fbef759e700 1 -- start start 2024-05-08T23:31:04.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.533+0000 7fbef759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80d1da0 0x7fbef80cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.533+0000 7fbef759e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef80d3d40 con 0x7fbef80cfe80 2024-05-08T23:31:04.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.533+0000 7fbef759e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef80d0d00 con 0x7fbef8160920 2024-05-08T23:31:04.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.533+0000 7fbef759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef80d0310 con 0x7fbef80d1da0 2024-05-08T23:31:04.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef6d9d700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fbef80cfe80 0x7fbef80b5e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44588/0 (socket says 172.21.15.81:44588) 2024-05-08T23:31:04.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef6d9d700 1 -- 172.21.15.81:0/3472824316 learned_addr learned my addr 172.21.15.81:0/3472824316 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:04.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef5d9b700 1 --2- 172.21.15.81:0/3472824316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80d1da0 0x7fbef80cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef559a700 1 -- 172.21.15.81:0/3472824316 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113020241 0 0) 0x7fbef80d3d40 con 0x7fbef80cfe80 2024-05-08T23:31:04.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef559a700 1 -- 172.21.15.81:0/3472824316 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee00035d0 con 0x7fbef80cfe80 2024-05-08T23:31:04.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef559a700 1 -- 172.21.15.81:0/3472824316 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225722147 0 0) 0x7fbef80d0d00 con 0x7fbef8160920 2024-05-08T23:31:04.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef559a700 1 -- 172.21.15.81:0/3472824316 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbef80d3d40 con 0x7fbef8160920 2024-05-08T23:31:04.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef5d9b700 1 -- 172.21.15.81:0/3472824316 >> v1:172.21.15.117:6789/0 conn(0x7fbef8160920 legacy=0x7fbef80ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef5d9b700 1 -- 172.21.15.81:0/3472824316 >> v1:172.21.15.81:6789/0 conn(0x7fbef80cfe80 legacy=0x7fbef80b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef5d9b700 1 -- 172.21.15.81:0/3472824316 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef80d0fe0 con 0x7fbef80d1da0 2024-05-08T23:31:04.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef5d9b700 1 --2- 172.21.15.81:0/3472824316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80d1da0 0x7fbef80cf940 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009880 tx=0x7fbee8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdd77c6fa4fb1ee4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:04.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef559a700 1 -- 172.21.15.81:0/3472824316 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8011830 con 0x7fbef80d1da0 2024-05-08T23:31:04.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef559a700 1 -- 172.21.15.81:0/3472824316 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee8011990 con 0x7fbef80d1da0 2024-05-08T23:31:04.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef559a700 1 -- 172.21.15.81:0/3472824316 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8011d20 con 0x7fbef80d1da0 2024-05-08T23:31:04.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef759e700 1 -- 172.21.15.81:0/3472824316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80d1da0 msgr2=0x7fbef80cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef759e700 1 --2- 172.21.15.81:0/3472824316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80d1da0 0x7fbef80cf940 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009880 tx=0x7fbee8009b90 comp rx=0 tx=0).stop 2024-05-08T23:31:04.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef759e700 1 -- 172.21.15.81:0/3472824316 shutdown_connections 2024-05-08T23:31:04.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef759e700 1 --2- 172.21.15.81:0/3472824316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef80d1da0 0x7fbef80cf940 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef759e700 1 -- 172.21.15.81:0/3472824316 >> 172.21.15.81:0/3472824316 conn(0x7fbef809c340 msgr2=0x7fbef80d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:04.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef759e700 1 -- 172.21.15.81:0/3472824316 shutdown_connections 2024-05-08T23:31:04.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.537+0000 7fbef759e700 1 -- 172.21.15.81:0/3472824316 wait complete. 2024-05-08T23:31:04.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef759e700 1 Processor -- start 2024-05-08T23:31:04.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef759e700 1 -- start start 2024-05-08T23:31:04.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef80cfe80 0x7fbef80c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbef80d1da0 0x7fbef80c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef8160920 0x7fbef80c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef759e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef80c8e60 con 0x7fbef80cfe80 2024-05-08T23:31:04.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef759e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef80c9140 con 0x7fbef80d1da0 2024-05-08T23:31:04.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef80c9420 con 0x7fbef8160920 2024-05-08T23:31:04.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef80cfe80 0x7fbef80c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef80cfe80 0x7fbef80c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38996/0 (socket says 172.21.15.81:38996) 2024-05-08T23:31:04.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef659c700 1 -- 172.21.15.81:0/3616993728 learned_addr learned my addr 172.21.15.81:0/3616993728 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:04.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef6d9d700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef8160920 0x7fbef80c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef5d9b700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbef80d1da0 0x7fbef80c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef659c700 1 -- 172.21.15.81:0/3616993728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef8160920 msgr2=0x7fbef80c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef659c700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef8160920 0x7fbef80c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef659c700 1 -- 172.21.15.81:0/3616993728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbef80d1da0 msgr2=0x7fbef80c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef659c700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbef80d1da0 0x7fbef80c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef659c700 1 -- 172.21.15.81:0/3616993728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee80094b0 con 0x7fbef80cfe80 2024-05-08T23:31:04.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef5d9b700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbef80d1da0 0x7fbef80c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:04.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef6d9d700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef8160920 0x7fbef80c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:04.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef659c700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef80cfe80 0x7fbef80c8870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00c490 tx=0x7fbeec00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:04.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbee6ffd700 1 -- 172.21.15.81:0/3616993728 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0052d0 con 0x7fbef80cfe80 2024-05-08T23:31:04.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef759e700 1 -- 172.21.15.81:0/3616993728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef80c9700 con 0x7fbef80cfe80 2024-05-08T23:31:04.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbee6ffd700 1 -- 172.21.15.81:0/3616993728 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec005450 con 0x7fbef80cfe80 2024-05-08T23:31:04.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.541+0000 7fbef759e700 1 -- 172.21.15.81:0/3616993728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef81f3440 con 0x7fbef80cfe80 2024-05-08T23:31:04.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.545+0000 7fbee6ffd700 1 -- 172.21.15.81:0/3616993728 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0038a0 con 0x7fbef80cfe80 2024-05-08T23:31:04.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.545+0000 7fbef759e700 1 -- 172.21.15.81:0/3616993728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef80684b0 con 0x7fbef80cfe80 2024-05-08T23:31:04.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.545+0000 7fbee6ffd700 1 -- 172.21.15.81:0/3616993728 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbeec003a00 con 0x7fbef80cfe80 2024-05-08T23:31:04.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.545+0000 7fbee6ffd700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbed8066730 0x7fbed8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.545+0000 7fbee6ffd700 1 -- 172.21.15.81:0/3616993728 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbeec083490 con 0x7fbef80cfe80 2024-05-08T23:31:04.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.545+0000 7fbef5d9b700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbed8066730 0x7fbed8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.545+0000 7fbef5d9b700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbed8066730 0x7fbed8068bf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbee80097d0 tx=0x7fbee8010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:04.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.557+0000 7fbee6ffd700 1 -- 172.21.15.81:0/3616993728 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbeec0508c0 con 0x7fbef80cfe80 2024-05-08T23:31:04.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.725+0000 7fbef759e700 1 -- 172.21.15.81:0/3616993728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbef80b6c40 con 0x7fbef80cfe80 2024-05-08T23:31:04.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.729+0000 7fbee6ffd700 1 -- 172.21.15.81:0/3616993728 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbeec003cb0 con 0x7fbef80cfe80 2024-05-08T23:31:04.732 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:04.734 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.729+0000 7fbef759e700 1 -- 172.21.15.81:0/3616993728 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbed8066730 msgr2=0x7fbed8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.734 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.729+0000 7fbef759e700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbed8066730 0x7fbed8068bf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbee80097d0 tx=0x7fbee8010000 comp rx=0 tx=0).stop 2024-05-08T23:31:04.735 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.729+0000 7fbef759e700 1 -- 172.21.15.81:0/3616993728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef80cfe80 msgr2=0x7fbef80c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.735 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.729+0000 7fbef759e700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef80cfe80 0x7fbef80c8870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00c490 tx=0x7fbeec00c850 comp rx=0 tx=0).stop 2024-05-08T23:31:04.735 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.729+0000 7fbef759e700 1 -- 172.21.15.81:0/3616993728 shutdown_connections 2024-05-08T23:31:04.735 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.729+0000 7fbef759e700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbed8066730 0x7fbed8068bf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.735 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.729+0000 7fbef759e700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbef8160920 0x7fbef80c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.735 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.729+0000 7fbef759e700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbef80d1da0 0x7fbef80c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.735 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.729+0000 7fbef759e700 1 --2- 172.21.15.81:0/3616993728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef80cfe80 0x7fbef80c8870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.735 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.729+0000 7fbef759e700 1 -- 172.21.15.81:0/3616993728 >> 172.21.15.81:0/3616993728 conn(0x7fbef809c340 msgr2=0x7fbef80cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:04.735 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.729+0000 7fbef759e700 1 -- 172.21.15.81:0/3616993728 shutdown_connections 2024-05-08T23:31:04.735 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.729+0000 7fbef759e700 1 -- 172.21.15.81:0/3616993728 wait complete. 2024-05-08T23:31:04.747 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:04.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f0608940700 1 Processor -- start 2024-05-08T23:31:04.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f0608940700 1 -- start start 2024-05-08T23:31:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f0608940700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f060415d410 0x7f06040d3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f0608940700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f06040d1cf0 0x7f06040b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f0608940700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06040d3e80 con 0x7f06040ce620 2024-05-08T23:31:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f0608940700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06040cebe0 con 0x7f06040d1cf0 2024-05-08T23:31:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f0608940700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06040ceec0 con 0x7f060415d410 2024-05-08T23:31:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f060359e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f06040d1cf0 0x7f06040b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f060359e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f06040d1cf0 0x7f06040b2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39012/0 (socket says 172.21.15.81:39012) 2024-05-08T23:31:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f060359e700 1 -- 172.21.15.81:0/123924780 learned_addr learned my addr 172.21.15.81:0/123924780 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f0602d9d700 1 --2- 172.21.15.81:0/123924780 >> v2:172.21.15.117:3300/0 conn(0x7f060415d410 0x7f06040d3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f060359e700 1 -- 172.21.15.81:0/123924780 >> v2:172.21.15.117:3300/0 conn(0x7f060415d410 msgr2=0x7f06040d3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f060359e700 1 --2- 172.21.15.81:0/123924780 >> v2:172.21.15.117:3300/0 conn(0x7f060415d410 0x7f06040d3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f060359e700 1 -- 172.21.15.81:0/123924780 >> v1:172.21.15.117:6789/0 conn(0x7f06040ce620 legacy=0x7f06040d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f060359e700 1 -- 172.21.15.81:0/123924780 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06040b24f0 con 0x7f06040d1cf0 2024-05-08T23:31:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f060359e700 1 --2- 172.21.15.81:0/123924780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06040d1cf0 0x7f06040b2fd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f060408bf30 tx=0x7f05fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfed95daae809b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f0601d9b700 1 -- 172.21.15.81:0/123924780 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00e070 con 0x7f06040d1cf0 2024-05-08T23:31:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f0601d9b700 1 -- 172.21.15.81:0/123924780 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc00b9c0 con 0x7f06040d1cf0 2024-05-08T23:31:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.865+0000 7f0601d9b700 1 -- 172.21.15.81:0/123924780 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00bd70 con 0x7f06040d1cf0 2024-05-08T23:31:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 -- 172.21.15.81:0/123924780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06040d1cf0 msgr2=0x7f06040b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 --2- 172.21.15.81:0/123924780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06040d1cf0 0x7f06040b2fd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f060408bf30 tx=0x7f05fc00b330 comp rx=0 tx=0).stop 2024-05-08T23:31:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 -- 172.21.15.81:0/123924780 shutdown_connections 2024-05-08T23:31:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 --2- 172.21.15.81:0/123924780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06040d1cf0 0x7f06040b2fd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 --2- 172.21.15.81:0/123924780 >> v2:172.21.15.117:3300/0 conn(0x7f060415d410 0x7f06040d3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 -- 172.21.15.81:0/123924780 >> 172.21.15.81:0/123924780 conn(0x7f060409c340 msgr2=0x7f06040d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:04.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 -- 172.21.15.81:0/123924780 shutdown_connections 2024-05-08T23:31:04.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 -- 172.21.15.81:0/123924780 wait complete. 2024-05-08T23:31:04.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 Processor -- start 2024-05-08T23:31:04.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 -- start start 2024-05-08T23:31:04.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06040ce620 0x7f0604159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06040d1cf0 0x7f06040c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f060415d410 0x7f06040c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060415ecb0 con 0x7f06040d1cf0 2024-05-08T23:31:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06040c73b0 con 0x7f060415d410 2024-05-08T23:31:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0608940700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060415e660 con 0x7f06040ce620 2024-05-08T23:31:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f060259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06040d1cf0 0x7f06040c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f060259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06040d1cf0 0x7f06040c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39016/0 (socket says 172.21.15.81:39016) 2024-05-08T23:31:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f060259c700 1 -- 172.21.15.81:0/1163553888 learned_addr learned my addr 172.21.15.81:0/1163553888 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f0602d9d700 1 --2- 172.21.15.81:0/1163553888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06040ce620 0x7f0604159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f060359e700 1 --2- 172.21.15.81:0/1163553888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f060415d410 0x7f06040c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f060259c700 1 -- 172.21.15.81:0/1163553888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06040ce620 msgr2=0x7f0604159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f060259c700 1 --2- 172.21.15.81:0/1163553888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06040ce620 0x7f0604159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f060259c700 1 -- 172.21.15.81:0/1163553888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f060415d410 msgr2=0x7f06040c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:04.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f060259c700 1 --2- 172.21.15.81:0/1163553888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f060415d410 0x7f06040c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:04.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.869+0000 7f060259c700 1 -- 172.21.15.81:0/1163553888 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc009b30 con 0x7f06040d1cf0 2024-05-08T23:31:04.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.873+0000 7f060259c700 1 --2- 172.21.15.81:0/1163553888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06040d1cf0 0x7f06040c69f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f05f400c500 tx=0x7f05f400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:04.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.873+0000 7f05eb7fe700 1 -- 172.21.15.81:0/1163553888 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f40052d0 con 0x7f06040d1cf0 2024-05-08T23:31:04.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.873+0000 7f05eb7fe700 1 -- 172.21.15.81:0/1163553888 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f4005430 con 0x7f06040d1cf0 2024-05-08T23:31:04.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.873+0000 7f0608940700 1 -- 172.21.15.81:0/1163553888 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060415e940 con 0x7f06040d1cf0 2024-05-08T23:31:04.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.873+0000 7f05eb7fe700 1 -- 172.21.15.81:0/1163553888 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f40055a0 con 0x7f06040d1cf0 2024-05-08T23:31:04.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.873+0000 7f0608940700 1 -- 172.21.15.81:0/1163553888 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06040a0c00 con 0x7f06040d1cf0 2024-05-08T23:31:04.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.873+0000 7f0608940700 1 -- 172.21.15.81:0/1163553888 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06040684b0 con 0x7f06040d1cf0 2024-05-08T23:31:04.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.873+0000 7f05eb7fe700 1 -- 172.21.15.81:0/1163553888 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f05f40038e0 con 0x7f06040d1cf0 2024-05-08T23:31:04.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.877+0000 7f05eb7fe700 1 --2- 172.21.15.81:0/1163553888 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f05f0066730 0x7f05f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:04.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.877+0000 7f05eb7fe700 1 -- 172.21.15.81:0/1163553888 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05f4083500 con 0x7f06040d1cf0 2024-05-08T23:31:04.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.877+0000 7f0602d9d700 1 --2- 172.21.15.81:0/1163553888 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f05f0066730 0x7f05f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:04.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.877+0000 7f0602d9d700 1 --2- 172.21.15.81:0/1163553888 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f05f0066730 0x7f05f0068bf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f05f8007a60 tx=0x7f05f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:04.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:04.881+0000 7f05eb7fe700 1 -- 172.21.15.81:0/1163553888 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f05f4050870 con 0x7f06040d1cf0 2024-05-08T23:31:05.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.049+0000 7f0608940700 1 -- 172.21.15.81:0/1163553888 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f060415edf0 con 0x7f06040d1cf0 2024-05-08T23:31:05.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.049+0000 7f05eb7fe700 1 -- 172.21.15.81:0/1163553888 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f05f4053e90 con 0x7f06040d1cf0 2024-05-08T23:31:05.055 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:05.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.053+0000 7f0608940700 1 -- 172.21.15.81:0/1163553888 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f05f0066730 msgr2=0x7f05f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.053+0000 7f0608940700 1 --2- 172.21.15.81:0/1163553888 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f05f0066730 0x7f05f0068bf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f05f8007a60 tx=0x7f05f8008040 comp rx=0 tx=0).stop 2024-05-08T23:31:05.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.053+0000 7f0608940700 1 -- 172.21.15.81:0/1163553888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06040d1cf0 msgr2=0x7f06040c69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.053+0000 7f0608940700 1 --2- 172.21.15.81:0/1163553888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06040d1cf0 0x7f06040c69f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f05f400c500 tx=0x7f05f400c810 comp rx=0 tx=0).stop 2024-05-08T23:31:05.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.053+0000 7f0608940700 1 -- 172.21.15.81:0/1163553888 shutdown_connections 2024-05-08T23:31:05.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.053+0000 7f0608940700 1 --2- 172.21.15.81:0/1163553888 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f05f0066730 0x7f05f0068bf0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.053+0000 7f0608940700 1 --2- 172.21.15.81:0/1163553888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f060415d410 0x7f06040c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.053+0000 7f0608940700 1 --2- 172.21.15.81:0/1163553888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06040d1cf0 0x7f06040c69f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.053+0000 7f0608940700 1 --2- 172.21.15.81:0/1163553888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06040ce620 0x7f0604159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.053+0000 7f0608940700 1 -- 172.21.15.81:0/1163553888 >> 172.21.15.81:0/1163553888 conn(0x7f060409c340 msgr2=0x7f06040d0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:05.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.053+0000 7f0608940700 1 -- 172.21.15.81:0/1163553888 shutdown_connections 2024-05-08T23:31:05.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.053+0000 7f0608940700 1 -- 172.21.15.81:0/1163553888 wait complete. 2024-05-08T23:31:05.069 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:05.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.177+0000 7f32994f7700 1 Processor -- start 2024-05-08T23:31:05.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.177+0000 7f32994f7700 1 -- start start 2024-05-08T23:31:05.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f32994f7700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f329415afa0 0x7f329415b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f32994f7700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32940ce990 0x7f32940b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f32994f7700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f329415eda0 con 0x7f329415b8c0 2024-05-08T23:31:05.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f32994f7700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329415ea70 con 0x7f329415afa0 2024-05-08T23:31:05.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f32994f7700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329415bd40 con 0x7f32940ce990 2024-05-08T23:31:05.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f3293fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f329415afa0 0x7f329415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f3298cf6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32940ce990 0x7f32940b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f32937fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f329415b8c0 0x7f329415f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:55340/0 (socket says 172.21.15.81:55340) 2024-05-08T23:31:05.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f32937fe700 1 -- 172.21.15.81:0/1198120617 learned_addr learned my addr 172.21.15.81:0/1198120617 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:05.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f3292ffd700 1 -- 172.21.15.81:0/1198120617 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393190207 0 0) 0x7f329415eda0 con 0x7f329415b8c0 2024-05-08T23:31:05.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f3292ffd700 1 -- 172.21.15.81:0/1198120617 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32740035d0 con 0x7f329415b8c0 2024-05-08T23:31:05.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f3293fff700 1 -- 172.21.15.81:0/1198120617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32940ce990 msgr2=0x7f32940b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:05.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f3293fff700 1 --2- 172.21.15.81:0/1198120617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32940ce990 0x7f32940b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f3293fff700 1 -- 172.21.15.81:0/1198120617 >> v1:172.21.15.117:6789/0 conn(0x7f329415b8c0 legacy=0x7f329415f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f3293fff700 1 -- 172.21.15.81:0/1198120617 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329415c020 con 0x7f329415afa0 2024-05-08T23:31:05.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f3298cf6700 1 --2- 172.21.15.81:0/1198120617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32940ce990 0x7f32940b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:05.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f3293fff700 1 --2- 172.21.15.81:0/1198120617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f329415afa0 0x7f329415b380 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3284009850 tx=0x7f3284009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78739698777b5561 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:05.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f3292ffd700 1 -- 172.21.15.81:0/1198120617 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284011830 con 0x7f329415afa0 2024-05-08T23:31:05.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f3292ffd700 1 -- 172.21.15.81:0/1198120617 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3284011990 con 0x7f329415afa0 2024-05-08T23:31:05.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f3292ffd700 1 -- 172.21.15.81:0/1198120617 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284011d40 con 0x7f329415afa0 2024-05-08T23:31:05.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f32994f7700 1 -- 172.21.15.81:0/1198120617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f329415afa0 msgr2=0x7f329415b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f32994f7700 1 --2- 172.21.15.81:0/1198120617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f329415afa0 0x7f329415b380 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3284009850 tx=0x7f3284009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:05.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f32994f7700 1 -- 172.21.15.81:0/1198120617 shutdown_connections 2024-05-08T23:31:05.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f32994f7700 1 --2- 172.21.15.81:0/1198120617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32940ce990 0x7f32940b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f32994f7700 1 --2- 172.21.15.81:0/1198120617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f329415afa0 0x7f329415b380 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f32994f7700 1 -- 172.21.15.81:0/1198120617 >> 172.21.15.81:0/1198120617 conn(0x7f329406c330 msgr2=0x7f329406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:05.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f32994f7700 1 -- 172.21.15.81:0/1198120617 shutdown_connections 2024-05-08T23:31:05.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.181+0000 7f32994f7700 1 -- 172.21.15.81:0/1198120617 wait complete. 2024-05-08T23:31:05.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32994f7700 1 Processor -- start 2024-05-08T23:31:05.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32994f7700 1 -- start start 2024-05-08T23:31:05.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32994f7700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32940ce990 0x7f32940b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32994f7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f329415afa0 0x7f329414fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32994f7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f329415b8c0 0x7f32940addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32994f7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329415dcc0 con 0x7f329415afa0 2024-05-08T23:31:05.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32994f7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32940ad4c0 con 0x7f329415b8c0 2024-05-08T23:31:05.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32994f7700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32940ad7a0 con 0x7f32940ce990 2024-05-08T23:31:05.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32937fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f329415afa0 0x7f329414fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32937fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f329415afa0 0x7f329414fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39024/0 (socket says 172.21.15.81:39024) 2024-05-08T23:31:05.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32937fe700 1 -- 172.21.15.81:0/554743332 learned_addr learned my addr 172.21.15.81:0/554743332 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:05.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f3298cf6700 1 --2- 172.21.15.81:0/554743332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f329415b8c0 0x7f32940addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f3293fff700 1 --2- 172.21.15.81:0/554743332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32940ce990 0x7f32940b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32937fe700 1 -- 172.21.15.81:0/554743332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32940ce990 msgr2=0x7f32940b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32937fe700 1 --2- 172.21.15.81:0/554743332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32940ce990 0x7f32940b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32937fe700 1 -- 172.21.15.81:0/554743332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f329415b8c0 msgr2=0x7f32940addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32937fe700 1 --2- 172.21.15.81:0/554743332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f329415b8c0 0x7f32940addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32937fe700 1 -- 172.21.15.81:0/554743332 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c009a70 con 0x7f329415afa0 2024-05-08T23:31:05.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f3298cf6700 1 --2- 172.21.15.81:0/554743332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f329415b8c0 0x7f32940addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:05.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.185+0000 7f32937fe700 1 --2- 172.21.15.81:0/554743332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f329415afa0 0x7f329414fc10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f327c00c4d0 tx=0x7f327c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:05.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.189+0000 7f3290ff9700 1 -- 172.21.15.81:0/554743332 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c005d90 con 0x7f329415afa0 2024-05-08T23:31:05.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.189+0000 7f3290ff9700 1 -- 172.21.15.81:0/554743332 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c00ce70 con 0x7f329415afa0 2024-05-08T23:31:05.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.189+0000 7f32994f7700 1 -- 172.21.15.81:0/554743332 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32840094b0 con 0x7f329415afa0 2024-05-08T23:31:05.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.189+0000 7f3290ff9700 1 -- 172.21.15.81:0/554743332 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c0072d0 con 0x7f329415afa0 2024-05-08T23:31:05.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.189+0000 7f32994f7700 1 -- 172.21.15.81:0/554743332 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32941539f0 con 0x7f329415afa0 2024-05-08T23:31:05.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.189+0000 7f32994f7700 1 -- 172.21.15.81:0/554743332 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32940684b0 con 0x7f329415afa0 2024-05-08T23:31:05.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.189+0000 7f3290ff9700 1 -- 172.21.15.81:0/554743332 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f327c007430 con 0x7f329415afa0 2024-05-08T23:31:05.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.189+0000 7f3290ff9700 1 --2- 172.21.15.81:0/554743332 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3278066760 0x7f3278068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.189+0000 7f3290ff9700 1 -- 172.21.15.81:0/554743332 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f327c083320 con 0x7f329415afa0 2024-05-08T23:31:05.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.189+0000 7f3293fff700 1 --2- 172.21.15.81:0/554743332 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3278066760 0x7f3278068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.193+0000 7f3293fff700 1 --2- 172.21.15.81:0/554743332 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3278066760 0x7f3278068c20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f32840090b0 tx=0x7f3284008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:05.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.197+0000 7f3290ff9700 1 -- 172.21.15.81:0/554743332 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f327c050740 con 0x7f329415afa0 2024-05-08T23:31:05.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.365+0000 7f32994f7700 1 -- 172.21.15.81:0/554743332 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f329406e220 con 0x7f329415afa0 2024-05-08T23:31:05.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.369+0000 7f3290ff9700 1 -- 172.21.15.81:0/554743332 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f327c053d60 con 0x7f329415afa0 2024-05-08T23:31:05.374 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:05.374 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:05.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.369+0000 7f32994f7700 1 -- 172.21.15.81:0/554743332 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3278066760 msgr2=0x7f3278068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.369+0000 7f32994f7700 1 --2- 172.21.15.81:0/554743332 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3278066760 0x7f3278068c20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f32840090b0 tx=0x7f3284008030 comp rx=0 tx=0).stop 2024-05-08T23:31:05.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.373+0000 7f32994f7700 1 -- 172.21.15.81:0/554743332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f329415afa0 msgr2=0x7f329414fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.373+0000 7f32994f7700 1 --2- 172.21.15.81:0/554743332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f329415afa0 0x7f329414fc10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f327c00c4d0 tx=0x7f327c00c7e0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.373+0000 7f32994f7700 1 -- 172.21.15.81:0/554743332 shutdown_connections 2024-05-08T23:31:05.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.373+0000 7f32994f7700 1 --2- 172.21.15.81:0/554743332 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3278066760 0x7f3278068c20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.373+0000 7f32994f7700 1 --2- 172.21.15.81:0/554743332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f329415b8c0 0x7f32940addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.373+0000 7f32994f7700 1 --2- 172.21.15.81:0/554743332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f329415afa0 0x7f329414fc10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.373+0000 7f32994f7700 1 --2- 172.21.15.81:0/554743332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32940ce990 0x7f32940b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.373+0000 7f32994f7700 1 -- 172.21.15.81:0/554743332 >> 172.21.15.81:0/554743332 conn(0x7f329406c330 msgr2=0x7f32940cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:05.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.373+0000 7f32994f7700 1 -- 172.21.15.81:0/554743332 shutdown_connections 2024-05-08T23:31:05.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.373+0000 7f32994f7700 1 -- 172.21.15.81:0/554743332 wait complete. 2024-05-08T23:31:05.389 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:05.389 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:05.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.505+0000 7fa26acef700 1 Processor -- start 2024-05-08T23:31:05.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.505+0000 7fa26acef700 1 -- start start 2024-05-08T23:31:05.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.505+0000 7fa26acef700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa26406d8c0 0x7fa26415e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.505+0000 7fa26acef700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa2640b20f0 0x7fa264071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.505+0000 7fa26acef700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2640721c0 con 0x7fa2640b3620 2024-05-08T23:31:05.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.505+0000 7fa26acef700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26415e650 con 0x7fa26406d8c0 2024-05-08T23:31:05.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.505+0000 7fa26acef700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2640ad700 con 0x7fa2640b20f0 2024-05-08T23:31:05.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.505+0000 7fa269ced700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa26406d8c0 0x7fa26415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.505+0000 7fa269ced700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa26406d8c0 0x7fa26415e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39028/0 (socket says 172.21.15.81:39028) 2024-05-08T23:31:05.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.505+0000 7fa269ced700 1 -- 172.21.15.81:0/574629917 learned_addr learned my addr 172.21.15.81:0/574629917 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:05.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.505+0000 7fa2694ec700 1 --2- 172.21.15.81:0/574629917 >> v2:172.21.15.117:3300/0 conn(0x7fa2640b20f0 0x7fa264071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.505+0000 7fa269ced700 1 -- 172.21.15.81:0/574629917 >> v2:172.21.15.117:3300/0 conn(0x7fa2640b20f0 msgr2=0x7fa264071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:05.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.505+0000 7fa269ced700 1 --2- 172.21.15.81:0/574629917 >> v2:172.21.15.117:3300/0 conn(0x7fa2640b20f0 0x7fa264071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.505+0000 7fa269ced700 1 -- 172.21.15.81:0/574629917 >> v1:172.21.15.117:6789/0 conn(0x7fa2640b3620 legacy=0x7fa264156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.505+0000 7fa269ced700 1 -- 172.21.15.81:0/574629917 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2640b2580 con 0x7fa26406d8c0 2024-05-08T23:31:05.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.509+0000 7fa269ced700 1 --2- 172.21.15.81:0/574629917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa26406d8c0 0x7fa26415e110 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa254009910 tx=0x7fa254009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fc5bcbb3ef57ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:05.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.509+0000 7fa268ceb700 1 -- 172.21.15.81:0/574629917 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25400ba20 con 0x7fa26406d8c0 2024-05-08T23:31:05.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.509+0000 7fa268ceb700 1 -- 172.21.15.81:0/574629917 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25400bb80 con 0x7fa26406d8c0 2024-05-08T23:31:05.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.509+0000 7fa268ceb700 1 -- 172.21.15.81:0/574629917 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254009e60 con 0x7fa26406d8c0 2024-05-08T23:31:05.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.509+0000 7fa26acef700 1 -- 172.21.15.81:0/574629917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa26406d8c0 msgr2=0x7fa26415e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.509+0000 7fa26acef700 1 --2- 172.21.15.81:0/574629917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa26406d8c0 0x7fa26415e110 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa254009910 tx=0x7fa254009c20 comp rx=0 tx=0).stop 2024-05-08T23:31:05.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.509+0000 7fa26acef700 1 -- 172.21.15.81:0/574629917 shutdown_connections 2024-05-08T23:31:05.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.509+0000 7fa26acef700 1 --2- 172.21.15.81:0/574629917 >> v2:172.21.15.117:3300/0 conn(0x7fa2640b20f0 0x7fa264071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.509+0000 7fa26acef700 1 --2- 172.21.15.81:0/574629917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa26406d8c0 0x7fa26415e110 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.509+0000 7fa26acef700 1 -- 172.21.15.81:0/574629917 >> 172.21.15.81:0/574629917 conn(0x7fa2640b0de0 msgr2=0x7fa2640b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:05.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.509+0000 7fa26acef700 1 -- 172.21.15.81:0/574629917 shutdown_connections 2024-05-08T23:31:05.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.509+0000 7fa26acef700 1 -- 172.21.15.81:0/574629917 wait complete. 2024-05-08T23:31:05.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26acef700 1 Processor -- start 2024-05-08T23:31:05.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26acef700 1 -- start start 2024-05-08T23:31:05.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26acef700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa26406d8c0 0x7fa26414e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26acef700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2640b3620 0x7fa2640c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26acef700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2640c9230 0x7fa26415f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26acef700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26409a720 con 0x7fa2640c9230 2024-05-08T23:31:05.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26acef700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26414eb20 con 0x7fa26406d8c0 2024-05-08T23:31:05.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26acef700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa264099b10 con 0x7fa2640b3620 2024-05-08T23:31:05.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26a4ee700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2640c9230 0x7fa26415f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26a4ee700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2640c9230 0x7fa26415f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39036/0 (socket says 172.21.15.81:39036) 2024-05-08T23:31:05.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26a4ee700 1 -- 172.21.15.81:0/1330514218 learned_addr learned my addr 172.21.15.81:0/1330514218 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:05.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa269ced700 1 --2- 172.21.15.81:0/1330514218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa26406d8c0 0x7fa26414e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa2694ec700 1 --2- 172.21.15.81:0/1330514218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2640b3620 0x7fa2640c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26a4ee700 1 -- 172.21.15.81:0/1330514218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2640b3620 msgr2=0x7fa2640c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26a4ee700 1 --2- 172.21.15.81:0/1330514218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2640b3620 0x7fa2640c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26a4ee700 1 -- 172.21.15.81:0/1330514218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa26406d8c0 msgr2=0x7fa26414e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26a4ee700 1 --2- 172.21.15.81:0/1330514218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa26406d8c0 0x7fa26414e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26a4ee700 1 -- 172.21.15.81:0/1330514218 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa254009570 con 0x7fa2640c9230 2024-05-08T23:31:05.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.513+0000 7fa26a4ee700 1 --2- 172.21.15.81:0/1330514218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2640c9230 0x7fa26415f880 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa26000c3c0 tx=0x7fa26000c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:05.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.517+0000 7fa25a7fc700 1 -- 172.21.15.81:0/1330514218 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2600052d0 con 0x7fa2640c9230 2024-05-08T23:31:05.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.517+0000 7fa25a7fc700 1 -- 172.21.15.81:0/1330514218 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa260005430 con 0x7fa2640c9230 2024-05-08T23:31:05.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.517+0000 7fa26acef700 1 -- 172.21.15.81:0/1330514218 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa264099df0 con 0x7fa2640c9230 2024-05-08T23:31:05.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.517+0000 7fa25a7fc700 1 -- 172.21.15.81:0/1330514218 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2600055a0 con 0x7fa2640c9230 2024-05-08T23:31:05.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.517+0000 7fa26acef700 1 -- 172.21.15.81:0/1330514218 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26409a3b0 con 0x7fa2640c9230 2024-05-08T23:31:05.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.517+0000 7fa26acef700 1 -- 172.21.15.81:0/1330514218 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26409a8d0 con 0x7fa2640c9230 2024-05-08T23:31:05.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.517+0000 7fa25a7fc700 1 -- 172.21.15.81:0/1330514218 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa2600036a0 con 0x7fa2640c9230 2024-05-08T23:31:05.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.517+0000 7fa25a7fc700 1 --2- 172.21.15.81:0/1330514218 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa248066730 0x7fa248068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.521+0000 7fa269ced700 1 --2- 172.21.15.81:0/1330514218 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa248066730 0x7fa248068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.521+0000 7fa25a7fc700 1 -- 172.21.15.81:0/1330514218 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2600832b0 con 0x7fa2640c9230 2024-05-08T23:31:05.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.521+0000 7fa269ced700 1 --2- 172.21.15.81:0/1330514218 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa248066730 0x7fa248068bf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa254000c00 tx=0x7fa254008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:05.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.525+0000 7fa25a7fc700 1 -- 172.21.15.81:0/1330514218 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa26409a8d0 con 0x7fa2640c9230 2024-05-08T23:31:05.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.693+0000 7fa26acef700 1 -- 172.21.15.81:0/1330514218 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa26409a8d0 con 0x7fa2640c9230 2024-05-08T23:31:05.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.697+0000 7fa25a7fc700 1 -- 172.21.15.81:0/1330514218 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa26409a8d0 con 0x7fa2640c9230 2024-05-08T23:31:05.700 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:05.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.697+0000 7fa26acef700 1 -- 172.21.15.81:0/1330514218 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa248066730 msgr2=0x7fa248068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.697+0000 7fa26acef700 1 --2- 172.21.15.81:0/1330514218 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa248066730 0x7fa248068bf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa254000c00 tx=0x7fa254008030 comp rx=0 tx=0).stop 2024-05-08T23:31:05.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.697+0000 7fa26acef700 1 -- 172.21.15.81:0/1330514218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2640c9230 msgr2=0x7fa26415f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.697+0000 7fa26acef700 1 --2- 172.21.15.81:0/1330514218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2640c9230 0x7fa26415f880 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa26000c3c0 tx=0x7fa26000c6d0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.697+0000 7fa26acef700 1 -- 172.21.15.81:0/1330514218 shutdown_connections 2024-05-08T23:31:05.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.697+0000 7fa26acef700 1 --2- 172.21.15.81:0/1330514218 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa248066730 0x7fa248068bf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.697+0000 7fa26acef700 1 --2- 172.21.15.81:0/1330514218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2640c9230 0x7fa26415f880 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.697+0000 7fa26acef700 1 --2- 172.21.15.81:0/1330514218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2640b3620 0x7fa2640c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.697+0000 7fa26acef700 1 --2- 172.21.15.81:0/1330514218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa26406d8c0 0x7fa26414e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.697+0000 7fa26acef700 1 -- 172.21.15.81:0/1330514218 >> 172.21.15.81:0/1330514218 conn(0x7fa2640b0de0 msgr2=0x7fa264161280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:05.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.697+0000 7fa26acef700 1 -- 172.21.15.81:0/1330514218 shutdown_connections 2024-05-08T23:31:05.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.697+0000 7fa26acef700 1 -- 172.21.15.81:0/1330514218 wait complete. 2024-05-08T23:31:05.716 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:05.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.829+0000 7fa10de46700 1 Processor -- start 2024-05-08T23:31:05.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.829+0000 7fa10de46700 1 -- start start 2024-05-08T23:31:05.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10de46700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa10815b080 0x7fa108099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10de46700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1080b2380 0x7fa10815c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10de46700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa10809a660 con 0x7fa1080b53c0 2024-05-08T23:31:05.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10de46700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10815b460 con 0x7fa10815b080 2024-05-08T23:31:05.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10de46700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10815b700 con 0x7fa1080b2380 2024-05-08T23:31:05.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa107fff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa1080b53c0 0x7fa1080b1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44594/0 (socket says 172.21.15.81:44594) 2024-05-08T23:31:05.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10ce44700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa10815b080 0x7fa108099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa107fff700 1 -- 172.21.15.81:0/3293677248 learned_addr learned my addr 172.21.15.81:0/3293677248 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:05.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10d645700 1 --2- 172.21.15.81:0/3293677248 >> v2:172.21.15.117:3300/0 conn(0x7fa1080b2380 0x7fa10815c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa1077fe700 1 -- 172.21.15.81:0/3293677248 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396504837 0 0) 0x7fa10809a660 con 0x7fa1080b53c0 2024-05-08T23:31:05.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa1077fe700 1 -- 172.21.15.81:0/3293677248 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0f00035d0 con 0x7fa1080b53c0 2024-05-08T23:31:05.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10ce44700 1 -- 172.21.15.81:0/3293677248 >> v2:172.21.15.117:3300/0 conn(0x7fa1080b2380 msgr2=0x7fa10815c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:05.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10ce44700 1 --2- 172.21.15.81:0/3293677248 >> v2:172.21.15.117:3300/0 conn(0x7fa1080b2380 0x7fa10815c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10ce44700 1 -- 172.21.15.81:0/3293677248 >> v1:172.21.15.81:6789/0 conn(0x7fa1080b53c0 legacy=0x7fa1080b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10ce44700 1 -- 172.21.15.81:0/3293677248 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10809a380 con 0x7fa10815b080 2024-05-08T23:31:05.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10ce44700 1 --2- 172.21.15.81:0/3293677248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10815b080 0x7fa108099a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa1080b57e0 tx=0x7fa0fc009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4092210b7525b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:05.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa1077fe700 1 -- 172.21.15.81:0/3293677248 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc011830 con 0x7fa10815b080 2024-05-08T23:31:05.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa1077fe700 1 -- 172.21.15.81:0/3293677248 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc011990 con 0x7fa10815b080 2024-05-08T23:31:05.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa1077fe700 1 -- 172.21.15.81:0/3293677248 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc011d00 con 0x7fa10815b080 2024-05-08T23:31:05.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10de46700 1 -- 172.21.15.81:0/3293677248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10815b080 msgr2=0x7fa108099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10de46700 1 --2- 172.21.15.81:0/3293677248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10815b080 0x7fa108099a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa1080b57e0 tx=0x7fa0fc009b30 comp rx=0 tx=0).stop 2024-05-08T23:31:05.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10de46700 1 -- 172.21.15.81:0/3293677248 shutdown_connections 2024-05-08T23:31:05.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10de46700 1 --2- 172.21.15.81:0/3293677248 >> v2:172.21.15.117:3300/0 conn(0x7fa1080b2380 0x7fa10815c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10de46700 1 --2- 172.21.15.81:0/3293677248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10815b080 0x7fa108099a70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10de46700 1 -- 172.21.15.81:0/3293677248 >> 172.21.15.81:0/3293677248 conn(0x7fa1080b2ae0 msgr2=0x7fa1080b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:05.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10de46700 1 -- 172.21.15.81:0/3293677248 shutdown_connections 2024-05-08T23:31:05.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.833+0000 7fa10de46700 1 -- 172.21.15.81:0/3293677248 wait complete. 2024-05-08T23:31:05.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.837+0000 7fa10de46700 1 Processor -- start 2024-05-08T23:31:05.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.837+0000 7fa10de46700 1 -- start start 2024-05-08T23:31:05.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.837+0000 7fa10de46700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1080b2380 0x7fa10809bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.837+0000 7fa10de46700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1080b53c0 0x7fa1080b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.837+0000 7fa10de46700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10815b080 0x7fa10809ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.837+0000 7fa10de46700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10809c270 con 0x7fa10815b080 2024-05-08T23:31:05.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.837+0000 7fa10de46700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10809af70 con 0x7fa1080b2380 2024-05-08T23:31:05.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.837+0000 7fa10de46700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10809b250 con 0x7fa1080b53c0 2024-05-08T23:31:05.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa10d645700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10815b080 0x7fa10809ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa10d645700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10815b080 0x7fa10809ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39058/0 (socket says 172.21.15.81:39058) 2024-05-08T23:31:05.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa10d645700 1 -- 172.21.15.81:0/1343116035 learned_addr learned my addr 172.21.15.81:0/1343116035 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:05.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa10ce44700 1 --2- 172.21.15.81:0/1343116035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1080b2380 0x7fa10809bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa107fff700 1 --2- 172.21.15.81:0/1343116035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1080b53c0 0x7fa1080b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa10d645700 1 -- 172.21.15.81:0/1343116035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1080b53c0 msgr2=0x7fa1080b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa10d645700 1 --2- 172.21.15.81:0/1343116035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1080b53c0 0x7fa1080b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa10d645700 1 -- 172.21.15.81:0/1343116035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1080b2380 msgr2=0x7fa10809bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:05.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa10d645700 1 --2- 172.21.15.81:0/1343116035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1080b2380 0x7fa10809bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:05.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa10d645700 1 -- 172.21.15.81:0/1343116035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc0094b0 con 0x7fa10815b080 2024-05-08T23:31:05.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa10d645700 1 --2- 172.21.15.81:0/1343116035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10815b080 0x7fa10809ca10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa10000efc0 tx=0x7fa10000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:05.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa1057fa700 1 -- 172.21.15.81:0/1343116035 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100009ea0 con 0x7fa10815b080 2024-05-08T23:31:05.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa1057fa700 1 -- 172.21.15.81:0/1343116035 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10000f040 con 0x7fa10815b080 2024-05-08T23:31:05.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa10de46700 1 -- 172.21.15.81:0/1343116035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10809b530 con 0x7fa10815b080 2024-05-08T23:31:05.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa1057fa700 1 -- 172.21.15.81:0/1343116035 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100014480 con 0x7fa10815b080 2024-05-08T23:31:05.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa10de46700 1 -- 172.21.15.81:0/1343116035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10809d720 con 0x7fa10815b080 2024-05-08T23:31:05.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa10de46700 1 -- 172.21.15.81:0/1343116035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10809ba50 con 0x7fa10815b080 2024-05-08T23:31:05.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.841+0000 7fa1057fa700 1 -- 172.21.15.81:0/1343116035 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa1000145e0 con 0x7fa10815b080 2024-05-08T23:31:05.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.845+0000 7fa1057fa700 1 --2- 172.21.15.81:0/1343116035 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa0f4066730 0x7fa0f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:05.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.845+0000 7fa1057fa700 1 -- 172.21.15.81:0/1343116035 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa100084c70 con 0x7fa10815b080 2024-05-08T23:31:05.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.845+0000 7fa10ce44700 1 --2- 172.21.15.81:0/1343116035 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa0f4066730 0x7fa0f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:05.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.845+0000 7fa10ce44700 1 --2- 172.21.15.81:0/1343116035 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa0f4066730 0x7fa0f4068bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc000c00 tx=0x7fa0fc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:05.853 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:05.849+0000 7fa1057fa700 1 -- 172.21.15.81:0/1343116035 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa10809ba50 con 0x7fa10815b080 2024-05-08T23:31:06.011 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:31:06.014 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:31:06.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.013+0000 7fa10de46700 1 -- 172.21.15.81:0/1343116035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa10809ba50 con 0x7fa10815b080 2024-05-08T23:31:06.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.017+0000 7fa1057fa700 1 -- 172.21.15.81:0/1343116035 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa100018020 con 0x7fa10815b080 2024-05-08T23:31:06.020 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:06.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.017+0000 7fa10de46700 1 -- 172.21.15.81:0/1343116035 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa0f4066730 msgr2=0x7fa0f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.017+0000 7fa10de46700 1 --2- 172.21.15.81:0/1343116035 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa0f4066730 0x7fa0f4068bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc000c00 tx=0x7fa0fc008030 comp rx=0 tx=0).stop 2024-05-08T23:31:06.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.017+0000 7fa10de46700 1 -- 172.21.15.81:0/1343116035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10815b080 msgr2=0x7fa10809ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.017+0000 7fa10de46700 1 --2- 172.21.15.81:0/1343116035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10815b080 0x7fa10809ca10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa10000efc0 tx=0x7fa10000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.017+0000 7fa10de46700 1 -- 172.21.15.81:0/1343116035 shutdown_connections 2024-05-08T23:31:06.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.017+0000 7fa10de46700 1 --2- 172.21.15.81:0/1343116035 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa0f4066730 0x7fa0f4068bf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.017+0000 7fa10de46700 1 --2- 172.21.15.81:0/1343116035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10815b080 0x7fa10809ca10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.017+0000 7fa10de46700 1 --2- 172.21.15.81:0/1343116035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1080b53c0 0x7fa1080b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.017+0000 7fa10de46700 1 --2- 172.21.15.81:0/1343116035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1080b2380 0x7fa10809bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.017+0000 7fa10de46700 1 -- 172.21.15.81:0/1343116035 >> 172.21.15.81:0/1343116035 conn(0x7fa1080b2ae0 msgr2=0x7fa1081562a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:06.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.017+0000 7fa10de46700 1 -- 172.21.15.81:0/1343116035 shutdown_connections 2024-05-08T23:31:06.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.017+0000 7fa10de46700 1 -- 172.21.15.81:0/1343116035 wait complete. 2024-05-08T23:31:06.034 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:06.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de9e7c700 1 Processor -- start 2024-05-08T23:31:06.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de9e7c700 1 -- start start 2024-05-08T23:31:06.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de9e7c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2de415d2f0 0x7f2de40ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de9e7c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2de40cf8b0 0x7f2de40cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de9e7c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de40ceb40 con 0x7f2de40b3550 2024-05-08T23:31:06.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de9e7c700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de40d3a10 con 0x7f2de415d2f0 2024-05-08T23:31:06.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de9e7c700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de415d6d0 con 0x7f2de40cf8b0 2024-05-08T23:31:06.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de8e7a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2de415d2f0 0x7f2de40ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de8e7a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2de415d2f0 0x7f2de40ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58206/0 (socket says 172.21.15.81:58206) 2024-05-08T23:31:06.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de8e7a700 1 -- 172.21.15.81:0/3154008633 learned_addr learned my addr 172.21.15.81:0/3154008633 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:06.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de3fff700 1 --2- 172.21.15.81:0/3154008633 >> v2:172.21.15.117:3300/0 conn(0x7f2de40cf8b0 0x7f2de40cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de8e7a700 1 -- 172.21.15.81:0/3154008633 >> v2:172.21.15.117:3300/0 conn(0x7f2de40cf8b0 msgr2=0x7f2de40cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:06.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de8e7a700 1 --2- 172.21.15.81:0/3154008633 >> v2:172.21.15.117:3300/0 conn(0x7f2de40cf8b0 0x7f2de40cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de8e7a700 1 -- 172.21.15.81:0/3154008633 >> v1:172.21.15.117:6789/0 conn(0x7f2de40b3550 legacy=0x7f2de40b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de8e7a700 1 -- 172.21.15.81:0/3154008633 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de40d0250 con 0x7f2de415d2f0 2024-05-08T23:31:06.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de8e7a700 1 --2- 172.21.15.81:0/3154008633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2de415d2f0 0x7f2de40ce600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2de408bea0 tx=0x7f2dcc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37892421bb229418 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:06.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de37fe700 1 -- 172.21.15.81:0/3154008633 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc011830 con 0x7f2de415d2f0 2024-05-08T23:31:06.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de37fe700 1 -- 172.21.15.81:0/3154008633 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc011990 con 0x7f2de415d2f0 2024-05-08T23:31:06.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.141+0000 7f2de37fe700 1 -- 172.21.15.81:0/3154008633 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc011d20 con 0x7f2de415d2f0 2024-05-08T23:31:06.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 -- 172.21.15.81:0/3154008633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2de415d2f0 msgr2=0x7f2de40ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 --2- 172.21.15.81:0/3154008633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2de415d2f0 0x7f2de40ce600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2de408bea0 tx=0x7f2dcc009bf0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 -- 172.21.15.81:0/3154008633 shutdown_connections 2024-05-08T23:31:06.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 --2- 172.21.15.81:0/3154008633 >> v2:172.21.15.117:3300/0 conn(0x7f2de40cf8b0 0x7f2de40cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 --2- 172.21.15.81:0/3154008633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2de415d2f0 0x7f2de40ce600 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 -- 172.21.15.81:0/3154008633 >> 172.21.15.81:0/3154008633 conn(0x7f2de409c320 msgr2=0x7f2de40d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:06.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 -- 172.21.15.81:0/3154008633 shutdown_connections 2024-05-08T23:31:06.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 -- 172.21.15.81:0/3154008633 wait complete. 2024-05-08T23:31:06.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 Processor -- start 2024-05-08T23:31:06.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 -- start start 2024-05-08T23:31:06.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de40b3550 0x7f2de415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2de40cf8b0 0x7f2de40caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de415d2f0 0x7f2de41599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de4159ee0 con 0x7f2de40cf8b0 2024-05-08T23:31:06.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de40cd3d0 con 0x7f2de415d2f0 2024-05-08T23:31:06.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de9e7c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de40ca0b0 con 0x7f2de40b3550 2024-05-08T23:31:06.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2de40cf8b0 0x7f2de40caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2de40cf8b0 0x7f2de40caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58220/0 (socket says 172.21.15.81:58220) 2024-05-08T23:31:06.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.145+0000 7f2de3fff700 1 -- 172.21.15.81:0/3391393347 learned_addr learned my addr 172.21.15.81:0/3391393347 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:06.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de8e7a700 1 --2- 172.21.15.81:0/3391393347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de40b3550 0x7f2de415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de967b700 1 --2- 172.21.15.81:0/3391393347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de415d2f0 0x7f2de41599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de3fff700 1 -- 172.21.15.81:0/3391393347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de40b3550 msgr2=0x7f2de415a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de3fff700 1 --2- 172.21.15.81:0/3391393347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de40b3550 0x7f2de415a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de3fff700 1 -- 172.21.15.81:0/3391393347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de415d2f0 msgr2=0x7f2de41599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de3fff700 1 --2- 172.21.15.81:0/3391393347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de415d2f0 0x7f2de41599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de3fff700 1 -- 172.21.15.81:0/3391393347 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc009570 con 0x7f2de40cf8b0 2024-05-08T23:31:06.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de8e7a700 1 --2- 172.21.15.81:0/3391393347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de40b3550 0x7f2de415a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:06.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de3fff700 1 --2- 172.21.15.81:0/3391393347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2de40cf8b0 0x7f2de40caa40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400e9d0 tx=0x7f2dd400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:06.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de17fa700 1 -- 172.21.15.81:0/3391393347 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4009720 con 0x7f2de40cf8b0 2024-05-08T23:31:06.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de9e7c700 1 -- 172.21.15.81:0/3391393347 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de40ca390 con 0x7f2de40cf8b0 2024-05-08T23:31:06.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de17fa700 1 -- 172.21.15.81:0/3391393347 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd400c850 con 0x7f2de40cf8b0 2024-05-08T23:31:06.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de9e7c700 1 -- 172.21.15.81:0/3391393347 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de40b67b0 con 0x7f2de40cf8b0 2024-05-08T23:31:06.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de17fa700 1 -- 172.21.15.81:0/3391393347 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4005c40 con 0x7f2de40cf8b0 2024-05-08T23:31:06.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de9e7c700 1 -- 172.21.15.81:0/3391393347 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de40684b0 con 0x7f2de40cf8b0 2024-05-08T23:31:06.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de17fa700 1 -- 172.21.15.81:0/3391393347 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2dd400c9c0 con 0x7f2de40cf8b0 2024-05-08T23:31:06.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de17fa700 1 --2- 172.21.15.81:0/3391393347 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2dc4066730 0x7f2dc4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de17fa700 1 -- 172.21.15.81:0/3391393347 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2dd4084310 con 0x7f2de40cf8b0 2024-05-08T23:31:06.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.149+0000 7f2de8e7a700 1 --2- 172.21.15.81:0/3391393347 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2dc4066730 0x7f2dc4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.153+0000 7f2de8e7a700 1 --2- 172.21.15.81:0/3391393347 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2dc4066730 0x7f2dc4068bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc000c00 tx=0x7f2dcc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:06.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.157+0000 7f2de17fa700 1 -- 172.21.15.81:0/3391393347 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2dd4051600 con 0x7f2de40cf8b0 2024-05-08T23:31:06.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.325+0000 7f2de9e7c700 1 -- 172.21.15.81:0/3391393347 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2de40a0d70 con 0x7f2de40cf8b0 2024-05-08T23:31:06.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.329+0000 7f2de17fa700 1 -- 172.21.15.81:0/3391393347 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2dd4054c20 con 0x7f2de40cf8b0 2024-05-08T23:31:06.332 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:06.337 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.333+0000 7f2de9e7c700 1 -- 172.21.15.81:0/3391393347 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2dc4066730 msgr2=0x7f2dc4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.337 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.333+0000 7f2de9e7c700 1 --2- 172.21.15.81:0/3391393347 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2dc4066730 0x7f2dc4068bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc000c00 tx=0x7f2dcc008030 comp rx=0 tx=0).stop 2024-05-08T23:31:06.337 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.333+0000 7f2de9e7c700 1 -- 172.21.15.81:0/3391393347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2de40cf8b0 msgr2=0x7f2de40caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.333+0000 7f2de9e7c700 1 --2- 172.21.15.81:0/3391393347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2de40cf8b0 0x7f2de40caa40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400e9d0 tx=0x7f2dd400ed90 comp rx=0 tx=0).stop 2024-05-08T23:31:06.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.333+0000 7f2de9e7c700 1 -- 172.21.15.81:0/3391393347 shutdown_connections 2024-05-08T23:31:06.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.333+0000 7f2de9e7c700 1 --2- 172.21.15.81:0/3391393347 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2dc4066730 0x7f2dc4068bf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.333+0000 7f2de9e7c700 1 --2- 172.21.15.81:0/3391393347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de415d2f0 0x7f2de41599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.333+0000 7f2de9e7c700 1 --2- 172.21.15.81:0/3391393347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2de40cf8b0 0x7f2de40caa40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.333+0000 7f2de9e7c700 1 --2- 172.21.15.81:0/3391393347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de40b3550 0x7f2de415a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.333+0000 7f2de9e7c700 1 -- 172.21.15.81:0/3391393347 >> 172.21.15.81:0/3391393347 conn(0x7f2de409c320 msgr2=0x7f2de40b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:06.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.333+0000 7f2de9e7c700 1 -- 172.21.15.81:0/3391393347 shutdown_connections 2024-05-08T23:31:06.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.333+0000 7f2de9e7c700 1 -- 172.21.15.81:0/3391393347 wait complete. 2024-05-08T23:31:06.354 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:06.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fdcc5700 1 Processor -- start 2024-05-08T23:31:06.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fdcc5700 1 -- start start 2024-05-08T23:31:06.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fdcc5700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f8159e80 0x7f19f8070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fdcc5700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f19f8071180 0x7f19f80cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fdcc5700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f80cc950 con 0x7f19f8142bf0 2024-05-08T23:31:06.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fdcc5700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f80ccbb0 con 0x7f19f8071180 2024-05-08T23:31:06.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fdcc5700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80719e0 con 0x7f19f8159e80 2024-05-08T23:31:06.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fd4c4700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f19f8071180 0x7f19f80cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fd4c4700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f19f8071180 0x7f19f80cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:59852/0 (socket says 172.21.15.81:59852) 2024-05-08T23:31:06.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fd4c4700 1 -- 172.21.15.81:0/1629243391 learned_addr learned my addr 172.21.15.81:0/1629243391 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:06.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19f7fff700 1 --2- 172.21.15.81:0/1629243391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f8159e80 0x7f19f8070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fccc3700 1 --1- 172.21.15.81:0/1629243391 >> v1:172.21.15.117:6789/0 conn(0x7f19f8142bf0 0x7f19f8159770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:60176/0 (socket says 172.21.15.81:60176) 2024-05-08T23:31:06.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fd4c4700 1 -- 172.21.15.81:0/1629243391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f8159e80 msgr2=0x7f19f8070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:06.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fd4c4700 1 --2- 172.21.15.81:0/1629243391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f8159e80 0x7f19f8070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fd4c4700 1 -- 172.21.15.81:0/1629243391 >> v1:172.21.15.117:6789/0 conn(0x7f19f8142bf0 legacy=0x7f19f8159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fd4c4700 1 -- 172.21.15.81:0/1629243391 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f80cce70 con 0x7f19f8071180 2024-05-08T23:31:06.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19f7fff700 1 --2- 172.21.15.81:0/1629243391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f8159e80 0x7f19f8070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:06.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19fd4c4700 1 --2- 172.21.15.81:0/1629243391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f8071180 0x7f19f80cd670 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009940 tx=0x7f19f0009c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aaef21e1c79b078 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:06.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19f77fe700 1 -- 172.21.15.81:0/1629243391 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0011830 con 0x7f19f8071180 2024-05-08T23:31:06.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19f77fe700 1 -- 172.21.15.81:0/1629243391 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f0011990 con 0x7f19f8071180 2024-05-08T23:31:06.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.469+0000 7f19f77fe700 1 -- 172.21.15.81:0/1629243391 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0011d20 con 0x7f19f8071180 2024-05-08T23:31:06.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.473+0000 7f19fdcc5700 1 -- 172.21.15.81:0/1629243391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f8071180 msgr2=0x7f19f80cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.473+0000 7f19fdcc5700 1 --2- 172.21.15.81:0/1629243391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f8071180 0x7f19f80cd670 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009940 tx=0x7f19f0009c50 comp rx=0 tx=0).stop 2024-05-08T23:31:06.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.473+0000 7f19fdcc5700 1 -- 172.21.15.81:0/1629243391 shutdown_connections 2024-05-08T23:31:06.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.473+0000 7f19fdcc5700 1 --2- 172.21.15.81:0/1629243391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f8071180 0x7f19f80cd670 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.473+0000 7f19fdcc5700 1 --2- 172.21.15.81:0/1629243391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f8159e80 0x7f19f8070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.473+0000 7f19fdcc5700 1 -- 172.21.15.81:0/1629243391 >> 172.21.15.81:0/1629243391 conn(0x7f19f806c330 msgr2=0x7f19f806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:06.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.473+0000 7f19fdcc5700 1 -- 172.21.15.81:0/1629243391 shutdown_connections 2024-05-08T23:31:06.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.473+0000 7f19fdcc5700 1 -- 172.21.15.81:0/1629243391 wait complete. 2024-05-08T23:31:06.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.473+0000 7f19fdcc5700 1 Processor -- start 2024-05-08T23:31:06.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.473+0000 7f19fdcc5700 1 -- start start 2024-05-08T23:31:06.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fdcc5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19f8071180 0x7f19f8153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fdcc5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f8142bf0 0x7f19f815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fdcc5700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f8159e80 0x7f19f806d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fdcc5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f8142210 con 0x7f19f8071180 2024-05-08T23:31:06.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fdcc5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80b3b90 con 0x7f19f8142bf0 2024-05-08T23:31:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fdcc5700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80b38b0 con 0x7f19f8159e80 2024-05-08T23:31:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fccc3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19f8071180 0x7f19f8153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fccc3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19f8071180 0x7f19f8153de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58224/0 (socket says 172.21.15.81:58224) 2024-05-08T23:31:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fccc3700 1 -- 172.21.15.81:0/2066603850 learned_addr learned my addr 172.21.15.81:0/2066603850 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19f7fff700 1 --2- 172.21.15.81:0/2066603850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f8142bf0 0x7f19f815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fd4c4700 1 --2- 172.21.15.81:0/2066603850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f8159e80 0x7f19f806d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fccc3700 1 -- 172.21.15.81:0/2066603850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f8159e80 msgr2=0x7f19f806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fccc3700 1 --2- 172.21.15.81:0/2066603850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f8159e80 0x7f19f806d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fccc3700 1 -- 172.21.15.81:0/2066603850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f8142bf0 msgr2=0x7f19f815ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fccc3700 1 --2- 172.21.15.81:0/2066603850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f8142bf0 0x7f19f815ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fccc3700 1 -- 172.21.15.81:0/2066603850 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0009a70 con 0x7f19f8071180 2024-05-08T23:31:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fccc3700 1 --2- 172.21.15.81:0/2066603850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19f8071180 0x7f19f8153de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c500 tx=0x7f19e800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19f57fa700 1 -- 172.21.15.81:0/2066603850 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8005cd0 con 0x7f19f8071180 2024-05-08T23:31:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19f57fa700 1 -- 172.21.15.81:0/2066603850 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e8005e30 con 0x7f19f8071180 2024-05-08T23:31:06.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fdcc5700 1 -- 172.21.15.81:0/2066603850 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f0009570 con 0x7f19f8071180 2024-05-08T23:31:06.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19f57fa700 1 -- 172.21.15.81:0/2066603850 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e80072f0 con 0x7f19f8071180 2024-05-08T23:31:06.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fdcc5700 1 -- 172.21.15.81:0/2066603850 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f8158900 con 0x7f19f8071180 2024-05-08T23:31:06.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19fdcc5700 1 -- 172.21.15.81:0/2066603850 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f80684b0 con 0x7f19f8071180 2024-05-08T23:31:06.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.477+0000 7f19f57fa700 1 -- 172.21.15.81:0/2066603850 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f19e8007450 con 0x7f19f8071180 2024-05-08T23:31:06.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.481+0000 7f19f57fa700 1 --2- 172.21.15.81:0/2066603850 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f19dc066760 0x7f19dc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.481+0000 7f19f7fff700 1 --2- 172.21.15.81:0/2066603850 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f19dc066760 0x7f19dc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.481+0000 7f19f57fa700 1 -- 172.21.15.81:0/2066603850 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19e80835c0 con 0x7f19f8071180 2024-05-08T23:31:06.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.481+0000 7f19f7fff700 1 --2- 172.21.15.81:0/2066603850 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f19dc066760 0x7f19dc068c20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f19f813f4e0 tx=0x7f19e000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:06.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.485+0000 7f19f57fa700 1 -- 172.21.15.81:0/2066603850 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f19e8050890 con 0x7f19f8071180 2024-05-08T23:31:06.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.657+0000 7f19fdcc5700 1 -- 172.21.15.81:0/2066603850 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19f80ae9e0 con 0x7f19f8071180 2024-05-08T23:31:06.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.661+0000 7f19f57fa700 1 -- 172.21.15.81:0/2066603850 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f19e8053eb0 con 0x7f19f8071180 2024-05-08T23:31:06.665 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:06.665 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:06.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.661+0000 7f19fdcc5700 1 -- 172.21.15.81:0/2066603850 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f19dc066760 msgr2=0x7f19dc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.661+0000 7f19fdcc5700 1 --2- 172.21.15.81:0/2066603850 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f19dc066760 0x7f19dc068c20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f19f813f4e0 tx=0x7f19e000b040 comp rx=0 tx=0).stop 2024-05-08T23:31:06.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.661+0000 7f19fdcc5700 1 -- 172.21.15.81:0/2066603850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19f8071180 msgr2=0x7f19f8153de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.661+0000 7f19fdcc5700 1 --2- 172.21.15.81:0/2066603850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19f8071180 0x7f19f8153de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c500 tx=0x7f19e800c810 comp rx=0 tx=0).stop 2024-05-08T23:31:06.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.661+0000 7f19fdcc5700 1 -- 172.21.15.81:0/2066603850 shutdown_connections 2024-05-08T23:31:06.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.661+0000 7f19fdcc5700 1 --2- 172.21.15.81:0/2066603850 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f19dc066760 0x7f19dc068c20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.661+0000 7f19fdcc5700 1 --2- 172.21.15.81:0/2066603850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f8159e80 0x7f19f806d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.661+0000 7f19fdcc5700 1 --2- 172.21.15.81:0/2066603850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f8142bf0 0x7f19f815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.661+0000 7f19fdcc5700 1 --2- 172.21.15.81:0/2066603850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19f8071180 0x7f19f8153de0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.661+0000 7f19fdcc5700 1 -- 172.21.15.81:0/2066603850 >> 172.21.15.81:0/2066603850 conn(0x7f19f806c330 msgr2=0x7f19f80aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:06.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.661+0000 7f19fdcc5700 1 -- 172.21.15.81:0/2066603850 shutdown_connections 2024-05-08T23:31:06.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.661+0000 7f19fdcc5700 1 -- 172.21.15.81:0/2066603850 wait complete. 2024-05-08T23:31:06.680 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:06.681 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:06.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.785+0000 7f4bc7b3c700 1 Processor -- start 2024-05-08T23:31:06.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.785+0000 7f4bc7b3c700 1 -- start start 2024-05-08T23:31:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc7b3c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4bc0146ae0 0x7f4bc006b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc7b3c700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc015d990 con 0x7f4bc00b40d0 2024-05-08T23:31:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc7b3c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc006c2f0 con 0x7f4bc00b37f0 2024-05-08T23:31:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc7b3c700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc0146ec0 con 0x7f4bc0146ae0 2024-05-08T23:31:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc50d7700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f4bc00b40d0 0x7f4bc00b30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46676/0 (socket says 172.21.15.81:46676) 2024-05-08T23:31:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc50d7700 1 -- 172.21.15.81:0/1111675660 learned_addr learned my addr 172.21.15.81:0/1111675660 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc58d8700 1 --2- 172.21.15.81:0/1111675660 >> v2:172.21.15.117:3300/0 conn(0x7f4bc0146ae0 0x7f4bc006b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595090152 0 0) 0x7f4bc015d990 con 0x7f4bc00b40d0 2024-05-08T23:31:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bac0035d0 con 0x7f4bc00b40d0 2024-05-08T23:31:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702534272 0 0) 0x7f4bc006c2f0 con 0x7f4bc00b37f0 2024-05-08T23:31:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc015d990 con 0x7f4bc00b37f0 2024-05-08T23:31:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3408377389 0 0) 0x7f4bac0035d0 con 0x7f4bc00b40d0 2024-05-08T23:31:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 >> v2:172.21.15.117:3300/0 conn(0x7f4bc0146ae0 msgr2=0x7f4bc006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 --2- 172.21.15.81:0/1111675660 >> v2:172.21.15.117:3300/0 conn(0x7f4bc0146ae0 0x7f4bc006b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 >> v1:172.21.15.117:6789/0 conn(0x7f4bc00b37f0 legacy=0x7f4bc015de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc006c5d0 con 0x7f4bc00b40d0 2024-05-08T23:31:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f4bbc001670 con 0x7f4bc00b40d0 2024-05-08T23:31:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 >> v1:172.21.15.81:6789/0 conn(0x7f4bc00b40d0 legacy=0x7f4bc00b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 --2- 172.21.15.81:0/1111675660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bac003bb0 0x7f4bac003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 --2- 172.21.15.81:0/1111675660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bac004870 0x7f4bac004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 --2- 172.21.15.81:0/1111675660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bac005230 0x7f4bac00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc0146ec0 con 0x7f4bac004870 2024-05-08T23:31:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac00c0b0 con 0x7f4bac003bb0 2024-05-08T23:31:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac00c270 con 0x7f4bac005230 2024-05-08T23:31:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc50d7700 1 --2- 172.21.15.81:0/1111675660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bac004870 0x7f4bac004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc58d8700 1 --2- 172.21.15.81:0/1111675660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bac003bb0 0x7f4bac003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc60d9700 1 --2- 172.21.15.81:0/1111675660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bac005230 0x7f4bac00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc50d7700 1 -- 172.21.15.81:0/1111675660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bac005230 msgr2=0x7f4bac00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:06.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc50d7700 1 --2- 172.21.15.81:0/1111675660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bac005230 0x7f4bac00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc50d7700 1 -- 172.21.15.81:0/1111675660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bac003bb0 msgr2=0x7f4bac003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:06.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc50d7700 1 --2- 172.21.15.81:0/1111675660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bac003bb0 0x7f4bac003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc50d7700 1 -- 172.21.15.81:0/1111675660 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac0035d0 con 0x7f4bac004870 2024-05-08T23:31:06.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc60d9700 1 -- 172.21.15.81:0/1111675660 reap_dead start 2024-05-08T23:31:06.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc50d7700 1 --2- 172.21.15.81:0/1111675660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bac004870 0x7f4bac004cf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc007fd0 tx=0x7f4bac0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1051d5889822bf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:06.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00cd20 con 0x7f4bac004870 2024-05-08T23:31:06.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc001af0 con 0x7f4bac004870 2024-05-08T23:31:06.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.789+0000 7f4bc48d6700 1 -- 172.21.15.81:0/1111675660 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00cea0 con 0x7f4bac004870 2024-05-08T23:31:06.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc7b3c700 1 -- 172.21.15.81:0/1111675660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bac004870 msgr2=0x7f4bac004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc7b3c700 1 --2- 172.21.15.81:0/1111675660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bac004870 0x7f4bac004cf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc007fd0 tx=0x7f4bac0058a0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc7b3c700 1 -- 172.21.15.81:0/1111675660 shutdown_connections 2024-05-08T23:31:06.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc7b3c700 1 --2- 172.21.15.81:0/1111675660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bac004870 0x7f4bac004cf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc7b3c700 1 -- 172.21.15.81:0/1111675660 >> 172.21.15.81:0/1111675660 conn(0x7f4bc0073ab0 msgr2=0x7f4bc0073ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:06.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc7b3c700 1 -- 172.21.15.81:0/1111675660 shutdown_connections 2024-05-08T23:31:06.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc7b3c700 1 -- 172.21.15.81:0/1111675660 wait complete. 2024-05-08T23:31:06.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc7b3c700 1 Processor -- start 2024-05-08T23:31:06.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc7b3c700 1 -- start start 2024-05-08T23:31:06.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc7b3c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bac004870 0x7f4bc0153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc7b3c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc015cae0 0x7f4bc0141010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc7b3c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bc00cb380 0x7f4bc015fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc7b3c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc013e200 con 0x7f4bc00cb380 2024-05-08T23:31:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc7b3c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00ca370 con 0x7f4bc015cae0 2024-05-08T23:31:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc7b3c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc0153dd0 con 0x7f4bac004870 2024-05-08T23:31:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc60d9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bc00cb380 0x7f4bc015fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc60d9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bc00cb380 0x7f4bc015fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58236/0 (socket says 172.21.15.81:58236) 2024-05-08T23:31:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc60d9700 1 -- 172.21.15.81:0/2112823789 learned_addr learned my addr 172.21.15.81:0/2112823789 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc58d8700 1 --2- 172.21.15.81:0/2112823789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bac004870 0x7f4bc0153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.793+0000 7f4bc50d7700 1 --2- 172.21.15.81:0/2112823789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc015cae0 0x7f4bc0141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.797+0000 7f4bc60d9700 1 -- 172.21.15.81:0/2112823789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bac004870 msgr2=0x7f4bc0153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.797+0000 7f4bc60d9700 1 --2- 172.21.15.81:0/2112823789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bac004870 0x7f4bc0153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.797+0000 7f4bc60d9700 1 -- 172.21.15.81:0/2112823789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc015cae0 msgr2=0x7f4bc0141010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.797+0000 7f4bc60d9700 1 --2- 172.21.15.81:0/2112823789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc015cae0 0x7f4bc0141010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.797+0000 7f4bc60d9700 1 -- 172.21.15.81:0/2112823789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc00ca5c0 con 0x7f4bc00cb380 2024-05-08T23:31:06.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.797+0000 7f4bc60d9700 1 --2- 172.21.15.81:0/2112823789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bc00cb380 0x7f4bc015fa10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4bac0047f0 tx=0x7f4bb800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:06.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.797+0000 7f4bb67fc700 1 -- 172.21.15.81:0/2112823789 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8003c30 con 0x7f4bc00cb380 2024-05-08T23:31:06.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.797+0000 7f4bb67fc700 1 -- 172.21.15.81:0/2112823789 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb8003d90 con 0x7f4bc00cb380 2024-05-08T23:31:06.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.797+0000 7f4bb67fc700 1 -- 172.21.15.81:0/2112823789 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800dbd0 con 0x7f4bc00cb380 2024-05-08T23:31:06.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.797+0000 7f4bc7b3c700 1 -- 172.21.15.81:0/2112823789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc00ca8a0 con 0x7f4bc00cb380 2024-05-08T23:31:06.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.797+0000 7f4bc7b3c700 1 -- 172.21.15.81:0/2112823789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc00cae60 con 0x7f4bc00cb380 2024-05-08T23:31:06.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.797+0000 7f4bc7b3c700 1 -- 172.21.15.81:0/2112823789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc00684b0 con 0x7f4bc00cb380 2024-05-08T23:31:06.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.801+0000 7f4bb67fc700 1 -- 172.21.15.81:0/2112823789 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4bb800dd30 con 0x7f4bc00cb380 2024-05-08T23:31:06.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.801+0000 7f4bb67fc700 1 --2- 172.21.15.81:0/2112823789 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4ba4066730 0x7f4ba4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:06.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.801+0000 7f4bb67fc700 1 -- 172.21.15.81:0/2112823789 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bb80842b0 con 0x7f4bc00cb380 2024-05-08T23:31:06.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.801+0000 7f4bc58d8700 1 --2- 172.21.15.81:0/2112823789 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4ba4066730 0x7f4ba4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:06.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.801+0000 7f4bc58d8700 1 --2- 172.21.15.81:0/2112823789 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4ba4066730 0x7f4ba4068bf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0064010 tx=0x7f4bb0009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:06.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.805+0000 7f4bb67fc700 1 -- 172.21.15.81:0/2112823789 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4bb80516b0 con 0x7f4bc00cb380 2024-05-08T23:31:06.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.977+0000 7f4bc7b3c700 1 -- 172.21.15.81:0/2112823789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4bc01452f0 con 0x7f4bc00cb380 2024-05-08T23:31:06.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.977+0000 7f4bb67fc700 1 -- 172.21.15.81:0/2112823789 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4bb8054cd0 con 0x7f4bc00cb380 2024-05-08T23:31:06.981 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:06.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.981+0000 7f4bc7b3c700 1 -- 172.21.15.81:0/2112823789 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4ba4066730 msgr2=0x7f4ba4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.981+0000 7f4bc7b3c700 1 --2- 172.21.15.81:0/2112823789 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4ba4066730 0x7f4ba4068bf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0064010 tx=0x7f4bb0009240 comp rx=0 tx=0).stop 2024-05-08T23:31:06.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.981+0000 7f4bc7b3c700 1 -- 172.21.15.81:0/2112823789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bc00cb380 msgr2=0x7f4bc015fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:06.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.981+0000 7f4bc7b3c700 1 --2- 172.21.15.81:0/2112823789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bc00cb380 0x7f4bc015fa10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4bac0047f0 tx=0x7f4bb800d430 comp rx=0 tx=0).stop 2024-05-08T23:31:06.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.981+0000 7f4bc7b3c700 1 -- 172.21.15.81:0/2112823789 shutdown_connections 2024-05-08T23:31:06.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.981+0000 7f4bc7b3c700 1 --2- 172.21.15.81:0/2112823789 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4ba4066730 0x7f4ba4068bf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.981+0000 7f4bc7b3c700 1 --2- 172.21.15.81:0/2112823789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bc00cb380 0x7f4bc015fa10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.981+0000 7f4bc7b3c700 1 --2- 172.21.15.81:0/2112823789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bc015cae0 0x7f4bc0141010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.981+0000 7f4bc7b3c700 1 --2- 172.21.15.81:0/2112823789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bac004870 0x7f4bc0153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:06.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.981+0000 7f4bc7b3c700 1 -- 172.21.15.81:0/2112823789 >> 172.21.15.81:0/2112823789 conn(0x7f4bc0073ab0 msgr2=0x7f4bc015f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:06.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.981+0000 7f4bc7b3c700 1 -- 172.21.15.81:0/2112823789 shutdown_connections 2024-05-08T23:31:06.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:06.981+0000 7f4bc7b3c700 1 -- 172.21.15.81:0/2112823789 wait complete. 2024-05-08T23:31:06.996 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:07.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fceed65d700 1 Processor -- start 2024-05-08T23:31:07.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fceed65d700 1 -- start start 2024-05-08T23:31:07.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fceed65d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcee815bc40 0x7fcee815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fceed65d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcee8162bc0 0x7fcee8164fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fceed65d700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee8155ad0 con 0x7fcee815b860 2024-05-08T23:31:07.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fceed65d700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee80cf2d0 con 0x7fcee8162bc0 2024-05-08T23:31:07.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fcee7fff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcee815b860 0x7fcee80ae850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46690/0 (socket says 172.21.15.81:46690) 2024-05-08T23:31:07.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fcee7fff700 1 -- 172.21.15.81:0/1550428633 learned_addr learned my addr 172.21.15.81:0/1550428633 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:07.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fceed65d700 1 -- 172.21.15.81:0/1550428633 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee80b3df0 con 0x7fcee815bc40 2024-05-08T23:31:07.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fceece5c700 1 --2- 172.21.15.81:0/1550428633 >> v2:172.21.15.81:3300/0 conn(0x7fcee8162bc0 0x7fcee8164fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fcee77fe700 1 --2- 172.21.15.81:0/1550428633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcee815bc40 0x7fcee815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fcee6ffd700 1 -- 172.21.15.81:0/1550428633 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225409977 0 0) 0x7fced8001110 con 0x7fcee815b860 2024-05-08T23:31:07.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fceece5c700 1 -- 172.21.15.81:0/1550428633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcee815bc40 msgr2=0x7fcee815c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:07.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fceece5c700 1 --2- 172.21.15.81:0/1550428633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcee815bc40 0x7fcee815c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fceece5c700 1 -- 172.21.15.81:0/1550428633 >> v1:172.21.15.81:6789/0 conn(0x7fcee815b860 legacy=0x7fcee80ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fceece5c700 1 -- 172.21.15.81:0/1550428633 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee815d780 con 0x7fcee8162bc0 2024-05-08T23:31:07.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fceece5c700 1 --2- 172.21.15.81:0/1550428633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcee8162bc0 0x7fcee8164fb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcee0009d90 tx=0x7fcee000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c65a5795910b90e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:07.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fcee6ffd700 1 -- 172.21.15.81:0/1550428633 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee000e070 con 0x7fcee8162bc0 2024-05-08T23:31:07.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fcee6ffd700 1 -- 172.21.15.81:0/1550428633 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee000bbd0 con 0x7fcee8162bc0 2024-05-08T23:31:07.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.105+0000 7fcee6ffd700 1 -- 172.21.15.81:0/1550428633 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0012430 con 0x7fcee8162bc0 2024-05-08T23:31:07.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.109+0000 7fceed65d700 1 -- 172.21.15.81:0/1550428633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcee8162bc0 msgr2=0x7fcee8164fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.109+0000 7fceed65d700 1 --2- 172.21.15.81:0/1550428633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcee8162bc0 0x7fcee8164fb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcee0009d90 tx=0x7fcee000b330 comp rx=0 tx=0).stop 2024-05-08T23:31:07.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.109+0000 7fceed65d700 1 -- 172.21.15.81:0/1550428633 shutdown_connections 2024-05-08T23:31:07.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.109+0000 7fceed65d700 1 --2- 172.21.15.81:0/1550428633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcee8162bc0 0x7fcee8164fb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.109+0000 7fceed65d700 1 --2- 172.21.15.81:0/1550428633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcee815bc40 0x7fcee815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.109+0000 7fceed65d700 1 -- 172.21.15.81:0/1550428633 >> 172.21.15.81:0/1550428633 conn(0x7fcee80717b0 msgr2=0x7fcee806a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:07.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.109+0000 7fceed65d700 1 -- 172.21.15.81:0/1550428633 shutdown_connections 2024-05-08T23:31:07.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.109+0000 7fceed65d700 1 -- 172.21.15.81:0/1550428633 wait complete. 2024-05-08T23:31:07.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.109+0000 7fceed65d700 1 Processor -- start 2024-05-08T23:31:07.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.109+0000 7fceed65d700 1 -- start start 2024-05-08T23:31:07.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceed65d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcee815b860 0x7fcee8151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceed65d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcee815bc40 0x7fcee8143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceed65d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcee8162bc0 0x7fcee81530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceed65d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee8152520 con 0x7fcee8162bc0 2024-05-08T23:31:07.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceed65d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee8148e70 con 0x7fcee815bc40 2024-05-08T23:31:07.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceed65d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee8149150 con 0x7fcee815b860 2024-05-08T23:31:07.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceece5c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcee8162bc0 0x7fcee81530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceece5c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcee8162bc0 0x7fcee81530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58254/0 (socket says 172.21.15.81:58254) 2024-05-08T23:31:07.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceece5c700 1 -- 172.21.15.81:0/1002255066 learned_addr learned my addr 172.21.15.81:0/1002255066 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:07.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fcee77fe700 1 --2- 172.21.15.81:0/1002255066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcee815bc40 0x7fcee8143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fcee7fff700 1 --2- 172.21.15.81:0/1002255066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcee815b860 0x7fcee8151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceece5c700 1 -- 172.21.15.81:0/1002255066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcee815b860 msgr2=0x7fcee8151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceece5c700 1 --2- 172.21.15.81:0/1002255066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcee815b860 0x7fcee8151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceece5c700 1 -- 172.21.15.81:0/1002255066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcee815bc40 msgr2=0x7fcee8143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceece5c700 1 --2- 172.21.15.81:0/1002255066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcee815bc40 0x7fcee8143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceece5c700 1 -- 172.21.15.81:0/1002255066 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee0009a70 con 0x7fcee8162bc0 2024-05-08T23:31:07.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fcee7fff700 1 --2- 172.21.15.81:0/1002255066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcee815b860 0x7fcee8151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:07.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceece5c700 1 --2- 172.21.15.81:0/1002255066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcee8162bc0 0x7fcee81530d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcee0013000 tx=0x7fcee0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:07.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fcee4ff9700 1 -- 172.21.15.81:0/1002255066 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0012430 con 0x7fcee8162bc0 2024-05-08T23:31:07.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fcee4ff9700 1 -- 172.21.15.81:0/1002255066 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee0020070 con 0x7fcee8162bc0 2024-05-08T23:31:07.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceed65d700 1 -- 172.21.15.81:0/1002255066 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee8149430 con 0x7fcee8162bc0 2024-05-08T23:31:07.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fcee4ff9700 1 -- 172.21.15.81:0/1002255066 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0004280 con 0x7fcee8162bc0 2024-05-08T23:31:07.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.113+0000 7fceed65d700 1 -- 172.21.15.81:0/1002255066 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee81499f0 con 0x7fcee8162bc0 2024-05-08T23:31:07.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.117+0000 7fcee4ff9700 1 -- 172.21.15.81:0/1002255066 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcee00043e0 con 0x7fcee8162bc0 2024-05-08T23:31:07.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.117+0000 7fceed65d700 1 -- 172.21.15.81:0/1002255066 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee80672a0 con 0x7fcee8162bc0 2024-05-08T23:31:07.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.117+0000 7fcee4ff9700 1 --2- 172.21.15.81:0/1002255066 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fced4066730 0x7fced4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.117+0000 7fcee7fff700 1 --2- 172.21.15.81:0/1002255066 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fced4066730 0x7fced4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.117+0000 7fcee4ff9700 1 -- 172.21.15.81:0/1002255066 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcee0022070 con 0x7fcee8162bc0 2024-05-08T23:31:07.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.117+0000 7fcee7fff700 1 --2- 172.21.15.81:0/1002255066 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fced4066730 0x7fced4068bf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fced8000f60 tx=0x7fced8007b00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:07.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.125+0000 7fcee4ff9700 1 -- 172.21.15.81:0/1002255066 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcee0053ac0 con 0x7fcee8162bc0 2024-05-08T23:31:07.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.297+0000 7fceed65d700 1 -- 172.21.15.81:0/1002255066 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcee814d2f0 con 0x7fcee8162bc0 2024-05-08T23:31:07.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.297+0000 7fcee4ff9700 1 -- 172.21.15.81:0/1002255066 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcee00570e0 con 0x7fcee8162bc0 2024-05-08T23:31:07.302 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:07.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.301+0000 7fceed65d700 1 -- 172.21.15.81:0/1002255066 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fced4066730 msgr2=0x7fced4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.301+0000 7fceed65d700 1 --2- 172.21.15.81:0/1002255066 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fced4066730 0x7fced4068bf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fced8000f60 tx=0x7fced8007b00 comp rx=0 tx=0).stop 2024-05-08T23:31:07.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.301+0000 7fceed65d700 1 -- 172.21.15.81:0/1002255066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcee8162bc0 msgr2=0x7fcee81530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.301+0000 7fceed65d700 1 --2- 172.21.15.81:0/1002255066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcee8162bc0 0x7fcee81530d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcee0013000 tx=0x7fcee0004af0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.301+0000 7fceed65d700 1 -- 172.21.15.81:0/1002255066 shutdown_connections 2024-05-08T23:31:07.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.301+0000 7fceed65d700 1 --2- 172.21.15.81:0/1002255066 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fced4066730 0x7fced4068bf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.301+0000 7fceed65d700 1 --2- 172.21.15.81:0/1002255066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcee8162bc0 0x7fcee81530d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.301+0000 7fceed65d700 1 --2- 172.21.15.81:0/1002255066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcee815bc40 0x7fcee8143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.301+0000 7fceed65d700 1 --2- 172.21.15.81:0/1002255066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcee815b860 0x7fcee8151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.301+0000 7fceed65d700 1 -- 172.21.15.81:0/1002255066 >> 172.21.15.81:0/1002255066 conn(0x7fcee80717b0 msgr2=0x7fcee81507e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:07.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.301+0000 7fceed65d700 1 -- 172.21.15.81:0/1002255066 shutdown_connections 2024-05-08T23:31:07.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.301+0000 7fceed65d700 1 -- 172.21.15.81:0/1002255066 wait complete. 2024-05-08T23:31:07.317 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:07.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.429+0000 7f3361885700 1 Processor -- start 2024-05-08T23:31:07.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.429+0000 7f3361885700 1 -- start start 2024-05-08T23:31:07.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3361885700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f33540e3b40 0x7f33540e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3361885700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3354006150 0x7f33540e4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3361885700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33540e49a0 con 0x7f33540e0250 2024-05-08T23:31:07.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3361885700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33540540b0 con 0x7f33540e3b40 2024-05-08T23:31:07.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3361885700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33540532f0 con 0x7f3354006150 2024-05-08T23:31:07.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335bfff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f33540e0250 0x7f3354038ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46704/0 (socket says 172.21.15.81:46704) 2024-05-08T23:31:07.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335bfff700 1 -- 172.21.15.81:0/684119524 learned_addr learned my addr 172.21.15.81:0/684119524 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:07.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3361084700 1 --2- 172.21.15.81:0/684119524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3354006150 0x7f33540e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3360883700 1 --2- 172.21.15.81:0/684119524 >> v2:172.21.15.117:3300/0 conn(0x7f33540e3b40 0x7f33540e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 -- 172.21.15.81:0/684119524 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111525020 0 0) 0x7f33540e49a0 con 0x7f33540e0250 2024-05-08T23:31:07.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 -- 172.21.15.81:0/684119524 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33440035d0 con 0x7f33540e0250 2024-05-08T23:31:07.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 -- 172.21.15.81:0/684119524 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3379641765 0 0) 0x7f33440035d0 con 0x7f33540e0250 2024-05-08T23:31:07.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 -- 172.21.15.81:0/684119524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3354006150 msgr2=0x7f33540e4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:07.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 --2- 172.21.15.81:0/684119524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3354006150 0x7f33540e4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 -- 172.21.15.81:0/684119524 >> v2:172.21.15.117:3300/0 conn(0x7f33540e3b40 msgr2=0x7f33540e3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:07.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 --2- 172.21.15.81:0/684119524 >> v2:172.21.15.117:3300/0 conn(0x7f33540e3b40 0x7f33540e3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 -- 172.21.15.81:0/684119524 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33540535d0 con 0x7f33540e0250 2024-05-08T23:31:07.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 -- 172.21.15.81:0/684119524 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f334c001670 con 0x7f33540e0250 2024-05-08T23:31:07.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 -- 172.21.15.81:0/684119524 >> v1:172.21.15.81:6789/0 conn(0x7f33540e0250 legacy=0x7f3354038ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 --2- 172.21.15.81:0/684119524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3354006150 0x7f3344003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 --2- 172.21.15.81:0/684119524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3344003f60 0x7f3344004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 --2- 172.21.15.81:0/684119524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3344004db0 0x7f3344005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 -- 172.21.15.81:0/684119524 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33540540b0 con 0x7f3344003f60 2024-05-08T23:31:07.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 -- 172.21.15.81:0/684119524 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33540532f0 con 0x7f3344004db0 2024-05-08T23:31:07.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 -- 172.21.15.81:0/684119524 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334400bab0 con 0x7f3354006150 2024-05-08T23:31:07.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335bfff700 1 --2- 172.21.15.81:0/684119524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3344003f60 0x7f3344004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3360883700 1 --2- 172.21.15.81:0/684119524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3354006150 0x7f3344003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3361084700 1 --2- 172.21.15.81:0/684119524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3344004db0 0x7f3344005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335bfff700 1 -- 172.21.15.81:0/684119524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3354006150 msgr2=0x7f3344003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:07.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335bfff700 1 --2- 172.21.15.81:0/684119524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3354006150 0x7f3344003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335bfff700 1 -- 172.21.15.81:0/684119524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3344004db0 msgr2=0x7f3344005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:07.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335bfff700 1 --2- 172.21.15.81:0/684119524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3344004db0 0x7f3344005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335bfff700 1 -- 172.21.15.81:0/684119524 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33440035d0 con 0x7f3344003f60 2024-05-08T23:31:07.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3361084700 1 --2- 172.21.15.81:0/684119524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3344004db0 0x7f3344005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:07.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335bfff700 1 --2- 172.21.15.81:0/684119524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3344003f60 0x7f3344004870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f334c0051b0 tx=0x7f33440058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7131a4af9fa12595 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:07.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 -- 172.21.15.81:0/684119524 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c005b80 con 0x7f3344003f60 2024-05-08T23:31:07.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 -- 172.21.15.81:0/684119524 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334c005ce0 con 0x7f3344003f60 2024-05-08T23:31:07.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f335b7fe700 1 -- 172.21.15.81:0/684119524 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c008390 con 0x7f3344003f60 2024-05-08T23:31:07.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3361885700 1 -- 172.21.15.81:0/684119524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3344003f60 msgr2=0x7f3344004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3361885700 1 --2- 172.21.15.81:0/684119524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3344003f60 0x7f3344004870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f334c0051b0 tx=0x7f33440058a0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3361084700 1 -- 172.21.15.81:0/684119524 reap_dead start 2024-05-08T23:31:07.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3361885700 1 -- 172.21.15.81:0/684119524 shutdown_connections 2024-05-08T23:31:07.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3361885700 1 -- 172.21.15.81:0/684119524 >> 172.21.15.81:0/684119524 conn(0x7f3354054b70 msgr2=0x7f3354054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:07.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3361885700 1 -- 172.21.15.81:0/684119524 shutdown_connections 2024-05-08T23:31:07.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.433+0000 7f3361885700 1 -- 172.21.15.81:0/684119524 wait complete. 2024-05-08T23:31:07.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3361885700 1 Processor -- start 2024-05-08T23:31:07.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3361885700 1 -- start start 2024-05-08T23:31:07.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3361885700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3344003f60 0x7f335404bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3361885700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33540e1a10 0x7f33540e24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3361885700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33540d8140 0x7f3354047950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3361885700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33540da940 con 0x7f3344003f60 2024-05-08T23:31:07.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3361885700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33540e29f0 con 0x7f33540e1a10 2024-05-08T23:31:07.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3361885700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33540e2c70 con 0x7f33540d8140 2024-05-08T23:31:07.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3360883700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3344003f60 0x7f335404bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3360883700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3344003f60 0x7f335404bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58278/0 (socket says 172.21.15.81:58278) 2024-05-08T23:31:07.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3360883700 1 -- 172.21.15.81:0/2282121242 learned_addr learned my addr 172.21.15.81:0/2282121242 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:07.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3361084700 1 --2- 172.21.15.81:0/2282121242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33540d8140 0x7f3354047950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f335bfff700 1 --2- 172.21.15.81:0/2282121242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33540e1a10 0x7f33540e24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3360883700 1 -- 172.21.15.81:0/2282121242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33540d8140 msgr2=0x7f3354047950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3360883700 1 --2- 172.21.15.81:0/2282121242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33540d8140 0x7f3354047950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3360883700 1 -- 172.21.15.81:0/2282121242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33540e1a10 msgr2=0x7f33540e24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3360883700 1 --2- 172.21.15.81:0/2282121242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33540e1a10 0x7f33540e24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3360883700 1 -- 172.21.15.81:0/2282121242 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33540e2f50 con 0x7f3344003f60 2024-05-08T23:31:07.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3360883700 1 --2- 172.21.15.81:0/2282121242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3344003f60 0x7f335404bc40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f335c04e9d0 tx=0x7f335c067c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:07.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f33597fa700 1 -- 172.21.15.81:0/2282121242 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c072790 con 0x7f3344003f60 2024-05-08T23:31:07.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f33597fa700 1 -- 172.21.15.81:0/2282121242 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335c0728f0 con 0x7f3344003f60 2024-05-08T23:31:07.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3361885700 1 -- 172.21.15.81:0/2282121242 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33540da5d0 con 0x7f3344003f60 2024-05-08T23:31:07.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f33597fa700 1 -- 172.21.15.81:0/2282121242 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c096660 con 0x7f3344003f60 2024-05-08T23:31:07.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.437+0000 7f3361885700 1 -- 172.21.15.81:0/2282121242 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33540da1f0 con 0x7f3344003f60 2024-05-08T23:31:07.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.441+0000 7f33597fa700 1 -- 172.21.15.81:0/2282121242 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f335c0967c0 con 0x7f3344003f60 2024-05-08T23:31:07.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.441+0000 7f3361885700 1 -- 172.21.15.81:0/2282121242 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33540da760 con 0x7f3344003f60 2024-05-08T23:31:07.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.441+0000 7f33597fa700 1 --2- 172.21.15.81:0/2282121242 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3348066730 0x7f3348068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.441+0000 7f335bfff700 1 --2- 172.21.15.81:0/2282121242 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3348066730 0x7f3348068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.441+0000 7f33597fa700 1 -- 172.21.15.81:0/2282121242 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f335c0fe730 con 0x7f3344003f60 2024-05-08T23:31:07.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.441+0000 7f335bfff700 1 --2- 172.21.15.81:0/2282121242 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3348066730 0x7f3348068bf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f334c0051b0 tx=0x7f334c00a070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:07.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.449+0000 7f33597fa700 1 -- 172.21.15.81:0/2282121242 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f335c0cb9f0 con 0x7f3344003f60 2024-05-08T23:31:07.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.617+0000 7f3361885700 1 -- 172.21.15.81:0/2282121242 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3354005440 con 0x7f3344003f60 2024-05-08T23:31:07.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.617+0000 7f33597fa700 1 -- 172.21.15.81:0/2282121242 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f335c0910a0 con 0x7f3344003f60 2024-05-08T23:31:07.622 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:07.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.617+0000 7f3361885700 1 -- 172.21.15.81:0/2282121242 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3348066730 msgr2=0x7f3348068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.617+0000 7f3361885700 1 --2- 172.21.15.81:0/2282121242 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3348066730 0x7f3348068bf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f334c0051b0 tx=0x7f334c00a070 comp rx=0 tx=0).stop 2024-05-08T23:31:07.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.617+0000 7f3361885700 1 -- 172.21.15.81:0/2282121242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3344003f60 msgr2=0x7f335404bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.617+0000 7f3361885700 1 --2- 172.21.15.81:0/2282121242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3344003f60 0x7f335404bc40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f335c04e9d0 tx=0x7f335c067c90 comp rx=0 tx=0).stop 2024-05-08T23:31:07.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.617+0000 7f3361885700 1 -- 172.21.15.81:0/2282121242 shutdown_connections 2024-05-08T23:31:07.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.617+0000 7f3361885700 1 --2- 172.21.15.81:0/2282121242 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3348066730 0x7f3348068bf0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.617+0000 7f3361885700 1 --2- 172.21.15.81:0/2282121242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33540d8140 0x7f3354047950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.617+0000 7f3361885700 1 --2- 172.21.15.81:0/2282121242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33540e1a10 0x7f33540e24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.617+0000 7f3361885700 1 --2- 172.21.15.81:0/2282121242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3344003f60 0x7f335404bc40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.617+0000 7f3361885700 1 -- 172.21.15.81:0/2282121242 >> 172.21.15.81:0/2282121242 conn(0x7f3354054b70 msgr2=0x7f335404f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:07.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.621+0000 7f3361885700 1 -- 172.21.15.81:0/2282121242 shutdown_connections 2024-05-08T23:31:07.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.621+0000 7f3361885700 1 -- 172.21.15.81:0/2282121242 wait complete. 2024-05-08T23:31:07.650 INFO:tasks.cephfs_test_runner:Starting test: test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) That a client who does not have the capability for setting 2024-05-08T23:31:07.650 INFO:tasks.cephfs_test_runner:test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-05-08T23:31:07.650 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:07.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2bedc8a700 1 Processor -- start 2024-05-08T23:31:07.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2bedc8a700 1 -- start start 2024-05-08T23:31:07.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2bedc8a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2be80ce510 0x7f2be815a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2bedc8a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2be815aaa0 0x7f2be813d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2bedc8a700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be80ce080 con 0x7f2be8071ab0 2024-05-08T23:31:07.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2bedc8a700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be80cd7c0 con 0x7f2be80ce510 2024-05-08T23:31:07.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2bedc8a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be80cda30 con 0x7f2be815aaa0 2024-05-08T23:31:07.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2be7fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2be80ce510 0x7f2be815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2be7fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2be80ce510 0x7f2be815a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58292/0 (socket says 172.21.15.81:58292) 2024-05-08T23:31:07.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2be7fff700 1 -- 172.21.15.81:0/1880258730 learned_addr learned my addr 172.21.15.81:0/1880258730 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:07.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2be7fff700 1 -- 172.21.15.81:0/1880258730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2be815aaa0 msgr2=0x7f2be813d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:07.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2bed489700 1 --2- 172.21.15.81:0/1880258730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2be815aaa0 0x7f2be813d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2be7fff700 1 --2- 172.21.15.81:0/1880258730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2be815aaa0 0x7f2be813d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2be7fff700 1 -- 172.21.15.81:0/1880258730 >> v1:172.21.15.117:6789/0 conn(0x7f2be8071ab0 legacy=0x7f2be8071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2be7fff700 1 -- 172.21.15.81:0/1880258730 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be80cdd10 con 0x7f2be80ce510 2024-05-08T23:31:07.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2bed489700 1 --2- 172.21.15.81:0/1880258730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2be815aaa0 0x7f2be813d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:07.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.761+0000 7f2be7fff700 1 --2- 172.21.15.81:0/1880258730 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2be80ce510 0x7f2be815a560 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800d150 tx=0x7f2bd800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f102aaae084974 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:07.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.765+0000 7f2be77fe700 1 -- 172.21.15.81:0/1880258730 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8010070 con 0x7f2be80ce510 2024-05-08T23:31:07.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.765+0000 7f2be77fe700 1 -- 172.21.15.81:0/1880258730 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd800ddc0 con 0x7f2be80ce510 2024-05-08T23:31:07.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.765+0000 7f2be77fe700 1 -- 172.21.15.81:0/1880258730 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd800b310 con 0x7f2be80ce510 2024-05-08T23:31:07.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.765+0000 7f2bedc8a700 1 -- 172.21.15.81:0/1880258730 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2be80ce510 msgr2=0x7f2be815a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.765+0000 7f2bedc8a700 1 --2- 172.21.15.81:0/1880258730 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2be80ce510 0x7f2be815a560 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800d150 tx=0x7f2bd800d460 comp rx=0 tx=0).stop 2024-05-08T23:31:07.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.765+0000 7f2bedc8a700 1 -- 172.21.15.81:0/1880258730 shutdown_connections 2024-05-08T23:31:07.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.765+0000 7f2bedc8a700 1 --2- 172.21.15.81:0/1880258730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2be815aaa0 0x7f2be813d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.765+0000 7f2bedc8a700 1 --2- 172.21.15.81:0/1880258730 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2be80ce510 0x7f2be815a560 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.765+0000 7f2bedc8a700 1 -- 172.21.15.81:0/1880258730 >> 172.21.15.81:0/1880258730 conn(0x7f2be806c330 msgr2=0x7f2be806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:07.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.765+0000 7f2bedc8a700 1 -- 172.21.15.81:0/1880258730 shutdown_connections 2024-05-08T23:31:07.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.765+0000 7f2bedc8a700 1 -- 172.21.15.81:0/1880258730 wait complete. 2024-05-08T23:31:07.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.765+0000 7f2bedc8a700 1 Processor -- start 2024-05-08T23:31:07.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.765+0000 7f2bedc8a700 1 -- start start 2024-05-08T23:31:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2bedc8a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2be8071ab0 0x7f2be81328d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2bedc8a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2be80ce510 0x7f2be8158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2bedc8a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2be815aaa0 0x7f2be81593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2bedc8a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be8142710 con 0x7f2be8071ab0 2024-05-08T23:31:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2bedc8a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be81429f0 con 0x7f2be815aaa0 2024-05-08T23:31:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2bedc8a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be8142cd0 con 0x7f2be80ce510 2024-05-08T23:31:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2becc88700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2be8071ab0 0x7f2be81328d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2becc88700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2be8071ab0 0x7f2be81328d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58302/0 (socket says 172.21.15.81:58302) 2024-05-08T23:31:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2becc88700 1 -- 172.21.15.81:0/386580231 learned_addr learned my addr 172.21.15.81:0/386580231 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2be7fff700 1 --2- 172.21.15.81:0/386580231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2be80ce510 0x7f2be8158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2bed489700 1 --2- 172.21.15.81:0/386580231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2be815aaa0 0x7f2be81593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2becc88700 1 -- 172.21.15.81:0/386580231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2be80ce510 msgr2=0x7f2be8158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2becc88700 1 --2- 172.21.15.81:0/386580231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2be80ce510 0x7f2be8158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2becc88700 1 -- 172.21.15.81:0/386580231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2be815aaa0 msgr2=0x7f2be81593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2becc88700 1 --2- 172.21.15.81:0/386580231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2be815aaa0 0x7f2be81593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2becc88700 1 -- 172.21.15.81:0/386580231 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd800bd40 con 0x7f2be8071ab0 2024-05-08T23:31:07.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2becc88700 1 --2- 172.21.15.81:0/386580231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2be8071ab0 0x7f2be81328d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000c500 tx=0x7f2bd000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:07.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2be57fa700 1 -- 172.21.15.81:0/386580231 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00052d0 con 0x7f2be8071ab0 2024-05-08T23:31:07.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2be57fa700 1 -- 172.21.15.81:0/386580231 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd0005430 con 0x7f2be8071ab0 2024-05-08T23:31:07.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2bedc8a700 1 -- 172.21.15.81:0/386580231 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be8142fb0 con 0x7f2be8071ab0 2024-05-08T23:31:07.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2be57fa700 1 -- 172.21.15.81:0/386580231 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00055a0 con 0x7f2be8071ab0 2024-05-08T23:31:07.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2bedc8a700 1 -- 172.21.15.81:0/386580231 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be8132e10 con 0x7f2be8071ab0 2024-05-08T23:31:07.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.769+0000 7f2bedc8a700 1 -- 172.21.15.81:0/386580231 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be81434d0 con 0x7f2be8071ab0 2024-05-08T23:31:07.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.773+0000 7f2be57fa700 1 -- 172.21.15.81:0/386580231 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2bd00038e0 con 0x7f2be8071ab0 2024-05-08T23:31:07.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.773+0000 7f2be57fa700 1 --2- 172.21.15.81:0/386580231 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2bc8066730 0x7f2bc8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:07.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.773+0000 7f2be57fa700 1 -- 172.21.15.81:0/386580231 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bd0083470 con 0x7f2be8071ab0 2024-05-08T23:31:07.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.773+0000 7f2be7fff700 1 --2- 172.21.15.81:0/386580231 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2bc8066730 0x7f2bc8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:07.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.773+0000 7f2be7fff700 1 --2- 172.21.15.81:0/386580231 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2bc8066730 0x7f2bc8068bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8000f80 tx=0x7f2bd800b000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:07.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.777+0000 7f2be57fa700 1 -- 172.21.15.81:0/386580231 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2be81434d0 con 0x7f2be8071ab0 2024-05-08T23:31:07.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.949+0000 7f2bedc8a700 1 -- 172.21.15.81:0/386580231 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2be81434d0 con 0x7f2be8071ab0 2024-05-08T23:31:07.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.949+0000 7f2be57fa700 1 -- 172.21.15.81:0/386580231 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f2bd0050870 con 0x7f2be8071ab0 2024-05-08T23:31:07.955 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:07.955 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:07.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.953+0000 7f2bedc8a700 1 -- 172.21.15.81:0/386580231 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2bc8066730 msgr2=0x7f2bc8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.953+0000 7f2bedc8a700 1 --2- 172.21.15.81:0/386580231 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2bc8066730 0x7f2bc8068bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8000f80 tx=0x7f2bd800b000 comp rx=0 tx=0).stop 2024-05-08T23:31:07.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.953+0000 7f2bedc8a700 1 -- 172.21.15.81:0/386580231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2be8071ab0 msgr2=0x7f2be81328d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:07.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.953+0000 7f2bedc8a700 1 --2- 172.21.15.81:0/386580231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2be8071ab0 0x7f2be81328d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000c500 tx=0x7f2bd000c810 comp rx=0 tx=0).stop 2024-05-08T23:31:07.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.953+0000 7f2bedc8a700 1 -- 172.21.15.81:0/386580231 shutdown_connections 2024-05-08T23:31:07.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.953+0000 7f2bedc8a700 1 --2- 172.21.15.81:0/386580231 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2bc8066730 0x7f2bc8068bf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.953+0000 7f2bedc8a700 1 --2- 172.21.15.81:0/386580231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2be815aaa0 0x7f2be81593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.953+0000 7f2bedc8a700 1 --2- 172.21.15.81:0/386580231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2be80ce510 0x7f2be8158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.953+0000 7f2bedc8a700 1 --2- 172.21.15.81:0/386580231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2be8071ab0 0x7f2be81328d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:07.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.953+0000 7f2bedc8a700 1 -- 172.21.15.81:0/386580231 >> 172.21.15.81:0/386580231 conn(0x7f2be806c330 msgr2=0x7f2be813e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:07.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.953+0000 7f2bedc8a700 1 -- 172.21.15.81:0/386580231 shutdown_connections 2024-05-08T23:31:07.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:07.953+0000 7f2bedc8a700 1 -- 172.21.15.81:0/386580231 wait complete. 2024-05-08T23:31:07.971 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:07.971 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:08.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc8406bc700 1 Processor -- start 2024-05-08T23:31:08.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc8406bc700 1 -- start start 2024-05-08T23:31:08.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc8406bc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8381478c0 0x7fc838165240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc8406bc700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc838147d20 con 0x7fc8380cb3b0 2024-05-08T23:31:08.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc8406bc700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc838165970 con 0x7fc838144d40 2024-05-08T23:31:08.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc8406bc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc838161c50 con 0x7fc8381478c0 2024-05-08T23:31:08.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc83e458700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc8380cb3b0 0x7fc838162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46714/0 (socket says 172.21.15.81:46714) 2024-05-08T23:31:08.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc83e458700 1 -- 172.21.15.81:0/108610959 learned_addr learned my addr 172.21.15.81:0/108610959 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:08.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc83dc57700 1 --2- 172.21.15.81:0/108610959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8381478c0 0x7fc838165240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc83d456700 1 -- 172.21.15.81:0/108610959 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512124796 0 0) 0x7fc828001110 con 0x7fc8380cb3b0 2024-05-08T23:31:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc83d456700 1 -- 172.21.15.81:0/108610959 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8240035d0 con 0x7fc8380cb3b0 2024-05-08T23:31:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc83dc57700 1 -- 172.21.15.81:0/108610959 >> v1:172.21.15.117:6789/0 conn(0x7fc838144d40 legacy=0x7fc83806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc83dc57700 1 -- 172.21.15.81:0/108610959 >> v1:172.21.15.81:6789/0 conn(0x7fc8380cb3b0 legacy=0x7fc838162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc83dc57700 1 -- 172.21.15.81:0/108610959 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc838165c50 con 0x7fc8381478c0 2024-05-08T23:31:08.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc83dc57700 1 --2- 172.21.15.81:0/108610959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8381478c0 0x7fc838165240 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc834009880 tx=0x7fc834009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1239b5efccc8bbb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:08.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc83d456700 1 -- 172.21.15.81:0/108610959 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834011830 con 0x7fc8381478c0 2024-05-08T23:31:08.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc83d456700 1 -- 172.21.15.81:0/108610959 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc834011990 con 0x7fc8381478c0 2024-05-08T23:31:08.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.089+0000 7fc83d456700 1 -- 172.21.15.81:0/108610959 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834011d00 con 0x7fc8381478c0 2024-05-08T23:31:08.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.093+0000 7fc8406bc700 1 -- 172.21.15.81:0/108610959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8381478c0 msgr2=0x7fc838165240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.093+0000 7fc8406bc700 1 --2- 172.21.15.81:0/108610959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8381478c0 0x7fc838165240 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc834009880 tx=0x7fc834009b90 comp rx=0 tx=0).stop 2024-05-08T23:31:08.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.093+0000 7fc8406bc700 1 -- 172.21.15.81:0/108610959 shutdown_connections 2024-05-08T23:31:08.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.093+0000 7fc8406bc700 1 --2- 172.21.15.81:0/108610959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8381478c0 0x7fc838165240 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.093+0000 7fc8406bc700 1 -- 172.21.15.81:0/108610959 >> 172.21.15.81:0/108610959 conn(0x7fc83806a360 msgr2=0x7fc83806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:08.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.093+0000 7fc8406bc700 1 -- 172.21.15.81:0/108610959 shutdown_connections 2024-05-08T23:31:08.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.093+0000 7fc8406bc700 1 -- 172.21.15.81:0/108610959 wait complete. 2024-05-08T23:31:08.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc8406bc700 1 Processor -- start 2024-05-08T23:31:08.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc8406bc700 1 -- start start 2024-05-08T23:31:08.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc8406bc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8380cb3b0 0x7fc83815bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc8406bc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc838144d40 0x7fc838157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc8406bc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8381478c0 0x7fc83815c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc8406bc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc838159e20 con 0x7fc8381478c0 2024-05-08T23:31:08.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc8406bc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83815c850 con 0x7fc838144d40 2024-05-08T23:31:08.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc8406bc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83815a0c0 con 0x7fc8380cb3b0 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc83ec59700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8381478c0 0x7fc83815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc83e458700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8380cb3b0 0x7fc83815bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc83ec59700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8381478c0 0x7fc83815c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58312/0 (socket says 172.21.15.81:58312) 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc83ec59700 1 -- 172.21.15.81:0/4061063242 learned_addr learned my addr 172.21.15.81:0/4061063242 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc83dc57700 1 --2- 172.21.15.81:0/4061063242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc838144d40 0x7fc838157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc83ec59700 1 -- 172.21.15.81:0/4061063242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8380cb3b0 msgr2=0x7fc83815bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc83ec59700 1 --2- 172.21.15.81:0/4061063242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8380cb3b0 0x7fc83815bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc83ec59700 1 -- 172.21.15.81:0/4061063242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc838144d40 msgr2=0x7fc838157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc83ec59700 1 --2- 172.21.15.81:0/4061063242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc838144d40 0x7fc838157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc83ec59700 1 -- 172.21.15.81:0/4061063242 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8340094b0 con 0x7fc8381478c0 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc83ec59700 1 --2- 172.21.15.81:0/4061063242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8381478c0 0x7fc83815c310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc83000cc50 tx=0x7fc83000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc82effd700 1 -- 172.21.15.81:0/4061063242 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc830003c20 con 0x7fc8381478c0 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc82effd700 1 -- 172.21.15.81:0/4061063242 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc830003d80 con 0x7fc8381478c0 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc8406bc700 1 -- 172.21.15.81:0/4061063242 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83815a3a0 con 0x7fc8381478c0 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc82effd700 1 -- 172.21.15.81:0/4061063242 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83000aab0 con 0x7fc8381478c0 2024-05-08T23:31:08.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.097+0000 7fc8406bc700 1 -- 172.21.15.81:0/4061063242 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83815a960 con 0x7fc8381478c0 2024-05-08T23:31:08.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.101+0000 7fc8406bc700 1 -- 172.21.15.81:0/4061063242 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83815ab50 con 0x7fc8381478c0 2024-05-08T23:31:08.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.101+0000 7fc82effd700 1 -- 172.21.15.81:0/4061063242 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc83000ac10 con 0x7fc8381478c0 2024-05-08T23:31:08.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.101+0000 7fc82effd700 1 --2- 172.21.15.81:0/4061063242 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc81c066730 0x7fc81c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.101+0000 7fc82effd700 1 -- 172.21.15.81:0/4061063242 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8300841c0 con 0x7fc8381478c0 2024-05-08T23:31:08.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.101+0000 7fc83e458700 1 --2- 172.21.15.81:0/4061063242 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc81c066730 0x7fc81c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.101+0000 7fc83e458700 1 --2- 172.21.15.81:0/4061063242 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc81c066730 0x7fc81c068bf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc828007fd0 tx=0x7fc828007c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:08.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.109+0000 7fc82effd700 1 -- 172.21.15.81:0/4061063242 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc8300514b0 con 0x7fc8381478c0 2024-05-08T23:31:08.287 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.281+0000 7fc8406bc700 1 -- 172.21.15.81:0/4061063242 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8380684b0 con 0x7fc8381478c0 2024-05-08T23:31:08.288 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.285+0000 7fc82effd700 1 -- 172.21.15.81:0/4061063242 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc830054ad0 con 0x7fc8381478c0 2024-05-08T23:31:08.289 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:08.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.285+0000 7fc8406bc700 1 -- 172.21.15.81:0/4061063242 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc81c066730 msgr2=0x7fc81c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.285+0000 7fc8406bc700 1 --2- 172.21.15.81:0/4061063242 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc81c066730 0x7fc81c068bf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc828007fd0 tx=0x7fc828007c60 comp rx=0 tx=0).stop 2024-05-08T23:31:08.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.285+0000 7fc8406bc700 1 -- 172.21.15.81:0/4061063242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8381478c0 msgr2=0x7fc83815c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.285+0000 7fc8406bc700 1 --2- 172.21.15.81:0/4061063242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8381478c0 0x7fc83815c310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc83000cc50 tx=0x7fc83000cf60 comp rx=0 tx=0).stop 2024-05-08T23:31:08.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.285+0000 7fc8406bc700 1 -- 172.21.15.81:0/4061063242 shutdown_connections 2024-05-08T23:31:08.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.285+0000 7fc8406bc700 1 --2- 172.21.15.81:0/4061063242 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc81c066730 0x7fc81c068bf0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.285+0000 7fc8406bc700 1 --2- 172.21.15.81:0/4061063242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8381478c0 0x7fc83815c310 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.285+0000 7fc8406bc700 1 --2- 172.21.15.81:0/4061063242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc838144d40 0x7fc838157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.285+0000 7fc8406bc700 1 --2- 172.21.15.81:0/4061063242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8380cb3b0 0x7fc83815bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.285+0000 7fc8406bc700 1 -- 172.21.15.81:0/4061063242 >> 172.21.15.81:0/4061063242 conn(0x7fc83806a360 msgr2=0x7fc8380cdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:08.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.285+0000 7fc8406bc700 1 -- 172.21.15.81:0/4061063242 shutdown_connections 2024-05-08T23:31:08.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.285+0000 7fc8406bc700 1 -- 172.21.15.81:0/4061063242 wait complete. 2024-05-08T23:31:08.303 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:08.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.417+0000 7fa0f9265700 1 Processor -- start 2024-05-08T23:31:08.424 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0f9265700 1 -- start start 2024-05-08T23:31:08.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0f9265700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa0f4160920 0x7fa0f40ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0f9265700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0f40d1da0 0x7fa0f40cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0f9265700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f40d3d40 con 0x7fa0f40cec30 2024-05-08T23:31:08.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0f9265700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f40d0d00 con 0x7fa0f4160920 2024-05-08T23:31:08.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0f9265700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f40d0fe0 con 0x7fa0f40d1da0 2024-05-08T23:31:08.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0f8a64700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa0f40cec30 0x7fa0f4161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46728/0 (socket says 172.21.15.81:46728) 2024-05-08T23:31:08.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0f8a64700 1 -- 172.21.15.81:0/1641614610 learned_addr learned my addr 172.21.15.81:0/1641614610 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:08.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0f3fff700 1 --2- 172.21.15.81:0/1641614610 >> v2:172.21.15.117:3300/0 conn(0x7fa0f4160920 0x7fa0f40ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0eb7fe700 1 --2- 172.21.15.81:0/1641614610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0f40d1da0 0x7fa0f40cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0f37fe700 1 -- 172.21.15.81:0/1641614610 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25080850 0 0) 0x7fa0f40d3d40 con 0x7fa0f40cec30 2024-05-08T23:31:08.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0f37fe700 1 -- 172.21.15.81:0/1641614610 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0e00035d0 con 0x7fa0f40cec30 2024-05-08T23:31:08.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0eb7fe700 1 -- 172.21.15.81:0/1641614610 >> v2:172.21.15.117:3300/0 conn(0x7fa0f4160920 msgr2=0x7fa0f40ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:08.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0eb7fe700 1 --2- 172.21.15.81:0/1641614610 >> v2:172.21.15.117:3300/0 conn(0x7fa0f4160920 0x7fa0f40ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0eb7fe700 1 -- 172.21.15.81:0/1641614610 >> v1:172.21.15.81:6789/0 conn(0x7fa0f40cec30 legacy=0x7fa0f4161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0eb7fe700 1 -- 172.21.15.81:0/1641614610 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f40d12c0 con 0x7fa0f40d1da0 2024-05-08T23:31:08.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0eb7fe700 1 --2- 172.21.15.81:0/1641614610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0f40d1da0 0x7fa0f40cf940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009850 tx=0x7fa0dc009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa5b97279cd5a3da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:08.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0f37fe700 1 -- 172.21.15.81:0/1641614610 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc011830 con 0x7fa0f40d1da0 2024-05-08T23:31:08.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0f37fe700 1 -- 172.21.15.81:0/1641614610 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0dc011990 con 0x7fa0f40d1da0 2024-05-08T23:31:08.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.421+0000 7fa0f37fe700 1 -- 172.21.15.81:0/1641614610 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc011d00 con 0x7fa0f40d1da0 2024-05-08T23:31:08.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.425+0000 7fa0f9265700 1 -- 172.21.15.81:0/1641614610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0f40d1da0 msgr2=0x7fa0f40cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.425+0000 7fa0f9265700 1 --2- 172.21.15.81:0/1641614610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0f40d1da0 0x7fa0f40cf940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009850 tx=0x7fa0dc009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:08.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.425+0000 7fa0f9265700 1 -- 172.21.15.81:0/1641614610 shutdown_connections 2024-05-08T23:31:08.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.425+0000 7fa0f9265700 1 --2- 172.21.15.81:0/1641614610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0f40d1da0 0x7fa0f40cf940 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.425+0000 7fa0f9265700 1 --2- 172.21.15.81:0/1641614610 >> v2:172.21.15.117:3300/0 conn(0x7fa0f4160920 0x7fa0f40ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.425+0000 7fa0f9265700 1 -- 172.21.15.81:0/1641614610 >> 172.21.15.81:0/1641614610 conn(0x7fa0f409c340 msgr2=0x7fa0f40d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:08.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.425+0000 7fa0f9265700 1 -- 172.21.15.81:0/1641614610 shutdown_connections 2024-05-08T23:31:08.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.425+0000 7fa0f9265700 1 -- 172.21.15.81:0/1641614610 wait complete. 2024-05-08T23:31:08.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.425+0000 7fa0f9265700 1 Processor -- start 2024-05-08T23:31:08.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.425+0000 7fa0f9265700 1 -- start start 2024-05-08T23:31:08.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f9265700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f40cec30 0x7fa0f4152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f9265700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0f40d1da0 0x7fa0f409d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f9265700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0f4160920 0x7fa0f409fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f9265700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f409e7c0 con 0x7fa0f40cec30 2024-05-08T23:31:08.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f9265700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f409dbb0 con 0x7fa0f40d1da0 2024-05-08T23:31:08.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f9265700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f409de90 con 0x7fa0f4160920 2024-05-08T23:31:08.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f40cec30 0x7fa0f4152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f40cec30 0x7fa0f4152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58322/0 (socket says 172.21.15.81:58322) 2024-05-08T23:31:08.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f3fff700 1 -- 172.21.15.81:0/1617206794 learned_addr learned my addr 172.21.15.81:0/1617206794 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:08.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0eb7fe700 1 --2- 172.21.15.81:0/1617206794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0f40d1da0 0x7fa0f409d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f8a64700 1 --2- 172.21.15.81:0/1617206794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0f4160920 0x7fa0f409fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f3fff700 1 -- 172.21.15.81:0/1617206794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0f4160920 msgr2=0x7fa0f409fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f3fff700 1 --2- 172.21.15.81:0/1617206794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0f4160920 0x7fa0f409fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f3fff700 1 -- 172.21.15.81:0/1617206794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0f40d1da0 msgr2=0x7fa0f409d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f3fff700 1 --2- 172.21.15.81:0/1617206794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0f40d1da0 0x7fa0f409d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f3fff700 1 -- 172.21.15.81:0/1617206794 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0dc0094b0 con 0x7fa0f40cec30 2024-05-08T23:31:08.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f3fff700 1 --2- 172.21.15.81:0/1617206794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f40cec30 0x7fa0f4152260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400e840 tx=0x7fa0e400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:08.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f17fa700 1 -- 172.21.15.81:0/1617206794 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40095a0 con 0x7fa0f40cec30 2024-05-08T23:31:08.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f17fa700 1 -- 172.21.15.81:0/1617206794 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e4013680 con 0x7fa0f40cec30 2024-05-08T23:31:08.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f9265700 1 -- 172.21.15.81:0/1617206794 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f4160e80 con 0x7fa0f40cec30 2024-05-08T23:31:08.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f17fa700 1 -- 172.21.15.81:0/1617206794 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40139f0 con 0x7fa0f40cec30 2024-05-08T23:31:08.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.429+0000 7fa0f9265700 1 -- 172.21.15.81:0/1617206794 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f409e450 con 0x7fa0f40cec30 2024-05-08T23:31:08.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.433+0000 7fa0f17fa700 1 -- 172.21.15.81:0/1617206794 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa0e4013b50 con 0x7fa0f40cec30 2024-05-08T23:31:08.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.433+0000 7fa0f9265700 1 -- 172.21.15.81:0/1617206794 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f40684b0 con 0x7fa0f40cec30 2024-05-08T23:31:08.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.433+0000 7fa0f17fa700 1 --2- 172.21.15.81:0/1617206794 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa0d4066730 0x7fa0d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.433+0000 7fa0f17fa700 1 -- 172.21.15.81:0/1617206794 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0e40841d0 con 0x7fa0f40cec30 2024-05-08T23:31:08.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.433+0000 7fa0eb7fe700 1 --2- 172.21.15.81:0/1617206794 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa0d4066730 0x7fa0d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.433+0000 7fa0eb7fe700 1 --2- 172.21.15.81:0/1617206794 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa0d4066730 0x7fa0d4068bf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc0090b0 tx=0x7fa0dc01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:08.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.437+0000 7fa0f17fa700 1 -- 172.21.15.81:0/1617206794 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa0e4051410 con 0x7fa0f40cec30 2024-05-08T23:31:08.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.609+0000 7fa0f9265700 1 -- 172.21.15.81:0/1617206794 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0f40cf510 con 0x7fa0f40cec30 2024-05-08T23:31:08.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.609+0000 7fa0f17fa700 1 -- 172.21.15.81:0/1617206794 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa0e4054a30 con 0x7fa0f40cec30 2024-05-08T23:31:08.612 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:08.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.613+0000 7fa0f9265700 1 -- 172.21.15.81:0/1617206794 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa0d4066730 msgr2=0x7fa0d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.613+0000 7fa0f9265700 1 --2- 172.21.15.81:0/1617206794 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa0d4066730 0x7fa0d4068bf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc0090b0 tx=0x7fa0dc01c040 comp rx=0 tx=0).stop 2024-05-08T23:31:08.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.613+0000 7fa0f9265700 1 -- 172.21.15.81:0/1617206794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f40cec30 msgr2=0x7fa0f4152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.613+0000 7fa0f9265700 1 --2- 172.21.15.81:0/1617206794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f40cec30 0x7fa0f4152260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400e840 tx=0x7fa0e400eb50 comp rx=0 tx=0).stop 2024-05-08T23:31:08.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.613+0000 7fa0f9265700 1 -- 172.21.15.81:0/1617206794 shutdown_connections 2024-05-08T23:31:08.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.613+0000 7fa0f9265700 1 --2- 172.21.15.81:0/1617206794 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa0d4066730 0x7fa0d4068bf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.613+0000 7fa0f9265700 1 --2- 172.21.15.81:0/1617206794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0f4160920 0x7fa0f409fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.613+0000 7fa0f9265700 1 --2- 172.21.15.81:0/1617206794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0f40d1da0 0x7fa0f409d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.613+0000 7fa0f9265700 1 --2- 172.21.15.81:0/1617206794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f40cec30 0x7fa0f4152260 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.613+0000 7fa0f9265700 1 -- 172.21.15.81:0/1617206794 >> 172.21.15.81:0/1617206794 conn(0x7fa0f409c340 msgr2=0x7fa0f415edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:08.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.613+0000 7fa0f9265700 1 -- 172.21.15.81:0/1617206794 shutdown_connections 2024-05-08T23:31:08.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.613+0000 7fa0f9265700 1 -- 172.21.15.81:0/1617206794 wait complete. 2024-05-08T23:31:08.634 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:08.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f82964d6700 1 Processor -- start 2024-05-08T23:31:08.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f82964d6700 1 -- start start 2024-05-08T23:31:08.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f82964d6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f829015d2f0 0x7f82900ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f82964d6700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f82900d1bc0 0x7f82900cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f82964d6700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82900ceb40 con 0x7f82900cf8b0 2024-05-08T23:31:08.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f82964d6700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82900d3a10 con 0x7f82900d1bc0 2024-05-08T23:31:08.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f82964d6700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829015d6d0 con 0x7f829015d2f0 2024-05-08T23:31:08.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f8295cd5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f82900d1bc0 0x7f82900cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f8295cd5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f82900d1bc0 0x7f82900cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58332/0 (socket says 172.21.15.81:58332) 2024-05-08T23:31:08.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f8295cd5700 1 -- 172.21.15.81:0/2463646837 learned_addr learned my addr 172.21.15.81:0/2463646837 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:08.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f82954d4700 1 --2- 172.21.15.81:0/2463646837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f829015d2f0 0x7f82900ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f8295cd5700 1 -- 172.21.15.81:0/2463646837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f829015d2f0 msgr2=0x7f82900ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:08.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f8295cd5700 1 --2- 172.21.15.81:0/2463646837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f829015d2f0 0x7f82900ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f8295cd5700 1 -- 172.21.15.81:0/2463646837 >> v1:172.21.15.117:6789/0 conn(0x7f82900cf8b0 legacy=0x7f82900b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f8295cd5700 1 -- 172.21.15.81:0/2463646837 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82900d0250 con 0x7f82900d1bc0 2024-05-08T23:31:08.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.749+0000 7f8295cd5700 1 --2- 172.21.15.81:0/2463646837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82900d1bc0 0x7f82900cfd10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8288009e50 tx=0x7f828800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b17cd56c3012a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:08.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.753+0000 7f8287fff700 1 -- 172.21.15.81:0/2463646837 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828800e070 con 0x7f82900d1bc0 2024-05-08T23:31:08.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.753+0000 7f8287fff700 1 -- 172.21.15.81:0/2463646837 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828800b910 con 0x7f82900d1bc0 2024-05-08T23:31:08.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.753+0000 7f8287fff700 1 -- 172.21.15.81:0/2463646837 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828800bcc0 con 0x7f82900d1bc0 2024-05-08T23:31:08.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.753+0000 7f82964d6700 1 -- 172.21.15.81:0/2463646837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82900d1bc0 msgr2=0x7f82900cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.753+0000 7f82964d6700 1 --2- 172.21.15.81:0/2463646837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82900d1bc0 0x7f82900cfd10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8288009e50 tx=0x7f828800b330 comp rx=0 tx=0).stop 2024-05-08T23:31:08.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.753+0000 7f82964d6700 1 -- 172.21.15.81:0/2463646837 shutdown_connections 2024-05-08T23:31:08.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.753+0000 7f82964d6700 1 --2- 172.21.15.81:0/2463646837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82900d1bc0 0x7f82900cfd10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.753+0000 7f82964d6700 1 --2- 172.21.15.81:0/2463646837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f829015d2f0 0x7f82900ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.753+0000 7f82964d6700 1 -- 172.21.15.81:0/2463646837 >> 172.21.15.81:0/2463646837 conn(0x7f829009c320 msgr2=0x7f82900d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:08.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.753+0000 7f82964d6700 1 -- 172.21.15.81:0/2463646837 shutdown_connections 2024-05-08T23:31:08.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.753+0000 7f82964d6700 1 -- 172.21.15.81:0/2463646837 wait complete. 2024-05-08T23:31:08.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.753+0000 7f82964d6700 1 Processor -- start 2024-05-08T23:31:08.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.753+0000 7f82964d6700 1 -- start start 2024-05-08T23:31:08.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f82964d6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82900cf8b0 0x7f82901f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f82964d6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82900d1bc0 0x7f82901f2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f82964d6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f829015d2f0 0x7f82900c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f82964d6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82900cd340 con 0x7f82900d1bc0 2024-05-08T23:31:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f82964d6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82900ccd80 con 0x7f82900cf8b0 2024-05-08T23:31:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f82964d6700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82900cd060 con 0x7f829015d2f0 2024-05-08T23:31:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f8294cd3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82900d1bc0 0x7f82901f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f8294cd3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82900d1bc0 0x7f82901f2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58336/0 (socket says 172.21.15.81:58336) 2024-05-08T23:31:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f82954d4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82900cf8b0 0x7f82901f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f8294cd3700 1 -- 172.21.15.81:0/3296277593 learned_addr learned my addr 172.21.15.81:0/3296277593 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f8295cd5700 1 --2- 172.21.15.81:0/3296277593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f829015d2f0 0x7f82900c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f8294cd3700 1 -- 172.21.15.81:0/3296277593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f829015d2f0 msgr2=0x7f82900c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f8294cd3700 1 --2- 172.21.15.81:0/3296277593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f829015d2f0 0x7f82900c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f8294cd3700 1 -- 172.21.15.81:0/3296277593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82900cf8b0 msgr2=0x7f82901f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f8294cd3700 1 --2- 172.21.15.81:0/3296277593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82900cf8b0 0x7f82901f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f8294cd3700 1 -- 172.21.15.81:0/3296277593 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8288009b30 con 0x7f82900d1bc0 2024-05-08T23:31:08.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f8294cd3700 1 --2- 172.21.15.81:0/3296277593 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82900d1bc0 0x7f82901f2510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f828c00c500 tx=0x7f828c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:08.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f8285ffb700 1 -- 172.21.15.81:0/3296277593 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c0052d0 con 0x7f82900d1bc0 2024-05-08T23:31:08.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f8285ffb700 1 -- 172.21.15.81:0/3296277593 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c005430 con 0x7f82900d1bc0 2024-05-08T23:31:08.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f82964d6700 1 -- 172.21.15.81:0/3296277593 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82900b5da0 con 0x7f82900d1bc0 2024-05-08T23:31:08.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f8285ffb700 1 -- 172.21.15.81:0/3296277593 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c0055a0 con 0x7f82900d1bc0 2024-05-08T23:31:08.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f82964d6700 1 -- 172.21.15.81:0/3296277593 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82901f2a50 con 0x7f82900d1bc0 2024-05-08T23:31:08.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.757+0000 7f82964d6700 1 -- 172.21.15.81:0/3296277593 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82900b5c00 con 0x7f82900d1bc0 2024-05-08T23:31:08.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.761+0000 7f8285ffb700 1 -- 172.21.15.81:0/3296277593 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f828c0038e0 con 0x7f82900d1bc0 2024-05-08T23:31:08.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.761+0000 7f8285ffb700 1 --2- 172.21.15.81:0/3296277593 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8274066760 0x7f8274068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:08.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.761+0000 7f82954d4700 1 --2- 172.21.15.81:0/3296277593 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8274066760 0x7f8274068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:08.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.761+0000 7f8285ffb700 1 -- 172.21.15.81:0/3296277593 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f828c083810 con 0x7f82900d1bc0 2024-05-08T23:31:08.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.761+0000 7f82954d4700 1 --2- 172.21.15.81:0/3296277593 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8274066760 0x7f8274068c20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8280009bf0 tx=0x7f8280009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:08.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.765+0000 7f8285ffb700 1 -- 172.21.15.81:0/3296277593 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f828c050ad0 con 0x7f82900d1bc0 2024-05-08T23:31:08.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.937+0000 7f82964d6700 1 -- 172.21.15.81:0/3296277593 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82900b62c0 con 0x7f82900d1bc0 2024-05-08T23:31:08.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.937+0000 7f8285ffb700 1 -- 172.21.15.81:0/3296277593 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f82900b62c0 con 0x7f82900d1bc0 2024-05-08T23:31:08.940 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:08.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.937+0000 7f82964d6700 1 -- 172.21.15.81:0/3296277593 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8274066760 msgr2=0x7f8274068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.937+0000 7f82964d6700 1 --2- 172.21.15.81:0/3296277593 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8274066760 0x7f8274068c20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8280009bf0 tx=0x7f8280009240 comp rx=0 tx=0).stop 2024-05-08T23:31:08.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.937+0000 7f82964d6700 1 -- 172.21.15.81:0/3296277593 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82900d1bc0 msgr2=0x7f82901f2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:08.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.937+0000 7f82964d6700 1 --2- 172.21.15.81:0/3296277593 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82900d1bc0 0x7f82901f2510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f828c00c500 tx=0x7f828c00c810 comp rx=0 tx=0).stop 2024-05-08T23:31:08.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.937+0000 7f82964d6700 1 -- 172.21.15.81:0/3296277593 shutdown_connections 2024-05-08T23:31:08.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.937+0000 7f82964d6700 1 --2- 172.21.15.81:0/3296277593 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8274066760 0x7f8274068c20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.937+0000 7f82964d6700 1 --2- 172.21.15.81:0/3296277593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f829015d2f0 0x7f82900c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.937+0000 7f82964d6700 1 --2- 172.21.15.81:0/3296277593 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82900d1bc0 0x7f82901f2510 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.937+0000 7f82964d6700 1 --2- 172.21.15.81:0/3296277593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82900cf8b0 0x7f82901f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:08.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.937+0000 7f82964d6700 1 -- 172.21.15.81:0/3296277593 >> 172.21.15.81:0/3296277593 conn(0x7f829009c320 msgr2=0x7f829015a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:08.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.937+0000 7f82964d6700 1 -- 172.21.15.81:0/3296277593 shutdown_connections 2024-05-08T23:31:08.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:08.937+0000 7f82964d6700 1 -- 172.21.15.81:0/3296277593 wait complete. 2024-05-08T23:31:08.955 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2024-05-08T23:31:09.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.061+0000 7fbde3033700 1 Processor -- start 2024-05-08T23:31:09.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.061+0000 7fbde3033700 1 -- start start 2024-05-08T23:31:09.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde3033700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbddc0b3c20 0x7fbddc1624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:09.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde3033700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbddc071e70 0x7fbddc0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:09.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde3033700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbddc158960 con 0x7fbddc165070 2024-05-08T23:31:09.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde3033700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbddc0b1f60 con 0x7fbddc071e70 2024-05-08T23:31:09.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde3033700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc071120 con 0x7fbddc0b3c20 2024-05-08T23:31:09.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde2832700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fbddc165070 0x7fbddc165500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46740/0 (socket says 172.21.15.81:46740) 2024-05-08T23:31:09.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde2832700 1 -- 172.21.15.81:0/160477828 learned_addr learned my addr 172.21.15.81:0/160477828 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:09.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde1830700 1 --2- 172.21.15.81:0/160477828 >> v2:172.21.15.117:3300/0 conn(0x7fbddc071e70 0x7fbddc0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:09.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde2031700 1 --2- 172.21.15.81:0/160477828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbddc0b3c20 0x7fbddc1624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:09.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde102f700 1 -- 172.21.15.81:0/160477828 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763721495 0 0) 0x7fbddc158960 con 0x7fbddc165070 2024-05-08T23:31:09.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde102f700 1 -- 172.21.15.81:0/160477828 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc80035d0 con 0x7fbddc165070 2024-05-08T23:31:09.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde1830700 1 -- 172.21.15.81:0/160477828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbddc0b3c20 msgr2=0x7fbddc1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:09.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde1830700 1 --2- 172.21.15.81:0/160477828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbddc0b3c20 0x7fbddc1624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:09.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde1830700 1 -- 172.21.15.81:0/160477828 >> v1:172.21.15.81:6789/0 conn(0x7fbddc165070 legacy=0x7fbddc165500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:09.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde1830700 1 -- 172.21.15.81:0/160477828 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc0b4540 con 0x7fbddc071e70 2024-05-08T23:31:09.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde1830700 1 --2- 172.21.15.81:0/160477828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbddc071e70 0x7fbddc0b4000 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009850 tx=0x7fbdd0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=160fa3c36152b94b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:09.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde102f700 1 -- 172.21.15.81:0/160477828 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0011830 con 0x7fbddc071e70 2024-05-08T23:31:09.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde102f700 1 -- 172.21.15.81:0/160477828 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd0011990 con 0x7fbddc071e70 2024-05-08T23:31:09.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde102f700 1 -- 172.21.15.81:0/160477828 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0011d00 con 0x7fbddc071e70 2024-05-08T23:31:09.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde3033700 1 -- 172.21.15.81:0/160477828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbddc071e70 msgr2=0x7fbddc0b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:09.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde3033700 1 --2- 172.21.15.81:0/160477828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbddc071e70 0x7fbddc0b4000 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009850 tx=0x7fbdd0009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:09.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde3033700 1 -- 172.21.15.81:0/160477828 shutdown_connections 2024-05-08T23:31:09.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde3033700 1 --2- 172.21.15.81:0/160477828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbddc071e70 0x7fbddc0b4000 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:09.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde3033700 1 --2- 172.21.15.81:0/160477828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbddc0b3c20 0x7fbddc1624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:09.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde3033700 1 -- 172.21.15.81:0/160477828 >> 172.21.15.81:0/160477828 conn(0x7fbddc06c330 msgr2=0x7fbddc06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:09.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde3033700 1 -- 172.21.15.81:0/160477828 shutdown_connections 2024-05-08T23:31:09.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.065+0000 7fbde3033700 1 -- 172.21.15.81:0/160477828 wait complete. 2024-05-08T23:31:09.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde3033700 1 Processor -- start 2024-05-08T23:31:09.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde3033700 1 -- start start 2024-05-08T23:31:09.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde3033700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbddc071e70 0x7fbddc160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:09.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde3033700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbddc0b3c20 0x7fbddc14d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:09.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde3033700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbddc165070 0x7fbddc1f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:09.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde3033700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc14d5b0 con 0x7fbddc165070 2024-05-08T23:31:09.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde3033700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc1f7590 con 0x7fbddc0b3c20 2024-05-08T23:31:09.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde3033700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc1f7870 con 0x7fbddc071e70 2024-05-08T23:31:09.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde2832700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbddc165070 0x7fbddc1f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:09.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde2031700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbddc071e70 0x7fbddc160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:09.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde2031700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbddc071e70 0x7fbddc160c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:47548/0 (socket says 172.21.15.81:47548) 2024-05-08T23:31:09.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde2031700 1 -- 172.21.15.81:0/280571374 learned_addr learned my addr 172.21.15.81:0/280571374 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:09.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde1830700 1 --2- 172.21.15.81:0/280571374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbddc0b3c20 0x7fbddc14d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:09.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde2832700 1 -- 172.21.15.81:0/280571374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbddc071e70 msgr2=0x7fbddc160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:09.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde2832700 1 --2- 172.21.15.81:0/280571374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbddc071e70 0x7fbddc160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:09.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde2832700 1 -- 172.21.15.81:0/280571374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbddc0b3c20 msgr2=0x7fbddc14d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:09.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde2832700 1 --2- 172.21.15.81:0/280571374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbddc0b3c20 0x7fbddc14d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:09.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde2832700 1 -- 172.21.15.81:0/280571374 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd00094b0 con 0x7fbddc165070 2024-05-08T23:31:09.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.069+0000 7fbde2832700 1 --2- 172.21.15.81:0/280571374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbddc165070 0x7fbddc1f6c90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400ca70 tx=0x7fbdd400caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:09.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.073+0000 7fbdceffd700 1 -- 172.21.15.81:0/280571374 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4007480 con 0x7fbddc165070 2024-05-08T23:31:09.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.073+0000 7fbdceffd700 1 -- 172.21.15.81:0/280571374 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd40075e0 con 0x7fbddc165070 2024-05-08T23:31:09.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.073+0000 7fbde3033700 1 -- 172.21.15.81:0/280571374 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbddc1f7b50 con 0x7fbddc165070 2024-05-08T23:31:09.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.073+0000 7fbdceffd700 1 -- 172.21.15.81:0/280571374 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4007970 con 0x7fbddc165070 2024-05-08T23:31:09.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.073+0000 7fbde3033700 1 -- 172.21.15.81:0/280571374 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc15c390 con 0x7fbddc165070 2024-05-08T23:31:09.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.073+0000 7fbde3033700 1 -- 172.21.15.81:0/280571374 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbddc15c580 con 0x7fbddc165070 2024-05-08T23:31:09.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.073+0000 7fbdceffd700 1 -- 172.21.15.81:0/280571374 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbdd401b3b0 con 0x7fbddc165070 2024-05-08T23:31:09.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.073+0000 7fbdceffd700 1 --2- 172.21.15.81:0/280571374 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbdc0066730 0x7fbdc0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:09.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.077+0000 7fbde2031700 1 --2- 172.21.15.81:0/280571374 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbdc0066730 0x7fbdc0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:09.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.077+0000 7fbdceffd700 1 -- 172.21.15.81:0/280571374 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdd40839e0 con 0x7fbddc165070 2024-05-08T23:31:09.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.077+0000 7fbde2031700 1 --2- 172.21.15.81:0/280571374 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbdc0066730 0x7fbdc0068bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0b4ed0 tx=0x7fbdd8009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:09.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.085+0000 7fbdceffd700 1 -- 172.21.15.81:0/280571374 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbdd4050d40 con 0x7fbddc165070 2024-05-08T23:31:09.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.321+0000 7fbde3033700 1 -- 172.21.15.81:0/280571374 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7fbddc0684b0 con 0x7fbddc165070 2024-05-08T23:31:09.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.801+0000 7fbdceffd700 1 -- 172.21.15.81:0/280571374 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v50) v1 ==== 150+0+0 (secure 0 0 0) 0x7fbdd40170d0 con 0x7fbddc165070 2024-05-08T23:31:09.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.805+0000 7fbde3033700 1 -- 172.21.15.81:0/280571374 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbdc0066730 msgr2=0x7fbdc0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:09.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.805+0000 7fbde3033700 1 --2- 172.21.15.81:0/280571374 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbdc0066730 0x7fbdc0068bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0b4ed0 tx=0x7fbdd8009240 comp rx=0 tx=0).stop 2024-05-08T23:31:09.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.805+0000 7fbde3033700 1 -- 172.21.15.81:0/280571374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbddc165070 msgr2=0x7fbddc1f6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:09.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.805+0000 7fbde3033700 1 --2- 172.21.15.81:0/280571374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbddc165070 0x7fbddc1f6c90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400ca70 tx=0x7fbdd400caa0 comp rx=0 tx=0).stop 2024-05-08T23:31:09.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.805+0000 7fbde3033700 1 -- 172.21.15.81:0/280571374 shutdown_connections 2024-05-08T23:31:09.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.805+0000 7fbde3033700 1 --2- 172.21.15.81:0/280571374 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbdc0066730 0x7fbdc0068bf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:09.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.805+0000 7fbde3033700 1 --2- 172.21.15.81:0/280571374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbddc165070 0x7fbddc1f6c90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:09.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.805+0000 7fbde3033700 1 --2- 172.21.15.81:0/280571374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbddc0b3c20 0x7fbddc14d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:09.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.805+0000 7fbde3033700 1 --2- 172.21.15.81:0/280571374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbddc071e70 0x7fbddc160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:09.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.805+0000 7fbde3033700 1 -- 172.21.15.81:0/280571374 >> 172.21.15.81:0/280571374 conn(0x7fbddc06c330 msgr2=0x7fbddc164920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:09.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.805+0000 7fbde3033700 1 -- 172.21.15.81:0/280571374 shutdown_connections 2024-05-08T23:31:09.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.805+0000 7fbde3033700 1 -- 172.21.15.81:0/280571374 wait complete. 2024-05-08T23:31:09.827 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T23:31:09.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb443ed4700 1 Processor -- start 2024-05-08T23:31:09.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb443ed4700 1 -- start start 2024-05-08T23:31:09.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb443ed4700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb43c160ce0 0x7fb43c0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:09.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb443ed4700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0d01c0 0x7fb43c0b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:09.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb443ed4700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb43c0cf480 con 0x7fb43c0d3d40 2024-05-08T23:31:09.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb443ed4700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb43c0ce890 con 0x7fb43c160ce0 2024-05-08T23:31:09.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb443ed4700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c0ceb50 con 0x7fb43c0d01c0 2024-05-08T23:31:09.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb441c70700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb43c160ce0 0x7fb43c0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:09.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb44146f700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb43c0d3d40 0x7fb43c0cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46756/0 (socket says 172.21.15.81:46756) 2024-05-08T23:31:09.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb44146f700 1 -- 172.21.15.81:0/3078295019 learned_addr learned my addr 172.21.15.81:0/3078295019 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:09.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb441c70700 1 --2- 172.21.15.81:0/3078295019 >> v2:172.21.15.81:3300/0 conn(0x7fb43c160ce0 0x7fb43c0b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58354/0 (socket says 172.21.15.81:58354) 2024-05-08T23:31:09.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb442471700 1 --2- 172.21.15.81:0/3078295019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0d01c0 0x7fb43c0b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:09.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb441c70700 1 -- 172.21.15.81:0/3078295019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0d01c0 msgr2=0x7fb43c0b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:09.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb441c70700 1 --2- 172.21.15.81:0/3078295019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0d01c0 0x7fb43c0b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:09.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb440c6e700 1 -- 172.21.15.81:0/3078295019 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697864253 0 0) 0x7fb43c0cf480 con 0x7fb43c0d3d40 2024-05-08T23:31:09.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb441c70700 1 -- 172.21.15.81:0/3078295019 >> v1:172.21.15.81:6789/0 conn(0x7fb43c0d3d40 legacy=0x7fb43c0cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:09.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb441c70700 1 -- 172.21.15.81:0/3078295019 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c0cee30 con 0x7fb43c160ce0 2024-05-08T23:31:09.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb441c70700 1 --2- 172.21.15.81:0/3078295019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb43c160ce0 0x7fb43c0b2040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009850 tx=0x7fb42c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1c74fbce188f79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:09.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb440c6e700 1 -- 172.21.15.81:0/3078295019 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c011830 con 0x7fb43c160ce0 2024-05-08T23:31:09.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb440c6e700 1 -- 172.21.15.81:0/3078295019 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42c011990 con 0x7fb43c160ce0 2024-05-08T23:31:09.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.949+0000 7fb440c6e700 1 -- 172.21.15.81:0/3078295019 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c011d40 con 0x7fb43c160ce0 2024-05-08T23:31:09.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.953+0000 7fb443ed4700 1 -- 172.21.15.81:0/3078295019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb43c160ce0 msgr2=0x7fb43c0b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:09.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.953+0000 7fb443ed4700 1 --2- 172.21.15.81:0/3078295019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb43c160ce0 0x7fb43c0b2040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009850 tx=0x7fb42c009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:09.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.953+0000 7fb443ed4700 1 -- 172.21.15.81:0/3078295019 shutdown_connections 2024-05-08T23:31:09.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.953+0000 7fb443ed4700 1 --2- 172.21.15.81:0/3078295019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0d01c0 0x7fb43c0b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:09.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.953+0000 7fb443ed4700 1 --2- 172.21.15.81:0/3078295019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb43c160ce0 0x7fb43c0b2040 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:09.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.953+0000 7fb443ed4700 1 -- 172.21.15.81:0/3078295019 >> 172.21.15.81:0/3078295019 conn(0x7fb43c09c4c0 msgr2=0x7fb43c0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:09.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.953+0000 7fb443ed4700 1 -- 172.21.15.81:0/3078295019 shutdown_connections 2024-05-08T23:31:09.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.953+0000 7fb443ed4700 1 -- 172.21.15.81:0/3078295019 wait complete. 2024-05-08T23:31:09.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.953+0000 7fb443ed4700 1 Processor -- start 2024-05-08T23:31:09.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.953+0000 7fb443ed4700 1 -- start start 2024-05-08T23:31:09.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb443ed4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c0d01c0 0x7fb43c1f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:09.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb443ed4700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0d3d40 0x7fb43c06b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:09.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb443ed4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb43c160ce0 0x7fb43c06bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:09.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb443ed4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c1f7260 con 0x7fb43c160ce0 2024-05-08T23:31:09.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb443ed4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c0b5d80 con 0x7fb43c0d01c0 2024-05-08T23:31:09.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb443ed4700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c0b6060 con 0x7fb43c0d3d40 2024-05-08T23:31:09.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb442471700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb43c160ce0 0x7fb43c06bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:09.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb442471700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb43c160ce0 0x7fb43c06bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58370/0 (socket says 172.21.15.81:58370) 2024-05-08T23:31:09.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb442471700 1 -- 172.21.15.81:0/2906884715 learned_addr learned my addr 172.21.15.81:0/2906884715 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:09.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb44146f700 1 --2- 172.21.15.81:0/2906884715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0d3d40 0x7fb43c06b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:09.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb441c70700 1 --2- 172.21.15.81:0/2906884715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c0d01c0 0x7fb43c1f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:09.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb442471700 1 -- 172.21.15.81:0/2906884715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0d3d40 msgr2=0x7fb43c06b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:09.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb442471700 1 --2- 172.21.15.81:0/2906884715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0d3d40 0x7fb43c06b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:09.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb442471700 1 -- 172.21.15.81:0/2906884715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c0d01c0 msgr2=0x7fb43c1f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:09.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb442471700 1 --2- 172.21.15.81:0/2906884715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c0d01c0 0x7fb43c1f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:09.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb442471700 1 -- 172.21.15.81:0/2906884715 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c0094b0 con 0x7fb43c160ce0 2024-05-08T23:31:09.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb442471700 1 --2- 172.21.15.81:0/2906884715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb43c160ce0 0x7fb43c06bd50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb43400efc0 tx=0x7fb43400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:09.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb4327fc700 1 -- 172.21.15.81:0/2906884715 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43400cd40 con 0x7fb43c160ce0 2024-05-08T23:31:09.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb4327fc700 1 -- 172.21.15.81:0/2906884715 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43400f040 con 0x7fb43c160ce0 2024-05-08T23:31:09.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb443ed4700 1 -- 172.21.15.81:0/2906884715 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43c0b6340 con 0x7fb43c160ce0 2024-05-08T23:31:09.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb4327fc700 1 -- 172.21.15.81:0/2906884715 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4340146c0 con 0x7fb43c160ce0 2024-05-08T23:31:09.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.957+0000 7fb443ed4700 1 -- 172.21.15.81:0/2906884715 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43c0b6900 con 0x7fb43c160ce0 2024-05-08T23:31:09.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.961+0000 7fb443ed4700 1 -- 172.21.15.81:0/2906884715 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43c0672a0 con 0x7fb43c160ce0 2024-05-08T23:31:09.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.961+0000 7fb4327fc700 1 -- 172.21.15.81:0/2906884715 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb434014820 con 0x7fb43c160ce0 2024-05-08T23:31:09.965 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.961+0000 7fb4327fc700 1 --2- 172.21.15.81:0/2906884715 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb420066730 0x7fb420068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:09.965 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.961+0000 7fb441c70700 1 --2- 172.21.15.81:0/2906884715 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb420066730 0x7fb420068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:09.965 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.961+0000 7fb4327fc700 1 -- 172.21.15.81:0/2906884715 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb434084d90 con 0x7fb43c160ce0 2024-05-08T23:31:09.965 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.961+0000 7fb441c70700 1 --2- 172.21.15.81:0/2906884715 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb420066730 0x7fb420068bf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0090b0 tx=0x7fb42c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:09.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:09.969+0000 7fb4327fc700 1 -- 172.21.15.81:0/2906884715 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb434052060 con 0x7fb43c160ce0 2024-05-08T23:31:10.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.137+0000 7fb443ed4700 1 -- 172.21.15.81:0/2906884715 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb43c0b9010 con 0x7fb43c160ce0 2024-05-08T23:31:10.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.145+0000 7fb4327fc700 1 -- 172.21.15.81:0/2906884715 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb4340180c0 con 0x7fb43c160ce0 2024-05-08T23:31:10.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.145+0000 7fb443ed4700 1 -- 172.21.15.81:0/2906884715 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb420066730 msgr2=0x7fb420068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:10.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.145+0000 7fb443ed4700 1 --2- 172.21.15.81:0/2906884715 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb420066730 0x7fb420068bf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0090b0 tx=0x7fb42c01c040 comp rx=0 tx=0).stop 2024-05-08T23:31:10.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.145+0000 7fb443ed4700 1 -- 172.21.15.81:0/2906884715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb43c160ce0 msgr2=0x7fb43c06bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:10.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.145+0000 7fb443ed4700 1 --2- 172.21.15.81:0/2906884715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb43c160ce0 0x7fb43c06bd50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb43400efc0 tx=0x7fb43400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.145+0000 7fb443ed4700 1 -- 172.21.15.81:0/2906884715 shutdown_connections 2024-05-08T23:31:10.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.145+0000 7fb443ed4700 1 --2- 172.21.15.81:0/2906884715 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb420066730 0x7fb420068bf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.145+0000 7fb443ed4700 1 --2- 172.21.15.81:0/2906884715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb43c160ce0 0x7fb43c06bd50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.145+0000 7fb443ed4700 1 --2- 172.21.15.81:0/2906884715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb43c0d3d40 0x7fb43c06b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.145+0000 7fb443ed4700 1 --2- 172.21.15.81:0/2906884715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c0d01c0 0x7fb43c1f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.145+0000 7fb443ed4700 1 -- 172.21.15.81:0/2906884715 >> 172.21.15.81:0/2906884715 conn(0x7fb43c09c4c0 msgr2=0x7fb43c06c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:10.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.145+0000 7fb443ed4700 1 -- 172.21.15.81:0/2906884715 shutdown_connections 2024-05-08T23:31:10.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.145+0000 7fb443ed4700 1 -- 172.21.15.81:0/2906884715 wait complete. 2024-05-08T23:31:10.164 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:31:10.164 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:31:10.182 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:31:10.203 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:31:10.204 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:31:10.204 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:31:10.211 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:31:10.211 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:31:10.217 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:10.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.365+0000 7f459b59e700 1 Processor -- start 2024-05-08T23:31:10.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.365+0000 7f459b59e700 1 -- start start 2024-05-08T23:31:10.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f459b59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f459c15b180 0x7f459c0b0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:10.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f459b59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f459c0cfaf0 0x7f459c0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:10.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f459b59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f459c0cd4b0 con 0x7f459c0ce9d0 2024-05-08T23:31:10.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f459b59e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f459c0b0b50 con 0x7f459c15b180 2024-05-08T23:31:10.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f459b59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f459c0cc5c0 con 0x7f459c0cfaf0 2024-05-08T23:31:10.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f459a59c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f459c15b180 0x7f459c0b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:10.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f459a59c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f459c15b180 0x7f459c0b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58374/0 (socket says 172.21.15.81:58374) 2024-05-08T23:31:10.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f459a59c700 1 -- 172.21.15.81:0/2548268269 learned_addr learned my addr 172.21.15.81:0/2548268269 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:10.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f459ad9d700 1 --2- 172.21.15.81:0/2548268269 >> v2:172.21.15.117:3300/0 conn(0x7f459c0cfaf0 0x7f459c0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:10.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f459a59c700 1 -- 172.21.15.81:0/2548268269 >> v2:172.21.15.117:3300/0 conn(0x7f459c0cfaf0 msgr2=0x7f459c0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:10.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f459a59c700 1 --2- 172.21.15.81:0/2548268269 >> v2:172.21.15.117:3300/0 conn(0x7f459c0cfaf0 0x7f459c0cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f459a59c700 1 -- 172.21.15.81:0/2548268269 >> v1:172.21.15.117:6789/0 conn(0x7f459c0ce9d0 legacy=0x7f459c0cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:10.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f459a59c700 1 -- 172.21.15.81:0/2548268269 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c0cc8a0 con 0x7f459c15b180 2024-05-08T23:31:10.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f459a59c700 1 --2- 172.21.15.81:0/2548268269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f459c15b180 0x7f459c0b0610 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f459c06db80 tx=0x7f458c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf647823bc9e890 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:10.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f4599d9b700 1 -- 172.21.15.81:0/2548268269 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c011830 con 0x7f459c15b180 2024-05-08T23:31:10.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f4599d9b700 1 -- 172.21.15.81:0/2548268269 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458c011990 con 0x7f459c15b180 2024-05-08T23:31:10.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.369+0000 7f4599d9b700 1 -- 172.21.15.81:0/2548268269 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c011d20 con 0x7f459c15b180 2024-05-08T23:31:10.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 -- 172.21.15.81:0/2548268269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f459c15b180 msgr2=0x7f459c0b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:10.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 --2- 172.21.15.81:0/2548268269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f459c15b180 0x7f459c0b0610 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f459c06db80 tx=0x7f458c009bf0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 -- 172.21.15.81:0/2548268269 shutdown_connections 2024-05-08T23:31:10.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 --2- 172.21.15.81:0/2548268269 >> v2:172.21.15.117:3300/0 conn(0x7f459c0cfaf0 0x7f459c0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.377 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 --2- 172.21.15.81:0/2548268269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f459c15b180 0x7f459c0b0610 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.377 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 -- 172.21.15.81:0/2548268269 >> 172.21.15.81:0/2548268269 conn(0x7f459c0b2970 msgr2=0x7f459c0b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:10.377 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 -- 172.21.15.81:0/2548268269 shutdown_connections 2024-05-08T23:31:10.377 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 -- 172.21.15.81:0/2548268269 wait complete. 2024-05-08T23:31:10.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 Processor -- start 2024-05-08T23:31:10.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 -- start start 2024-05-08T23:31:10.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f459c0ce9d0 0x7f459c14a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:10.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f459c0cfaf0 0x7f459c0c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:10.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f459c15b180 0x7f459c0c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:10.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c155860 con 0x7f459c15b180 2024-05-08T23:31:10.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c0b5a80 con 0x7f459c0cfaf0 2024-05-08T23:31:10.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.373+0000 7f459b59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459c0b5d60 con 0x7f459c0ce9d0 2024-05-08T23:31:10.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f459ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f459c15b180 0x7f459c0c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:10.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f459a59c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f459c0ce9d0 0x7f459c14a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:10.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f4591d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f459c0cfaf0 0x7f459c0c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:10.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f459ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f459c15b180 0x7f459c0c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58376/0 (socket says 172.21.15.81:58376) 2024-05-08T23:31:10.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f459ad9d700 1 -- 172.21.15.81:0/1843937041 learned_addr learned my addr 172.21.15.81:0/1843937041 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:10.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f459ad9d700 1 -- 172.21.15.81:0/1843937041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f459c0ce9d0 msgr2=0x7f459c14a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:10.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f459ad9d700 1 --2- 172.21.15.81:0/1843937041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f459c0ce9d0 0x7f459c14a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f459ad9d700 1 -- 172.21.15.81:0/1843937041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f459c0cfaf0 msgr2=0x7f459c0c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:10.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f459ad9d700 1 --2- 172.21.15.81:0/1843937041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f459c0cfaf0 0x7f459c0c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f459ad9d700 1 -- 172.21.15.81:0/1843937041 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c009570 con 0x7f459c15b180 2024-05-08T23:31:10.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f459ad9d700 1 --2- 172.21.15.81:0/1843937041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f459c15b180 0x7f459c0c8410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f459400efc0 tx=0x7f459400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:10.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f45937fe700 1 -- 172.21.15.81:0/1843937041 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459400cc90 con 0x7f459c15b180 2024-05-08T23:31:10.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f45937fe700 1 -- 172.21.15.81:0/1843937041 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459400f040 con 0x7f459c15b180 2024-05-08T23:31:10.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f459b59e700 1 -- 172.21.15.81:0/1843937041 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f459c0b6040 con 0x7f459c15b180 2024-05-08T23:31:10.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f45937fe700 1 -- 172.21.15.81:0/1843937041 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45940146c0 con 0x7f459c15b180 2024-05-08T23:31:10.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f459b59e700 1 -- 172.21.15.81:0/1843937041 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f459c14a540 con 0x7f459c15b180 2024-05-08T23:31:10.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f459b59e700 1 -- 172.21.15.81:0/1843937041 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459c0d0360 con 0x7f459c15b180 2024-05-08T23:31:10.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f45937fe700 1 -- 172.21.15.81:0/1843937041 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4594014820 con 0x7f459c15b180 2024-05-08T23:31:10.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.377+0000 7f45937fe700 1 --2- 172.21.15.81:0/1843937041 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f457c066760 0x7f457c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:10.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.381+0000 7f45937fe700 1 -- 172.21.15.81:0/1843937041 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4594084d80 con 0x7f459c15b180 2024-05-08T23:31:10.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.381+0000 7f459a59c700 1 --2- 172.21.15.81:0/1843937041 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f457c066760 0x7f457c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:10.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.381+0000 7f459a59c700 1 --2- 172.21.15.81:0/1843937041 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f457c066760 0x7f457c068c20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f458c000c00 tx=0x7f458c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:10.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.385+0000 7f45937fe700 1 -- 172.21.15.81:0/1843937041 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4594052060 con 0x7f459c15b180 2024-05-08T23:31:10.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.561+0000 7f459b59e700 1 -- 172.21.15.81:0/1843937041 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f459c0672a0 con 0x7f459c15b180 2024-05-08T23:31:10.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.561+0000 7f45937fe700 1 -- 172.21.15.81:0/1843937041 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f4594018070 con 0x7f459c15b180 2024-05-08T23:31:10.567 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:10.567 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:10.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.565+0000 7f459b59e700 1 -- 172.21.15.81:0/1843937041 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f457c066760 msgr2=0x7f457c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:10.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.565+0000 7f459b59e700 1 --2- 172.21.15.81:0/1843937041 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f457c066760 0x7f457c068c20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f458c000c00 tx=0x7f458c008030 comp rx=0 tx=0).stop 2024-05-08T23:31:10.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.565+0000 7f459b59e700 1 -- 172.21.15.81:0/1843937041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f459c15b180 msgr2=0x7f459c0c8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:10.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.565+0000 7f459b59e700 1 --2- 172.21.15.81:0/1843937041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f459c15b180 0x7f459c0c8410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f459400efc0 tx=0x7f459400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.565+0000 7f459b59e700 1 -- 172.21.15.81:0/1843937041 shutdown_connections 2024-05-08T23:31:10.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.565+0000 7f459b59e700 1 --2- 172.21.15.81:0/1843937041 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f457c066760 0x7f457c068c20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.565+0000 7f459b59e700 1 --2- 172.21.15.81:0/1843937041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f459c15b180 0x7f459c0c8410 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.565+0000 7f459b59e700 1 --2- 172.21.15.81:0/1843937041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f459c0cfaf0 0x7f459c0c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.565+0000 7f459b59e700 1 --2- 172.21.15.81:0/1843937041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f459c0ce9d0 0x7f459c14a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.565+0000 7f459b59e700 1 -- 172.21.15.81:0/1843937041 >> 172.21.15.81:0/1843937041 conn(0x7f459c0b2970 msgr2=0x7f459c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:10.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.565+0000 7f459b59e700 1 -- 172.21.15.81:0/1843937041 shutdown_connections 2024-05-08T23:31:10.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.565+0000 7f459b59e700 1 -- 172.21.15.81:0/1843937041 wait complete. 2024-05-08T23:31:10.582 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:10.582 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:10.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6289e700 1 Processor -- start 2024-05-08T23:31:10.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6289e700 1 -- start start 2024-05-08T23:31:10.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6289e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1d54007990 0x7f1d54058950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:10.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6289e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d540578b0 con 0x7f1d5403ebd0 2024-05-08T23:31:10.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6289e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d54057b90 con 0x7f1d540e33a0 2024-05-08T23:31:10.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6289e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d540576c0 con 0x7f1d54007990 2024-05-08T23:31:10.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6109b700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f1d5403ebd0 0x7f1d54007280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46770/0 (socket says 172.21.15.81:46770) 2024-05-08T23:31:10.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6109b700 1 -- 172.21.15.81:0/1715409666 learned_addr learned my addr 172.21.15.81:0/1715409666 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:10.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6209d700 1 --2- 172.21.15.81:0/1715409666 >> v2:172.21.15.117:3300/0 conn(0x7f1d54007990 0x7f1d54058950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:10.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6089a700 1 -- 172.21.15.81:0/1715409666 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1470088933 0 0) 0x7f1d540578b0 con 0x7f1d5403ebd0 2024-05-08T23:31:10.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6089a700 1 -- 172.21.15.81:0/1715409666 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d480035d0 con 0x7f1d5403ebd0 2024-05-08T23:31:10.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6209d700 1 -- 172.21.15.81:0/1715409666 >> v1:172.21.15.117:6789/0 conn(0x7f1d540e33a0 legacy=0x7f1d54056fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:10.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6209d700 1 -- 172.21.15.81:0/1715409666 >> v1:172.21.15.81:6789/0 conn(0x7f1d5403ebd0 legacy=0x7f1d54007280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:10.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6209d700 1 -- 172.21.15.81:0/1715409666 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d54057e70 con 0x7f1d54007990 2024-05-08T23:31:10.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6209d700 1 --2- 172.21.15.81:0/1715409666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d54007990 0x7f1d54058950 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c069b40 tx=0x7f1d5c068970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1006e170278ee1c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:10.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6089a700 1 -- 172.21.15.81:0/1715409666 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c068790 con 0x7f1d54007990 2024-05-08T23:31:10.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6089a700 1 -- 172.21.15.81:0/1715409666 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5c04ef50 con 0x7f1d54007990 2024-05-08T23:31:10.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.697+0000 7f1d6089a700 1 -- 172.21.15.81:0/1715409666 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c067bb0 con 0x7f1d54007990 2024-05-08T23:31:10.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6289e700 1 -- 172.21.15.81:0/1715409666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d54007990 msgr2=0x7f1d54058950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:10.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6289e700 1 --2- 172.21.15.81:0/1715409666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d54007990 0x7f1d54058950 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c069b40 tx=0x7f1d5c068970 comp rx=0 tx=0).stop 2024-05-08T23:31:10.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6289e700 1 -- 172.21.15.81:0/1715409666 shutdown_connections 2024-05-08T23:31:10.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6289e700 1 --2- 172.21.15.81:0/1715409666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d54007990 0x7f1d54058950 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6289e700 1 -- 172.21.15.81:0/1715409666 >> 172.21.15.81:0/1715409666 conn(0x7f1d54055a10 msgr2=0x7f1d54055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:10.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6289e700 1 -- 172.21.15.81:0/1715409666 shutdown_connections 2024-05-08T23:31:10.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6289e700 1 -- 172.21.15.81:0/1715409666 wait complete. 2024-05-08T23:31:10.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6289e700 1 Processor -- start 2024-05-08T23:31:10.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6289e700 1 -- start start 2024-05-08T23:31:10.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6289e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d54007990 0x7f1d54178cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:10.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6289e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d5403ebd0 0x7f1d540ded60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:10.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6289e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d540e33a0 0x7f1d540389f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:10.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6289e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540e40a0 con 0x7f1d54007990 2024-05-08T23:31:10.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6289e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540e4380 con 0x7f1d5403ebd0 2024-05-08T23:31:10.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6289e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540e4660 con 0x7f1d540e33a0 2024-05-08T23:31:10.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6189c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d54007990 0x7f1d54178cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:10.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6109b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d5403ebd0 0x7f1d540ded60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:10.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6189c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d54007990 0x7f1d54178cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58384/0 (socket says 172.21.15.81:58384) 2024-05-08T23:31:10.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6109b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d5403ebd0 0x7f1d540ded60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:60044/0 (socket says 172.21.15.81:60044) 2024-05-08T23:31:10.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6209d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d540e33a0 0x7f1d540389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:10.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.701+0000 7f1d6189c700 1 -- 172.21.15.81:0/182894925 learned_addr learned my addr 172.21.15.81:0/182894925 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:10.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.705+0000 7f1d6189c700 1 -- 172.21.15.81:0/182894925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d540e33a0 msgr2=0x7f1d540389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:10.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.705+0000 7f1d6189c700 1 --2- 172.21.15.81:0/182894925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d540e33a0 0x7f1d540389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.705+0000 7f1d6189c700 1 -- 172.21.15.81:0/182894925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d5403ebd0 msgr2=0x7f1d540ded60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:10.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.705+0000 7f1d6189c700 1 --2- 172.21.15.81:0/182894925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d5403ebd0 0x7f1d540ded60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.705+0000 7f1d6189c700 1 -- 172.21.15.81:0/182894925 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c04f5c0 con 0x7f1d54007990 2024-05-08T23:31:10.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.705+0000 7f1d6189c700 1 --2- 172.21.15.81:0/182894925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d54007990 0x7f1d54178cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00c500 tx=0x7f1d4c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:10.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.705+0000 7f1d527fc700 1 -- 172.21.15.81:0/182894925 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c0052d0 con 0x7f1d54007990 2024-05-08T23:31:10.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.705+0000 7f1d527fc700 1 -- 172.21.15.81:0/182894925 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c005430 con 0x7f1d54007990 2024-05-08T23:31:10.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.705+0000 7f1d6289e700 1 -- 172.21.15.81:0/182894925 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d540e4940 con 0x7f1d54007990 2024-05-08T23:31:10.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.705+0000 7f1d527fc700 1 -- 172.21.15.81:0/182894925 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c0055a0 con 0x7f1d54007990 2024-05-08T23:31:10.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.705+0000 7f1d6289e700 1 -- 172.21.15.81:0/182894925 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d540df2a0 con 0x7f1d54007990 2024-05-08T23:31:10.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.705+0000 7f1d6289e700 1 -- 172.21.15.81:0/182894925 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d54005440 con 0x7f1d54007990 2024-05-08T23:31:10.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.709+0000 7f1d527fc700 1 -- 172.21.15.81:0/182894925 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1d4c0038e0 con 0x7f1d54007990 2024-05-08T23:31:10.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.709+0000 7f1d527fc700 1 --2- 172.21.15.81:0/182894925 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1d40066730 0x7f1d40068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:10.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.709+0000 7f1d527fc700 1 -- 172.21.15.81:0/182894925 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d4c083470 con 0x7f1d54007990 2024-05-08T23:31:10.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.709+0000 7f1d6109b700 1 --2- 172.21.15.81:0/182894925 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1d40066730 0x7f1d40068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:10.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.709+0000 7f1d6109b700 1 --2- 172.21.15.81:0/182894925 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1d40066730 0x7f1d40068bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1d54057a90 tx=0x7f1d58007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:10.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.713+0000 7f1d527fc700 1 -- 172.21.15.81:0/182894925 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1d4c0507e0 con 0x7f1d54007990 2024-05-08T23:31:10.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.885+0000 7f1d6289e700 1 -- 172.21.15.81:0/182894925 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d540e41e0 con 0x7f1d54007990 2024-05-08T23:31:10.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.885+0000 7f1d527fc700 1 -- 172.21.15.81:0/182894925 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d540e41e0 con 0x7f1d54007990 2024-05-08T23:31:10.889 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:10.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.889+0000 7f1d6289e700 1 -- 172.21.15.81:0/182894925 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1d40066730 msgr2=0x7f1d40068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:10.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.889+0000 7f1d6289e700 1 --2- 172.21.15.81:0/182894925 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1d40066730 0x7f1d40068bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1d54057a90 tx=0x7f1d58007de0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.889+0000 7f1d6289e700 1 -- 172.21.15.81:0/182894925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d54007990 msgr2=0x7f1d54178cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:10.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.889+0000 7f1d6289e700 1 --2- 172.21.15.81:0/182894925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d54007990 0x7f1d54178cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00c500 tx=0x7f1d4c00c810 comp rx=0 tx=0).stop 2024-05-08T23:31:10.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.889+0000 7f1d6289e700 1 -- 172.21.15.81:0/182894925 shutdown_connections 2024-05-08T23:31:10.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.889+0000 7f1d6289e700 1 --2- 172.21.15.81:0/182894925 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1d40066730 0x7f1d40068bf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.889+0000 7f1d6289e700 1 --2- 172.21.15.81:0/182894925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d540e33a0 0x7f1d540389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.889+0000 7f1d6289e700 1 --2- 172.21.15.81:0/182894925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d5403ebd0 0x7f1d540ded60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.889+0000 7f1d6289e700 1 --2- 172.21.15.81:0/182894925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d54007990 0x7f1d54178cc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:10.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.889+0000 7f1d6289e700 1 -- 172.21.15.81:0/182894925 >> 172.21.15.81:0/182894925 conn(0x7f1d54055a10 msgr2=0x7f1d540535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:10.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.889+0000 7f1d6289e700 1 -- 172.21.15.81:0/182894925 shutdown_connections 2024-05-08T23:31:10.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:10.889+0000 7f1d6289e700 1 -- 172.21.15.81:0/182894925 wait complete. 2024-05-08T23:31:10.905 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:11.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d8c05700 1 Processor -- start 2024-05-08T23:31:11.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d8c05700 1 -- start start 2024-05-08T23:31:11.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d8c05700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f80d415d180 0x7f80d40cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d8c05700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80d40d2f60 0x7f80d40d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d8c05700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d40ce200 con 0x7f80d415d560 2024-05-08T23:31:11.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d8c05700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d40d3950 con 0x7f80d415d180 2024-05-08T23:31:11.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d8c05700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d40ce480 con 0x7f80d40d2f60 2024-05-08T23:31:11.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d2ffd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f80d415d560 0x7f80d40d1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46786/0 (socket says 172.21.15.81:46786) 2024-05-08T23:31:11.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d2ffd700 1 -- 172.21.15.81:0/3064456776 learned_addr learned my addr 172.21.15.81:0/3064456776 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:11.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d3fff700 1 --2- 172.21.15.81:0/3064456776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80d40d2f60 0x7f80d40d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d37fe700 1 --2- 172.21.15.81:0/3064456776 >> v2:172.21.15.117:3300/0 conn(0x7f80d415d180 0x7f80d40cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d27fc700 1 -- 172.21.15.81:0/3064456776 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701634794 0 0) 0x7f80d40ce200 con 0x7f80d415d560 2024-05-08T23:31:11.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d27fc700 1 -- 172.21.15.81:0/3064456776 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80b40035d0 con 0x7f80d415d560 2024-05-08T23:31:11.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d3fff700 1 -- 172.21.15.81:0/3064456776 >> v2:172.21.15.117:3300/0 conn(0x7f80d415d180 msgr2=0x7f80d40cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:11.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d3fff700 1 --2- 172.21.15.81:0/3064456776 >> v2:172.21.15.117:3300/0 conn(0x7f80d415d180 0x7f80d40cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d3fff700 1 -- 172.21.15.81:0/3064456776 >> v1:172.21.15.81:6789/0 conn(0x7f80d415d560 legacy=0x7f80d40d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d3fff700 1 -- 172.21.15.81:0/3064456776 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d40ce760 con 0x7f80d40d2f60 2024-05-08T23:31:11.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.021+0000 7f80d3fff700 1 --2- 172.21.15.81:0/3064456776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80d40d2f60 0x7f80d40d3410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009850 tx=0x7f80c4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a40af4d88678458a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d27fc700 1 -- 172.21.15.81:0/3064456776 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4011830 con 0x7f80d40d2f60 2024-05-08T23:31:11.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d27fc700 1 -- 172.21.15.81:0/3064456776 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c4011990 con 0x7f80d40d2f60 2024-05-08T23:31:11.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d27fc700 1 -- 172.21.15.81:0/3064456776 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4011d40 con 0x7f80d40d2f60 2024-05-08T23:31:11.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 -- 172.21.15.81:0/3064456776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80d40d2f60 msgr2=0x7f80d40d3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 --2- 172.21.15.81:0/3064456776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80d40d2f60 0x7f80d40d3410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009850 tx=0x7f80c4009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:11.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 -- 172.21.15.81:0/3064456776 shutdown_connections 2024-05-08T23:31:11.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 --2- 172.21.15.81:0/3064456776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80d40d2f60 0x7f80d40d3410 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 --2- 172.21.15.81:0/3064456776 >> v2:172.21.15.117:3300/0 conn(0x7f80d415d180 0x7f80d40cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 -- 172.21.15.81:0/3064456776 >> 172.21.15.81:0/3064456776 conn(0x7f80d408c530 msgr2=0x7f80d409c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:11.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 -- 172.21.15.81:0/3064456776 shutdown_connections 2024-05-08T23:31:11.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 -- 172.21.15.81:0/3064456776 wait complete. 2024-05-08T23:31:11.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 Processor -- start 2024-05-08T23:31:11.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 -- start start 2024-05-08T23:31:11.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80d40d2f60 0x7f80d415fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80d415d180 0x7f80d4159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80d415d560 0x7f80d40b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d415ea30 con 0x7f80d415d180 2024-05-08T23:31:11.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d415ed10 con 0x7f80d415d560 2024-05-08T23:31:11.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d415eff0 con 0x7f80d40d2f60 2024-05-08T23:31:11.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80d415d180 0x7f80d4159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80d415d180 0x7f80d4159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58390/0 (socket says 172.21.15.81:58390) 2024-05-08T23:31:11.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d2ffd700 1 -- 172.21.15.81:0/2776686731 learned_addr learned my addr 172.21.15.81:0/2776686731 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:11.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d37fe700 1 --2- 172.21.15.81:0/2776686731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80d40d2f60 0x7f80d415fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d3fff700 1 --2- 172.21.15.81:0/2776686731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80d415d560 0x7f80d40b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d2ffd700 1 -- 172.21.15.81:0/2776686731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80d40d2f60 msgr2=0x7f80d415fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d2ffd700 1 --2- 172.21.15.81:0/2776686731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80d40d2f60 0x7f80d415fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d2ffd700 1 -- 172.21.15.81:0/2776686731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80d415d560 msgr2=0x7f80d40b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d2ffd700 1 --2- 172.21.15.81:0/2776686731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80d415d560 0x7f80d40b6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d2ffd700 1 -- 172.21.15.81:0/2776686731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c40094b0 con 0x7f80d415d180 2024-05-08T23:31:11.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d2ffd700 1 --2- 172.21.15.81:0/2776686731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80d415d180 0x7f80d4159260 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f80bc005930 tx=0x7f80bc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80cbfff700 1 -- 172.21.15.81:0/2776686731 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc005d90 con 0x7f80d415d180 2024-05-08T23:31:11.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 -- 172.21.15.81:0/2776686731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d415f2d0 con 0x7f80d415d180 2024-05-08T23:31:11.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.025+0000 7f80d8c05700 1 -- 172.21.15.81:0/2776686731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d415f890 con 0x7f80d415d180 2024-05-08T23:31:11.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.029+0000 7f80cbfff700 1 -- 172.21.15.81:0/2776686731 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80bc0070d0 con 0x7f80d415d180 2024-05-08T23:31:11.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.029+0000 7f80cbfff700 1 -- 172.21.15.81:0/2776686731 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc001230 con 0x7f80d415d180 2024-05-08T23:31:11.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.029+0000 7f80d8c05700 1 -- 172.21.15.81:0/2776686731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ac005310 con 0x7f80d415d180 2024-05-08T23:31:11.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.029+0000 7f80cbfff700 1 -- 172.21.15.81:0/2776686731 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f80bc007440 con 0x7f80d415d180 2024-05-08T23:31:11.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.029+0000 7f80cbfff700 1 --2- 172.21.15.81:0/2776686731 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80b8066760 0x7f80b8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.029+0000 7f80d37fe700 1 --2- 172.21.15.81:0/2776686731 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80b8066760 0x7f80b8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.033+0000 7f80cbfff700 1 -- 172.21.15.81:0/2776686731 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80bc083490 con 0x7f80d415d180 2024-05-08T23:31:11.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.033+0000 7f80d37fe700 1 --2- 172.21.15.81:0/2776686731 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80b8066760 0x7f80b8068c20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f80d41f66c0 tx=0x7f80cc00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.037+0000 7f80cbfff700 1 -- 172.21.15.81:0/2776686731 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f80bc0508b0 con 0x7f80d415d180 2024-05-08T23:31:11.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.201+0000 7f80d8c05700 1 -- 172.21.15.81:0/2776686731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80ac005b40 con 0x7f80d415d180 2024-05-08T23:31:11.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.201+0000 7f80cbfff700 1 -- 172.21.15.81:0/2776686731 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f80bc053ed0 con 0x7f80d415d180 2024-05-08T23:31:11.206 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:11.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.205+0000 7f80d8c05700 1 -- 172.21.15.81:0/2776686731 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80b8066760 msgr2=0x7f80b8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.205+0000 7f80d8c05700 1 --2- 172.21.15.81:0/2776686731 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80b8066760 0x7f80b8068c20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f80d41f66c0 tx=0x7f80cc00b040 comp rx=0 tx=0).stop 2024-05-08T23:31:11.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.205+0000 7f80d8c05700 1 -- 172.21.15.81:0/2776686731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80d415d180 msgr2=0x7f80d4159260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.205+0000 7f80d8c05700 1 --2- 172.21.15.81:0/2776686731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80d415d180 0x7f80d4159260 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f80bc005930 tx=0x7f80bc00c860 comp rx=0 tx=0).stop 2024-05-08T23:31:11.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.205+0000 7f80d8c05700 1 -- 172.21.15.81:0/2776686731 shutdown_connections 2024-05-08T23:31:11.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.205+0000 7f80d8c05700 1 --2- 172.21.15.81:0/2776686731 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80b8066760 0x7f80b8068c20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.205+0000 7f80d8c05700 1 --2- 172.21.15.81:0/2776686731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80d415d560 0x7f80d40b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.205+0000 7f80d8c05700 1 --2- 172.21.15.81:0/2776686731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80d415d180 0x7f80d4159260 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.205+0000 7f80d8c05700 1 --2- 172.21.15.81:0/2776686731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80d40d2f60 0x7f80d415fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.205+0000 7f80d8c05700 1 -- 172.21.15.81:0/2776686731 >> 172.21.15.81:0/2776686731 conn(0x7f80d408c530 msgr2=0x7f80d40b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:11.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.205+0000 7f80d8c05700 1 -- 172.21.15.81:0/2776686731 shutdown_connections 2024-05-08T23:31:11.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.205+0000 7f80d8c05700 1 -- 172.21.15.81:0/2776686731 wait complete. 2024-05-08T23:31:11.221 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:11.342 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.337+0000 7f8ddbfff700 1 Processor -- start 2024-05-08T23:31:11.342 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.337+0000 7f8ddbfff700 1 -- start start 2024-05-08T23:31:11.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddbfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc1544a0 0x7f8ddc0711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddbfff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc0aee50 con 0x7f8ddc15aa40 2024-05-08T23:31:11.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddbfff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc15b260 con 0x7f8ddc15e530 2024-05-08T23:31:11.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddbfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc0ae2f0 con 0x7f8ddc1544a0 2024-05-08T23:31:11.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddb7fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f8ddc15aa40 0x7f8ddc06dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46798/0 (socket says 172.21.15.81:46798) 2024-05-08T23:31:11.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddb7fe700 1 -- 172.21.15.81:0/3600123955 learned_addr learned my addr 172.21.15.81:0/3600123955 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:11.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddaffd700 1 --2- 172.21.15.81:0/3600123955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc1544a0 0x7f8ddc0711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8dda7fc700 1 -- 172.21.15.81:0/3600123955 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551296067 0 0) 0x7f8dd4001490 con 0x7f8ddc15aa40 2024-05-08T23:31:11.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8dda7fc700 1 -- 172.21.15.81:0/3600123955 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dc80035d0 con 0x7f8ddc15aa40 2024-05-08T23:31:11.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddaffd700 1 -- 172.21.15.81:0/3600123955 >> v1:172.21.15.117:6789/0 conn(0x7f8ddc15e530 legacy=0x7f8ddc15a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddaffd700 1 -- 172.21.15.81:0/3600123955 >> v1:172.21.15.81:6789/0 conn(0x7f8ddc15aa40 legacy=0x7f8ddc06dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddaffd700 1 -- 172.21.15.81:0/3600123955 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc15af80 con 0x7f8ddc1544a0 2024-05-08T23:31:11.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddaffd700 1 --2- 172.21.15.81:0/3600123955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc1544a0 0x7f8ddc0711a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009880 tx=0x7f8dcc009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca72f7a6d9f96b6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8dda7fc700 1 -- 172.21.15.81:0/3600123955 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc011830 con 0x7f8ddc1544a0 2024-05-08T23:31:11.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8dda7fc700 1 -- 172.21.15.81:0/3600123955 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc011990 con 0x7f8ddc1544a0 2024-05-08T23:31:11.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8dda7fc700 1 -- 172.21.15.81:0/3600123955 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc011d00 con 0x7f8ddc1544a0 2024-05-08T23:31:11.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddbfff700 1 -- 172.21.15.81:0/3600123955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc1544a0 msgr2=0x7f8ddc0711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddbfff700 1 --2- 172.21.15.81:0/3600123955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc1544a0 0x7f8ddc0711a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009880 tx=0x7f8dcc009b90 comp rx=0 tx=0).stop 2024-05-08T23:31:11.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddbfff700 1 -- 172.21.15.81:0/3600123955 shutdown_connections 2024-05-08T23:31:11.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddbfff700 1 --2- 172.21.15.81:0/3600123955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc1544a0 0x7f8ddc0711a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.341+0000 7f8ddbfff700 1 -- 172.21.15.81:0/3600123955 >> 172.21.15.81:0/3600123955 conn(0x7f8ddc0b1a60 msgr2=0x7f8ddc0b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:11.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.345+0000 7f8ddbfff700 1 -- 172.21.15.81:0/3600123955 shutdown_connections 2024-05-08T23:31:11.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.345+0000 7f8ddbfff700 1 -- 172.21.15.81:0/3600123955 wait complete. 2024-05-08T23:31:11.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.345+0000 7f8ddbfff700 1 Processor -- start 2024-05-08T23:31:11.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.345+0000 7f8ddbfff700 1 -- start start 2024-05-08T23:31:11.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.345+0000 7f8ddbfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc1544a0 0x7f8ddc139060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.345+0000 7f8ddbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ddc15aa40 0x7f8ddc0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.345+0000 7f8ddbfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc15e530 0x7f8ddc15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.345+0000 7f8ddbfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc13c790 con 0x7f8ddc15aa40 2024-05-08T23:31:11.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.345+0000 7f8ddbfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc13bb30 con 0x7f8ddc15e530 2024-05-08T23:31:11.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.345+0000 7f8ddbfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc13bdf0 con 0x7f8ddc1544a0 2024-05-08T23:31:11.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.345+0000 7f8dd27fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ddc15aa40 0x7f8ddc0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.345+0000 7f8dd27fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ddc15aa40 0x7f8ddc0cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58398/0 (socket says 172.21.15.81:58398) 2024-05-08T23:31:11.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.345+0000 7f8dd27fc700 1 -- 172.21.15.81:0/833183493 learned_addr learned my addr 172.21.15.81:0/833183493 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:11.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.345+0000 7f8ddaffd700 1 --2- 172.21.15.81:0/833183493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc1544a0 0x7f8ddc139060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.345+0000 7f8ddb7fe700 1 --2- 172.21.15.81:0/833183493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc15e530 0x7f8ddc15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.349+0000 7f8dd27fc700 1 -- 172.21.15.81:0/833183493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc1544a0 msgr2=0x7f8ddc139060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.349+0000 7f8dd27fc700 1 --2- 172.21.15.81:0/833183493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc1544a0 0x7f8ddc139060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.349+0000 7f8dd27fc700 1 -- 172.21.15.81:0/833183493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc15e530 msgr2=0x7f8ddc15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.349+0000 7f8dd27fc700 1 --2- 172.21.15.81:0/833183493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc15e530 0x7f8ddc15bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.349+0000 7f8dd27fc700 1 -- 172.21.15.81:0/833183493 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc0094b0 con 0x7f8ddc15aa40 2024-05-08T23:31:11.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.349+0000 7f8dd27fc700 1 --2- 172.21.15.81:0/833183493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ddc15aa40 0x7f8ddc0cca70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400c410 tx=0x7f8dc400c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.349+0000 7f8dd3fff700 1 -- 172.21.15.81:0/833183493 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc40052d0 con 0x7f8ddc15aa40 2024-05-08T23:31:11.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.349+0000 7f8dd3fff700 1 -- 172.21.15.81:0/833183493 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc4005430 con 0x7f8ddc15aa40 2024-05-08T23:31:11.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.349+0000 7f8ddbfff700 1 -- 172.21.15.81:0/833183493 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ddc13c0d0 con 0x7f8ddc15aa40 2024-05-08T23:31:11.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.349+0000 7f8dd3fff700 1 -- 172.21.15.81:0/833183493 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc40055a0 con 0x7f8ddc15aa40 2024-05-08T23:31:11.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.349+0000 7f8ddbfff700 1 -- 172.21.15.81:0/833183493 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc1395a0 con 0x7f8ddc15aa40 2024-05-08T23:31:11.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.349+0000 7f8ddbfff700 1 -- 172.21.15.81:0/833183493 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ddc13c8d0 con 0x7f8ddc15aa40 2024-05-08T23:31:11.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.349+0000 7f8dd3fff700 1 -- 172.21.15.81:0/833183493 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8dc40036a0 con 0x7f8ddc15aa40 2024-05-08T23:31:11.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.353+0000 7f8dd3fff700 1 --2- 172.21.15.81:0/833183493 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8dbc066730 0x7f8dbc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.353+0000 7f8dd3fff700 1 -- 172.21.15.81:0/833183493 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8dc4083630 con 0x7f8ddc15aa40 2024-05-08T23:31:11.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.353+0000 7f8ddaffd700 1 --2- 172.21.15.81:0/833183493 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8dbc066730 0x7f8dbc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.353+0000 7f8ddaffd700 1 --2- 172.21.15.81:0/833183493 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8dbc066730 0x7f8dbc068bf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009bd0 tx=0x7f8dcc010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.357+0000 7f8dd3fff700 1 -- 172.21.15.81:0/833183493 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8dc40509a0 con 0x7f8ddc15aa40 2024-05-08T23:31:11.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.525+0000 7f8ddbfff700 1 -- 172.21.15.81:0/833183493 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ddc13c8d0 con 0x7f8ddc15aa40 2024-05-08T23:31:11.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.525+0000 7f8dd3fff700 1 -- 172.21.15.81:0/833183493 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8ddc13c8d0 con 0x7f8ddc15aa40 2024-05-08T23:31:11.531 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:11.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.529+0000 7f8ddbfff700 1 -- 172.21.15.81:0/833183493 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8dbc066730 msgr2=0x7f8dbc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.529+0000 7f8ddbfff700 1 --2- 172.21.15.81:0/833183493 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8dbc066730 0x7f8dbc068bf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009bd0 tx=0x7f8dcc010000 comp rx=0 tx=0).stop 2024-05-08T23:31:11.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.529+0000 7f8ddbfff700 1 -- 172.21.15.81:0/833183493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ddc15aa40 msgr2=0x7f8ddc0cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.529+0000 7f8ddbfff700 1 --2- 172.21.15.81:0/833183493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ddc15aa40 0x7f8ddc0cca70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400c410 tx=0x7f8dc400c720 comp rx=0 tx=0).stop 2024-05-08T23:31:11.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.529+0000 7f8ddbfff700 1 -- 172.21.15.81:0/833183493 shutdown_connections 2024-05-08T23:31:11.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.529+0000 7f8ddbfff700 1 --2- 172.21.15.81:0/833183493 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8dbc066730 0x7f8dbc068bf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.529+0000 7f8ddbfff700 1 --2- 172.21.15.81:0/833183493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc15e530 0x7f8ddc15bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.529+0000 7f8ddbfff700 1 --2- 172.21.15.81:0/833183493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ddc15aa40 0x7f8ddc0cca70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.529+0000 7f8ddbfff700 1 --2- 172.21.15.81:0/833183493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc1544a0 0x7f8ddc139060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.529+0000 7f8ddbfff700 1 -- 172.21.15.81:0/833183493 >> 172.21.15.81:0/833183493 conn(0x7f8ddc0b1a60 msgr2=0x7f8ddc071f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:11.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.529+0000 7f8ddbfff700 1 -- 172.21.15.81:0/833183493 shutdown_connections 2024-05-08T23:31:11.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.529+0000 7f8ddbfff700 1 -- 172.21.15.81:0/833183493 wait complete. 2024-05-08T23:31:11.547 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:31:11.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.657+0000 7f6ff4d32700 1 Processor -- start 2024-05-08T23:31:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.657+0000 7f6ff4d32700 1 -- start start 2024-05-08T23:31:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.657+0000 7f6ff4d32700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff015e760 0x7f6ff00d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.657+0000 7f6ff4d32700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6ff00d1140 0x7f6ff00cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.657+0000 7f6ff4d32700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6ff00cef90 0x7f6ff00cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.657+0000 7f6ff4d32700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff015eb40 con 0x7f6ff00d1140 2024-05-08T23:31:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.657+0000 7f6ff4d32700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff00ce5c0 con 0x7f6ff00cef90 2024-05-08T23:31:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.657+0000 7f6ff4d32700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00d1e90 con 0x7f6ff015e760 2024-05-08T23:31:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.657+0000 7f6feeffd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6ff00d1140 0x7f6ff00cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.657+0000 7f6feeffd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6ff00d1140 0x7f6ff00cea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58414/0 (socket says 172.21.15.81:58414) 2024-05-08T23:31:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.657+0000 7f6feeffd700 1 -- 172.21.15.81:0/2880217289 learned_addr learned my addr 172.21.15.81:0/2880217289 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6feffff700 1 --2- 172.21.15.81:0/2880217289 >> v2:172.21.15.117:3300/0 conn(0x7f6ff00cef90 0x7f6ff00cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.657+0000 7f6fef7fe700 1 --2- 172.21.15.81:0/2880217289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff015e760 0x7f6ff00d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6feeffd700 1 -- 172.21.15.81:0/2880217289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff015e760 msgr2=0x7f6ff00d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6feeffd700 1 --2- 172.21.15.81:0/2880217289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff015e760 0x7f6ff00d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6feeffd700 1 -- 172.21.15.81:0/2880217289 >> v2:172.21.15.117:3300/0 conn(0x7f6ff00cef90 msgr2=0x7f6ff00cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6feeffd700 1 --2- 172.21.15.81:0/2880217289 >> v2:172.21.15.117:3300/0 conn(0x7f6ff00cef90 0x7f6ff00cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6feeffd700 1 -- 172.21.15.81:0/2880217289 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff00d2170 con 0x7f6ff00d1140 2024-05-08T23:31:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6fef7fe700 1 --2- 172.21.15.81:0/2880217289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff015e760 0x7f6ff00d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6feeffd700 1 --2- 172.21.15.81:0/2880217289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ff00d1140 0x7f6ff00cea50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6ff008cae0 tx=0x7f6fe4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8584941b7112d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6fee7fc700 1 -- 172.21.15.81:0/2880217289 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe400b8a0 con 0x7f6ff00d1140 2024-05-08T23:31:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6fee7fc700 1 -- 172.21.15.81:0/2880217289 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe400ba00 con 0x7f6ff00d1140 2024-05-08T23:31:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6fee7fc700 1 -- 172.21.15.81:0/2880217289 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe400bd90 con 0x7f6ff00d1140 2024-05-08T23:31:11.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 -- 172.21.15.81:0/2880217289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ff00d1140 msgr2=0x7f6ff00cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 --2- 172.21.15.81:0/2880217289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ff00d1140 0x7f6ff00cea50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6ff008cae0 tx=0x7f6fe4009bf0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 -- 172.21.15.81:0/2880217289 shutdown_connections 2024-05-08T23:31:11.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 --2- 172.21.15.81:0/2880217289 >> v2:172.21.15.117:3300/0 conn(0x7f6ff00cef90 0x7f6ff00cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 --2- 172.21.15.81:0/2880217289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ff00d1140 0x7f6ff00cea50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 --2- 172.21.15.81:0/2880217289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff015e760 0x7f6ff00d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 -- 172.21.15.81:0/2880217289 >> 172.21.15.81:0/2880217289 conn(0x7f6ff009c3b0 msgr2=0x7f6ff00d3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:11.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 -- 172.21.15.81:0/2880217289 shutdown_connections 2024-05-08T23:31:11.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 -- 172.21.15.81:0/2880217289 wait complete. 2024-05-08T23:31:11.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 Processor -- start 2024-05-08T23:31:11.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 -- start start 2024-05-08T23:31:11.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff00cef90 0x7f6ff0159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ff00d1140 0x7f6ff00cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff015e760 0x7f6ff015c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00cc810 con 0x7f6ff00d1140 2024-05-08T23:31:11.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00c8330 con 0x7f6ff00cef90 2024-05-08T23:31:11.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6ff4d32700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00cc530 con 0x7f6ff015e760 2024-05-08T23:31:11.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6feeffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ff00d1140 0x7f6ff00cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6feeffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ff00d1140 0x7f6ff00cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58426/0 (socket says 172.21.15.81:58426) 2024-05-08T23:31:11.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.661+0000 7f6feeffd700 1 -- 172.21.15.81:0/2341824289 learned_addr learned my addr 172.21.15.81:0/2341824289 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:11.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.665+0000 7f6fef7fe700 1 --2- 172.21.15.81:0/2341824289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff00cef90 0x7f6ff0159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.665+0000 7f6feffff700 1 --2- 172.21.15.81:0/2341824289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff015e760 0x7f6ff015c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.665+0000 7f6feeffd700 1 -- 172.21.15.81:0/2341824289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff015e760 msgr2=0x7f6ff015c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.665+0000 7f6feeffd700 1 --2- 172.21.15.81:0/2341824289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff015e760 0x7f6ff015c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.665+0000 7f6feeffd700 1 -- 172.21.15.81:0/2341824289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff00cef90 msgr2=0x7f6ff0159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.665+0000 7f6feeffd700 1 --2- 172.21.15.81:0/2341824289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff00cef90 0x7f6ff0159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.665+0000 7f6feeffd700 1 -- 172.21.15.81:0/2341824289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe4009570 con 0x7f6ff00d1140 2024-05-08T23:31:11.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.665+0000 7f6feeffd700 1 --2- 172.21.15.81:0/2341824289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ff00d1140 0x7f6ff00cbae0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400b0e0 tx=0x7f6fe4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.665+0000 7f6fd7fff700 1 -- 172.21.15.81:0/2341824289 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe40036c0 con 0x7f6ff00d1140 2024-05-08T23:31:11.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.665+0000 7f6fd7fff700 1 -- 172.21.15.81:0/2341824289 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe401f070 con 0x7f6ff00d1140 2024-05-08T23:31:11.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.665+0000 7f6fd7fff700 1 -- 172.21.15.81:0/2341824289 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4003a80 con 0x7f6ff00d1140 2024-05-08T23:31:11.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.665+0000 7f6ff4d32700 1 -- 172.21.15.81:0/2341824289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff00c7ce0 con 0x7f6ff00d1140 2024-05-08T23:31:11.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.665+0000 7f6ff4d32700 1 -- 172.21.15.81:0/2341824289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff0159e50 con 0x7f6ff00d1140 2024-05-08T23:31:11.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.665+0000 7f6ff4d32700 1 -- 172.21.15.81:0/2341824289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff00d2ad0 con 0x7f6ff00d1140 2024-05-08T23:31:11.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.665+0000 7f6fd7fff700 1 -- 172.21.15.81:0/2341824289 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6fe4021030 con 0x7f6ff00d1140 2024-05-08T23:31:11.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.669+0000 7f6fd7fff700 1 --2- 172.21.15.81:0/2341824289 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6fdc066730 0x7f6fdc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.669+0000 7f6fef7fe700 1 --2- 172.21.15.81:0/2341824289 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6fdc066730 0x7f6fdc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.669+0000 7f6fd7fff700 1 -- 172.21.15.81:0/2341824289 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fe40857d0 con 0x7f6ff00d1140 2024-05-08T23:31:11.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.669+0000 7f6fef7fe700 1 --2- 172.21.15.81:0/2341824289 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6fdc066730 0x7f6fdc068bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0006fd0 tx=0x7f6fe0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.677+0000 7f6fd7fff700 1 -- 172.21.15.81:0/2341824289 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6fe4052bb0 con 0x7f6ff00d1140 2024-05-08T23:31:11.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.857+0000 7f6ff4d32700 1 -- 172.21.15.81:0/2341824289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6ff00684b0 con 0x7f6ff00d1140 2024-05-08T23:31:11.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.857+0000 7f6fd7fff700 1 -- 172.21.15.81:0/2341824289 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6fe40561d0 con 0x7f6ff00d1140 2024-05-08T23:31:11.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.861+0000 7f6ff4d32700 1 -- 172.21.15.81:0/2341824289 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6fdc066730 msgr2=0x7f6fdc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.861+0000 7f6ff4d32700 1 --2- 172.21.15.81:0/2341824289 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6fdc066730 0x7f6fdc068bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0006fd0 tx=0x7f6fe0008040 comp rx=0 tx=0).stop 2024-05-08T23:31:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.861+0000 7f6ff4d32700 1 -- 172.21.15.81:0/2341824289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ff00d1140 msgr2=0x7f6ff00cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.861+0000 7f6ff4d32700 1 --2- 172.21.15.81:0/2341824289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ff00d1140 0x7f6ff00cbae0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400b0e0 tx=0x7f6fe4004770 comp rx=0 tx=0).stop 2024-05-08T23:31:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.861+0000 7f6ff4d32700 1 -- 172.21.15.81:0/2341824289 shutdown_connections 2024-05-08T23:31:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.861+0000 7f6ff4d32700 1 --2- 172.21.15.81:0/2341824289 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6fdc066730 0x7f6fdc068bf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.861+0000 7f6ff4d32700 1 --2- 172.21.15.81:0/2341824289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ff015e760 0x7f6ff015c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.861+0000 7f6ff4d32700 1 --2- 172.21.15.81:0/2341824289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ff00d1140 0x7f6ff00cbae0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.861+0000 7f6ff4d32700 1 --2- 172.21.15.81:0/2341824289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff00cef90 0x7f6ff0159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.861+0000 7f6ff4d32700 1 -- 172.21.15.81:0/2341824289 >> 172.21.15.81:0/2341824289 conn(0x7f6ff009c3b0 msgr2=0x7f6ff006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.861+0000 7f6ff4d32700 1 -- 172.21.15.81:0/2341824289 shutdown_connections 2024-05-08T23:31:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:11.861+0000 7f6ff4d32700 1 -- 172.21.15.81:0/2341824289 wait complete. 2024-05-08T23:31:11.868 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:31:11.879 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:31:11.879 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:31:11.885 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:12.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.041+0000 7f4bdcb82700 1 Processor -- start 2024-05-08T23:31:12.048 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.041+0000 7f4bdcb82700 1 -- start start 2024-05-08T23:31:12.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bdcb82700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bd8140130 0x7f4bd80b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bdcb82700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd80b4800 con 0x7f4bd815e430 2024-05-08T23:31:12.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bdcb82700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd8140510 con 0x7f4bd8142880 2024-05-08T23:31:12.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bdcb82700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80b3ed0 con 0x7f4bd8140130 2024-05-08T23:31:12.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bd6ffd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f4bd815e430 0x7f4bd80b1f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46806/0 (socket says 172.21.15.81:46806) 2024-05-08T23:31:12.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bd6ffd700 1 -- 172.21.15.81:0/3592813460 learned_addr learned my addr 172.21.15.81:0/3592813460 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:12.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bd77fe700 1 --2- 172.21.15.81:0/3592813460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bd8140130 0x7f4bd80b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bd67fc700 1 -- 172.21.15.81:0/3592813460 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721445231 0 0) 0x7f4bd80b4800 con 0x7f4bd815e430 2024-05-08T23:31:12.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bd67fc700 1 -- 172.21.15.81:0/3592813460 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc40035d0 con 0x7f4bd815e430 2024-05-08T23:31:12.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bd67fc700 1 -- 172.21.15.81:0/3592813460 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077271430 0 0) 0x7f4bd8140510 con 0x7f4bd8142880 2024-05-08T23:31:12.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bd67fc700 1 -- 172.21.15.81:0/3592813460 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bd80b4800 con 0x7f4bd8142880 2024-05-08T23:31:12.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bd77fe700 1 -- 172.21.15.81:0/3592813460 >> v1:172.21.15.117:6789/0 conn(0x7f4bd8142880 legacy=0x7f4bd80ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bd67fc700 1 -- 172.21.15.81:0/3592813460 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 823979106 0 0) 0x7f4bc40035d0 con 0x7f4bd815e430 2024-05-08T23:31:12.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bd77fe700 1 -- 172.21.15.81:0/3592813460 >> v1:172.21.15.81:6789/0 conn(0x7f4bd815e430 legacy=0x7f4bd80b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bd77fe700 1 -- 172.21.15.81:0/3592813460 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd80b41b0 con 0x7f4bd8140130 2024-05-08T23:31:12.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bd77fe700 1 --2- 172.21.15.81:0/3592813460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bd8140130 0x7f4bd80b4c90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009880 tx=0x7f4bc8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a314998caeafaef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bd67fc700 1 -- 172.21.15.81:0/3592813460 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00019f0 con 0x7f4bd8140130 2024-05-08T23:31:12.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bd67fc700 1 -- 172.21.15.81:0/3592813460 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc8004030 con 0x7f4bd8140130 2024-05-08T23:31:12.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bd67fc700 1 -- 172.21.15.81:0/3592813460 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80043a0 con 0x7f4bd8140130 2024-05-08T23:31:12.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bdcb82700 1 -- 172.21.15.81:0/3592813460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bd8140130 msgr2=0x7f4bd80b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bdcb82700 1 --2- 172.21.15.81:0/3592813460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bd8140130 0x7f4bd80b4c90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009880 tx=0x7f4bc8009b90 comp rx=0 tx=0).stop 2024-05-08T23:31:12.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bdcb82700 1 -- 172.21.15.81:0/3592813460 shutdown_connections 2024-05-08T23:31:12.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bdcb82700 1 --2- 172.21.15.81:0/3592813460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bd8140130 0x7f4bd80b4c90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bdcb82700 1 -- 172.21.15.81:0/3592813460 >> 172.21.15.81:0/3592813460 conn(0x7f4bd806c330 msgr2=0x7f4bd806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.045+0000 7f4bdcb82700 1 -- 172.21.15.81:0/3592813460 shutdown_connections 2024-05-08T23:31:12.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bdcb82700 1 -- 172.21.15.81:0/3592813460 wait complete. 2024-05-08T23:31:12.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bdcb82700 1 Processor -- start 2024-05-08T23:31:12.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bdcb82700 1 -- start start 2024-05-08T23:31:12.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bdcb82700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd8140130 0x7f4bd813d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bdcb82700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bd8142880 0x7f4bd80b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bdcb82700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bd815e430 0x7f4bd80b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bdcb82700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd81f6ee0 con 0x7f4bd8142880 2024-05-08T23:31:12.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bdcb82700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd813dd30 con 0x7f4bd8140130 2024-05-08T23:31:12.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bdcb82700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80af9e0 con 0x7f4bd815e430 2024-05-08T23:31:12.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bd6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bd8142880 0x7f4bd80b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bd6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bd8142880 0x7f4bd80b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58430/0 (socket says 172.21.15.81:58430) 2024-05-08T23:31:12.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bd6ffd700 1 -- 172.21.15.81:0/1542523723 learned_addr learned my addr 172.21.15.81:0/1542523723 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:12.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bd7fff700 1 --2- 172.21.15.81:0/1542523723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bd815e430 0x7f4bd80b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bd77fe700 1 --2- 172.21.15.81:0/1542523723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd8140130 0x7f4bd813d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bd6ffd700 1 -- 172.21.15.81:0/1542523723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bd815e430 msgr2=0x7f4bd80b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bd6ffd700 1 --2- 172.21.15.81:0/1542523723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bd815e430 0x7f4bd80b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bd6ffd700 1 -- 172.21.15.81:0/1542523723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd8140130 msgr2=0x7f4bd813d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bd6ffd700 1 --2- 172.21.15.81:0/1542523723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd8140130 0x7f4bd813d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bd6ffd700 1 -- 172.21.15.81:0/1542523723 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc80094b0 con 0x7f4bd8142880 2024-05-08T23:31:12.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bd77fe700 1 --2- 172.21.15.81:0/1542523723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd8140130 0x7f4bd813d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:12.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bd6ffd700 1 --2- 172.21.15.81:0/1542523723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bd8142880 0x7f4bd80b2800 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000cc20 tx=0x7f4bc000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bcffff700 1 -- 172.21.15.81:0/1542523723 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc000ac20 con 0x7f4bd8142880 2024-05-08T23:31:12.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bcffff700 1 -- 172.21.15.81:0/1542523723 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc000ad80 con 0x7f4bd8142880 2024-05-08T23:31:12.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bdcb82700 1 -- 172.21.15.81:0/1542523723 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd81f6a50 con 0x7f4bd8142880 2024-05-08T23:31:12.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bcffff700 1 -- 172.21.15.81:0/1542523723 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0007310 con 0x7f4bd8142880 2024-05-08T23:31:12.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.049+0000 7f4bdcb82700 1 -- 172.21.15.81:0/1542523723 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd81f6400 con 0x7f4bd8142880 2024-05-08T23:31:12.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.053+0000 7f4bcffff700 1 -- 172.21.15.81:0/1542523723 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4bc0007470 con 0x7f4bd8142880 2024-05-08T23:31:12.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.053+0000 7f4bdcb82700 1 -- 172.21.15.81:0/1542523723 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd81f6be0 con 0x7f4bd8142880 2024-05-08T23:31:12.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.053+0000 7f4bcffff700 1 --2- 172.21.15.81:0/1542523723 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4bb8066730 0x7f4bb8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.053+0000 7f4bcffff700 1 -- 172.21.15.81:0/1542523723 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bc0084050 con 0x7f4bd8142880 2024-05-08T23:31:12.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.053+0000 7f4bd77fe700 1 --2- 172.21.15.81:0/1542523723 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4bb8066730 0x7f4bb8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.053+0000 7f4bd77fe700 1 --2- 172.21.15.81:0/1542523723 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4bb8066730 0x7f4bb8068bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80090b0 tx=0x7f4bc8010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.057+0000 7f4bcffff700 1 -- 172.21.15.81:0/1542523723 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4bc00513c0 con 0x7f4bd8142880 2024-05-08T23:31:12.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.229+0000 7f4bdcb82700 1 -- 172.21.15.81:0/1542523723 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bd80684b0 con 0x7f4bd8142880 2024-05-08T23:31:12.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.229+0000 7f4bcffff700 1 -- 172.21.15.81:0/1542523723 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f4bc00549e0 con 0x7f4bd8142880 2024-05-08T23:31:12.235 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:12.236 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:12.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.233+0000 7f4bdcb82700 1 -- 172.21.15.81:0/1542523723 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4bb8066730 msgr2=0x7f4bb8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.233+0000 7f4bdcb82700 1 --2- 172.21.15.81:0/1542523723 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4bb8066730 0x7f4bb8068bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80090b0 tx=0x7f4bc8010000 comp rx=0 tx=0).stop 2024-05-08T23:31:12.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.233+0000 7f4bdcb82700 1 -- 172.21.15.81:0/1542523723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bd8142880 msgr2=0x7f4bd80b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.233+0000 7f4bdcb82700 1 --2- 172.21.15.81:0/1542523723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bd8142880 0x7f4bd80b2800 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000cc20 tx=0x7f4bc000a060 comp rx=0 tx=0).stop 2024-05-08T23:31:12.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.233+0000 7f4bdcb82700 1 -- 172.21.15.81:0/1542523723 shutdown_connections 2024-05-08T23:31:12.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.233+0000 7f4bdcb82700 1 --2- 172.21.15.81:0/1542523723 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4bb8066730 0x7f4bb8068bf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.233+0000 7f4bdcb82700 1 --2- 172.21.15.81:0/1542523723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bd815e430 0x7f4bd80b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.233+0000 7f4bdcb82700 1 --2- 172.21.15.81:0/1542523723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bd8142880 0x7f4bd80b2800 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.233+0000 7f4bdcb82700 1 --2- 172.21.15.81:0/1542523723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd8140130 0x7f4bd813d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.233+0000 7f4bdcb82700 1 -- 172.21.15.81:0/1542523723 >> 172.21.15.81:0/1542523723 conn(0x7f4bd806c330 msgr2=0x7f4bd80add20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.233+0000 7f4bdcb82700 1 -- 172.21.15.81:0/1542523723 shutdown_connections 2024-05-08T23:31:12.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.233+0000 7f4bdcb82700 1 -- 172.21.15.81:0/1542523723 wait complete. 2024-05-08T23:31:12.251 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:12.251 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:12.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.365+0000 7fa862fca700 1 Processor -- start 2024-05-08T23:31:12.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.365+0000 7fa862fca700 1 -- start start 2024-05-08T23:31:12.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa862fca700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa85c06d3a0 0x7fa85c0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa862fca700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa85c071280 0x7fa85c0b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa862fca700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85c070730 con 0x7fa85c159540 2024-05-08T23:31:12.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa862fca700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa85c070a10 con 0x7fa85c06d3a0 2024-05-08T23:31:12.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa862fca700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa85c070cf0 con 0x7fa85c071280 2024-05-08T23:31:12.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa8617c7700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa85c06d3a0 0x7fa85c0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa8617c7700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa85c06d3a0 0x7fa85c0700e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58436/0 (socket says 172.21.15.81:58436) 2024-05-08T23:31:12.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa8617c7700 1 -- 172.21.15.81:0/139792608 learned_addr learned my addr 172.21.15.81:0/139792608 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:12.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa8627c9700 1 --2- 172.21.15.81:0/139792608 >> v2:172.21.15.117:3300/0 conn(0x7fa85c071280 0x7fa85c0b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa8617c7700 1 -- 172.21.15.81:0/139792608 >> v2:172.21.15.117:3300/0 conn(0x7fa85c071280 msgr2=0x7fa85c0b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa8617c7700 1 --2- 172.21.15.81:0/139792608 >> v2:172.21.15.117:3300/0 conn(0x7fa85c071280 0x7fa85c0b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa8617c7700 1 -- 172.21.15.81:0/139792608 >> v1:172.21.15.117:6789/0 conn(0x7fa85c159540 legacy=0x7fa85c06cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa8617c7700 1 -- 172.21.15.81:0/139792608 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c070fd0 con 0x7fa85c06d3a0 2024-05-08T23:31:12.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa8617c7700 1 --2- 172.21.15.81:0/139792608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa85c06d3a0 0x7fa85c0700e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa85c070620 tx=0x7fa85000ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6d59d6ca95c182 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa860fc6700 1 -- 172.21.15.81:0/139792608 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85000cb70 con 0x7fa85c06d3a0 2024-05-08T23:31:12.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa860fc6700 1 -- 172.21.15.81:0/139792608 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85000ccd0 con 0x7fa85c06d3a0 2024-05-08T23:31:12.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.369+0000 7fa860fc6700 1 -- 172.21.15.81:0/139792608 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850018600 con 0x7fa85c06d3a0 2024-05-08T23:31:12.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.373+0000 7fa862fca700 1 -- 172.21.15.81:0/139792608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa85c06d3a0 msgr2=0x7fa85c0700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.377 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.373+0000 7fa862fca700 1 --2- 172.21.15.81:0/139792608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa85c06d3a0 0x7fa85c0700e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa85c070620 tx=0x7fa85000ad90 comp rx=0 tx=0).stop 2024-05-08T23:31:12.377 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.373+0000 7fa862fca700 1 -- 172.21.15.81:0/139792608 shutdown_connections 2024-05-08T23:31:12.377 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.373+0000 7fa862fca700 1 --2- 172.21.15.81:0/139792608 >> v2:172.21.15.117:3300/0 conn(0x7fa85c071280 0x7fa85c0b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.377 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.373+0000 7fa862fca700 1 --2- 172.21.15.81:0/139792608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa85c06d3a0 0x7fa85c0700e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.377 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.373+0000 7fa862fca700 1 -- 172.21.15.81:0/139792608 >> 172.21.15.81:0/139792608 conn(0x7fa85c0b0de0 msgr2=0x7fa85c0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.377 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.373+0000 7fa862fca700 1 -- 172.21.15.81:0/139792608 shutdown_connections 2024-05-08T23:31:12.377 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.373+0000 7fa862fca700 1 -- 172.21.15.81:0/139792608 wait complete. 2024-05-08T23:31:12.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.373+0000 7fa862fca700 1 Processor -- start 2024-05-08T23:31:12.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.373+0000 7fa862fca700 1 -- start start 2024-05-08T23:31:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa862fca700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c06d3a0 0x7fa85c15c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa862fca700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa85c071280 0x7fa85c0b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa862fca700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c159540 0x7fa85c1e40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa862fca700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c1ef710 con 0x7fa85c071280 2024-05-08T23:31:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa862fca700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c1ee7c0 con 0x7fa85c06d3a0 2024-05-08T23:31:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa862fca700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c1eea90 con 0x7fa85c159540 2024-05-08T23:31:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa8617c7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa85c071280 0x7fa85c0b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa8617c7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa85c071280 0x7fa85c0b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58444/0 (socket says 172.21.15.81:58444) 2024-05-08T23:31:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa8617c7700 1 -- 172.21.15.81:0/2259734251 learned_addr learned my addr 172.21.15.81:0/2259734251 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa861fc8700 1 --2- 172.21.15.81:0/2259734251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c06d3a0 0x7fa85c15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa8627c9700 1 --2- 172.21.15.81:0/2259734251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c159540 0x7fa85c1e40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa8617c7700 1 -- 172.21.15.81:0/2259734251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c159540 msgr2=0x7fa85c1e40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa8617c7700 1 --2- 172.21.15.81:0/2259734251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c159540 0x7fa85c1e40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa8617c7700 1 -- 172.21.15.81:0/2259734251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c06d3a0 msgr2=0x7fa85c15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa8617c7700 1 --2- 172.21.15.81:0/2259734251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c06d3a0 0x7fa85c15c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa8617c7700 1 -- 172.21.15.81:0/2259734251 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85000a710 con 0x7fa85c071280 2024-05-08T23:31:12.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa861fc8700 1 --2- 172.21.15.81:0/2259734251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c06d3a0 0x7fa85c15c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:12.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa8617c7700 1 --2- 172.21.15.81:0/2259734251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa85c071280 0x7fa85c0b2d10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa850000c00 tx=0x7fa850004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa84e7fc700 1 -- 172.21.15.81:0/2259734251 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8500038d0 con 0x7fa85c071280 2024-05-08T23:31:12.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa84e7fc700 1 -- 172.21.15.81:0/2259734251 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa850003a50 con 0x7fa85c071280 2024-05-08T23:31:12.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa862fca700 1 -- 172.21.15.81:0/2259734251 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85c1eed70 con 0x7fa85c071280 2024-05-08T23:31:12.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa84e7fc700 1 -- 172.21.15.81:0/2259734251 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850003e00 con 0x7fa85c071280 2024-05-08T23:31:12.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.377+0000 7fa862fca700 1 -- 172.21.15.81:0/2259734251 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85c1ef330 con 0x7fa85c071280 2024-05-08T23:31:12.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.381+0000 7fa862fca700 1 -- 172.21.15.81:0/2259734251 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85c0684b0 con 0x7fa85c071280 2024-05-08T23:31:12.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.381+0000 7fa84e7fc700 1 -- 172.21.15.81:0/2259734251 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa850018400 con 0x7fa85c071280 2024-05-08T23:31:12.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.381+0000 7fa84e7fc700 1 --2- 172.21.15.81:0/2259734251 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa840066730 0x7fa840068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.381+0000 7fa861fc8700 1 --2- 172.21.15.81:0/2259734251 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa840066730 0x7fa840068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.381+0000 7fa84e7fc700 1 -- 172.21.15.81:0/2259734251 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa85001f070 con 0x7fa85c071280 2024-05-08T23:31:12.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.381+0000 7fa861fc8700 1 --2- 172.21.15.81:0/2259734251 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa840066730 0x7fa840068bf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa858000f60 tx=0x7fa858009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.389 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.385+0000 7fa84e7fc700 1 -- 172.21.15.81:0/2259734251 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa850053d60 con 0x7fa85c071280 2024-05-08T23:31:12.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.553+0000 7fa862fca700 1 -- 172.21.15.81:0/2259734251 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa85c14e9a0 con 0x7fa85c071280 2024-05-08T23:31:12.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.553+0000 7fa84e7fc700 1 -- 172.21.15.81:0/2259734251 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa850057380 con 0x7fa85c071280 2024-05-08T23:31:12.559 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:12.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.557+0000 7fa862fca700 1 -- 172.21.15.81:0/2259734251 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa840066730 msgr2=0x7fa840068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.557+0000 7fa862fca700 1 --2- 172.21.15.81:0/2259734251 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa840066730 0x7fa840068bf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa858000f60 tx=0x7fa858009040 comp rx=0 tx=0).stop 2024-05-08T23:31:12.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.557+0000 7fa862fca700 1 -- 172.21.15.81:0/2259734251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa85c071280 msgr2=0x7fa85c0b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.557+0000 7fa862fca700 1 --2- 172.21.15.81:0/2259734251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa85c071280 0x7fa85c0b2d10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa850000c00 tx=0x7fa850004770 comp rx=0 tx=0).stop 2024-05-08T23:31:12.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.557+0000 7fa862fca700 1 -- 172.21.15.81:0/2259734251 shutdown_connections 2024-05-08T23:31:12.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.557+0000 7fa862fca700 1 --2- 172.21.15.81:0/2259734251 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa840066730 0x7fa840068bf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.557+0000 7fa862fca700 1 --2- 172.21.15.81:0/2259734251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa85c159540 0x7fa85c1e40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.557+0000 7fa862fca700 1 --2- 172.21.15.81:0/2259734251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa85c071280 0x7fa85c0b2d10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.557+0000 7fa862fca700 1 --2- 172.21.15.81:0/2259734251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85c06d3a0 0x7fa85c15c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.557+0000 7fa862fca700 1 -- 172.21.15.81:0/2259734251 >> 172.21.15.81:0/2259734251 conn(0x7fa85c0b0de0 msgr2=0x7fa85c14f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.557+0000 7fa862fca700 1 -- 172.21.15.81:0/2259734251 shutdown_connections 2024-05-08T23:31:12.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.557+0000 7fa862fca700 1 -- 172.21.15.81:0/2259734251 wait complete. 2024-05-08T23:31:12.574 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:12.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.689+0000 7faa30dcc700 1 Processor -- start 2024-05-08T23:31:12.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.689+0000 7faa30dcc700 1 -- start start 2024-05-08T23:31:12.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.689+0000 7faa30dcc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faa2c157b30 0x7faa2c15e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.689+0000 7faa30dcc700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7faa2c0adf30 0x7faa2c0b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.689+0000 7faa30dcc700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2c15a350 con 0x7faa2c0abda0 2024-05-08T23:31:12.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.689+0000 7faa30dcc700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2c15e980 con 0x7faa2c0adf30 2024-05-08T23:31:12.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.689+0000 7faa30dcc700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2c15b370 con 0x7faa2c157b30 2024-05-08T23:31:12.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.689+0000 7faa2bfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7faa2c0adf30 0x7faa2c0b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.689+0000 7faa2affd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7faa2c0abda0 0x7faa2c0ad820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46808/0 (socket says 172.21.15.81:46808) 2024-05-08T23:31:12.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.689+0000 7faa2affd700 1 -- 172.21.15.81:0/569646328 learned_addr learned my addr 172.21.15.81:0/569646328 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:12.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.689+0000 7faa2b7fe700 1 --2- 172.21.15.81:0/569646328 >> v2:172.21.15.117:3300/0 conn(0x7faa2c157b30 0x7faa2c15e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.689+0000 7faa2bfff700 1 -- 172.21.15.81:0/569646328 >> v2:172.21.15.117:3300/0 conn(0x7faa2c157b30 msgr2=0x7faa2c15e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.689+0000 7faa2a7fc700 1 -- 172.21.15.81:0/569646328 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772879882 0 0) 0x7faa2c15a350 con 0x7faa2c0abda0 2024-05-08T23:31:12.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa2bfff700 1 --2- 172.21.15.81:0/569646328 >> v2:172.21.15.117:3300/0 conn(0x7faa2c157b30 0x7faa2c15e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa2bfff700 1 -- 172.21.15.81:0/569646328 >> v1:172.21.15.81:6789/0 conn(0x7faa2c0abda0 legacy=0x7faa2c0ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa2bfff700 1 -- 172.21.15.81:0/569646328 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c15a5b0 con 0x7faa2c0adf30 2024-05-08T23:31:12.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa2b7fe700 1 --2- 172.21.15.81:0/569646328 >> v2:172.21.15.117:3300/0 conn(0x7faa2c157b30 0x7faa2c15e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:12.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa2bfff700 1 --2- 172.21.15.81:0/569646328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa2c0adf30 0x7faa2c0b2fe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faa2c06e4a0 tx=0x7faa1c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c857465a961d27 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa2a7fc700 1 -- 172.21.15.81:0/569646328 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c011830 con 0x7faa2c0adf30 2024-05-08T23:31:12.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa2a7fc700 1 -- 172.21.15.81:0/569646328 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c011990 con 0x7faa2c0adf30 2024-05-08T23:31:12.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa2a7fc700 1 -- 172.21.15.81:0/569646328 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c011d40 con 0x7faa2c0adf30 2024-05-08T23:31:12.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa30dcc700 1 -- 172.21.15.81:0/569646328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa2c0adf30 msgr2=0x7faa2c0b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa30dcc700 1 --2- 172.21.15.81:0/569646328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa2c0adf30 0x7faa2c0b2fe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faa2c06e4a0 tx=0x7faa1c009b30 comp rx=0 tx=0).stop 2024-05-08T23:31:12.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa30dcc700 1 -- 172.21.15.81:0/569646328 shutdown_connections 2024-05-08T23:31:12.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa30dcc700 1 --2- 172.21.15.81:0/569646328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa2c0adf30 0x7faa2c0b2fe0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa30dcc700 1 --2- 172.21.15.81:0/569646328 >> v2:172.21.15.117:3300/0 conn(0x7faa2c157b30 0x7faa2c15e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa30dcc700 1 -- 172.21.15.81:0/569646328 >> 172.21.15.81:0/569646328 conn(0x7faa2c0b0de0 msgr2=0x7faa2c0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa30dcc700 1 -- 172.21.15.81:0/569646328 shutdown_connections 2024-05-08T23:31:12.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa30dcc700 1 -- 172.21.15.81:0/569646328 wait complete. 2024-05-08T23:31:12.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa30dcc700 1 Processor -- start 2024-05-08T23:31:12.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.693+0000 7faa30dcc700 1 -- start start 2024-05-08T23:31:12.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa30dcc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c0abda0 0x7faa2c1549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa30dcc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa2c0adf30 0x7faa2c0b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa30dcc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa2c157b30 0x7faa2c159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa30dcc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c09aa60 con 0x7faa2c0adf30 2024-05-08T23:31:12.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa30dcc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c099ee0 con 0x7faa2c0abda0 2024-05-08T23:31:12.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa30dcc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c09a1c0 con 0x7faa2c157b30 2024-05-08T23:31:12.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa2affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa2c0adf30 0x7faa2c0b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa2affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa2c0adf30 0x7faa2c0b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58456/0 (socket says 172.21.15.81:58456) 2024-05-08T23:31:12.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa2affd700 1 -- 172.21.15.81:0/3879880688 learned_addr learned my addr 172.21.15.81:0/3879880688 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:12.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa2b7fe700 1 --2- 172.21.15.81:0/3879880688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c0abda0 0x7faa2c1549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa2bfff700 1 --2- 172.21.15.81:0/3879880688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa2c157b30 0x7faa2c159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa2affd700 1 -- 172.21.15.81:0/3879880688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa2c157b30 msgr2=0x7faa2c159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa2affd700 1 --2- 172.21.15.81:0/3879880688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa2c157b30 0x7faa2c159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa2affd700 1 -- 172.21.15.81:0/3879880688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c0abda0 msgr2=0x7faa2c1549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa2affd700 1 --2- 172.21.15.81:0/3879880688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c0abda0 0x7faa2c1549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa2affd700 1 -- 172.21.15.81:0/3879880688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c0094b0 con 0x7faa2c0adf30 2024-05-08T23:31:12.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa2b7fe700 1 --2- 172.21.15.81:0/3879880688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c0abda0 0x7faa2c1549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:12.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa2affd700 1 --2- 172.21.15.81:0/3879880688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa2c0adf30 0x7faa2c0b2640 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faa24001f80 tx=0x7faa2400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa23fff700 1 -- 172.21.15.81:0/3879880688 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2400acd0 con 0x7faa2c0adf30 2024-05-08T23:31:12.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa23fff700 1 -- 172.21.15.81:0/3879880688 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2400ae50 con 0x7faa2c0adf30 2024-05-08T23:31:12.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa30dcc700 1 -- 172.21.15.81:0/3879880688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2c09a4a0 con 0x7faa2c0adf30 2024-05-08T23:31:12.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa23fff700 1 -- 172.21.15.81:0/3879880688 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa24007310 con 0x7faa2c0adf30 2024-05-08T23:31:12.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa30dcc700 1 -- 172.21.15.81:0/3879880688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c099a70 con 0x7faa2c0adf30 2024-05-08T23:31:12.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.697+0000 7faa30dcc700 1 -- 172.21.15.81:0/3879880688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2c0684b0 con 0x7faa2c0adf30 2024-05-08T23:31:12.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.701+0000 7faa23fff700 1 -- 172.21.15.81:0/3879880688 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faa24007470 con 0x7faa2c0adf30 2024-05-08T23:31:12.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.701+0000 7faa23fff700 1 --2- 172.21.15.81:0/3879880688 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa0c066730 0x7faa0c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.701+0000 7faa2b7fe700 1 --2- 172.21.15.81:0/3879880688 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa0c066730 0x7faa0c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.701+0000 7faa23fff700 1 -- 172.21.15.81:0/3879880688 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa24083ef0 con 0x7faa2c0adf30 2024-05-08T23:31:12.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.701+0000 7faa2b7fe700 1 --2- 172.21.15.81:0/3879880688 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa0c066730 0x7faa0c068bf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faa1400af30 tx=0x7faa14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.705+0000 7faa23fff700 1 -- 172.21.15.81:0/3879880688 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faa240512f0 con 0x7faa2c0adf30 2024-05-08T23:31:12.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.877+0000 7faa30dcc700 1 -- 172.21.15.81:0/3879880688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faa2c0b00d0 con 0x7faa2c0adf30 2024-05-08T23:31:12.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.877+0000 7faa23fff700 1 -- 172.21.15.81:0/3879880688 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7faa24018020 con 0x7faa2c0adf30 2024-05-08T23:31:12.882 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.881+0000 7faa30dcc700 1 -- 172.21.15.81:0/3879880688 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa0c066730 msgr2=0x7faa0c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.881+0000 7faa30dcc700 1 --2- 172.21.15.81:0/3879880688 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa0c066730 0x7faa0c068bf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faa1400af30 tx=0x7faa14008040 comp rx=0 tx=0).stop 2024-05-08T23:31:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.881+0000 7faa30dcc700 1 -- 172.21.15.81:0/3879880688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa2c0adf30 msgr2=0x7faa2c0b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.881+0000 7faa30dcc700 1 --2- 172.21.15.81:0/3879880688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa2c0adf30 0x7faa2c0b2640 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faa24001f80 tx=0x7faa2400a340 comp rx=0 tx=0).stop 2024-05-08T23:31:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.881+0000 7faa30dcc700 1 -- 172.21.15.81:0/3879880688 shutdown_connections 2024-05-08T23:31:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.881+0000 7faa30dcc700 1 --2- 172.21.15.81:0/3879880688 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa0c066730 0x7faa0c068bf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.881+0000 7faa30dcc700 1 --2- 172.21.15.81:0/3879880688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa2c157b30 0x7faa2c159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.881+0000 7faa30dcc700 1 --2- 172.21.15.81:0/3879880688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa2c0adf30 0x7faa2c0b2640 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.881+0000 7faa30dcc700 1 --2- 172.21.15.81:0/3879880688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c0abda0 0x7faa2c1549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.881+0000 7faa30dcc700 1 -- 172.21.15.81:0/3879880688 >> 172.21.15.81:0/3879880688 conn(0x7faa2c0b0de0 msgr2=0x7faa2c0ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.881+0000 7faa30dcc700 1 -- 172.21.15.81:0/3879880688 shutdown_connections 2024-05-08T23:31:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:12.881+0000 7faa30dcc700 1 -- 172.21.15.81:0/3879880688 wait complete. 2024-05-08T23:31:12.896 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:13.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.013+0000 7fdb44905700 1 Processor -- start 2024-05-08T23:31:13.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.013+0000 7fdb44905700 1 -- start start 2024-05-08T23:31:13.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb44905700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdb40160920 0x7fdb400ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb44905700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400d1da0 0x7fdb400cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb44905700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdb400cec30 0x7fdb400cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb44905700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb400d3d40 con 0x7fdb400cec30 2024-05-08T23:31:13.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb44905700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb400d0d00 con 0x7fdb40160920 2024-05-08T23:31:13.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb44905700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb400d0fe0 con 0x7fdb400d1da0 2024-05-08T23:31:13.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb3f59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdb400cec30 0x7fdb400cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb3f59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdb400cec30 0x7fdb400cfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58470/0 (socket says 172.21.15.81:58470) 2024-05-08T23:31:13.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb3f59e700 1 -- 172.21.15.81:0/4195767646 learned_addr learned my addr 172.21.15.81:0/4195767646 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:13.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb37fff700 1 --2- 172.21.15.81:0/4195767646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400d1da0 0x7fdb400cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb3ed9d700 1 --2- 172.21.15.81:0/4195767646 >> v2:172.21.15.117:3300/0 conn(0x7fdb40160920 0x7fdb400ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb3f59e700 1 -- 172.21.15.81:0/4195767646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400d1da0 msgr2=0x7fdb400cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb3f59e700 1 --2- 172.21.15.81:0/4195767646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400d1da0 0x7fdb400cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb3f59e700 1 -- 172.21.15.81:0/4195767646 >> v2:172.21.15.117:3300/0 conn(0x7fdb40160920 msgr2=0x7fdb400ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb3f59e700 1 --2- 172.21.15.81:0/4195767646 >> v2:172.21.15.117:3300/0 conn(0x7fdb40160920 0x7fdb400ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb3f59e700 1 -- 172.21.15.81:0/4195767646 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb400d12c0 con 0x7fdb400cec30 2024-05-08T23:31:13.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb3ed9d700 1 --2- 172.21.15.81:0/4195767646 >> v2:172.21.15.117:3300/0 conn(0x7fdb40160920 0x7fdb400ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:13.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb3f59e700 1 --2- 172.21.15.81:0/4195767646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb400cec30 0x7fdb400cfe80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdb4008c490 tx=0x7fdb3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2df4243338021b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb3e59c700 1 -- 172.21.15.81:0/4195767646 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3800e070 con 0x7fdb400cec30 2024-05-08T23:31:13.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb3e59c700 1 -- 172.21.15.81:0/4195767646 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3800b9c0 con 0x7fdb400cec30 2024-05-08T23:31:13.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb3e59c700 1 -- 172.21.15.81:0/4195767646 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3800bd70 con 0x7fdb400cec30 2024-05-08T23:31:13.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb44905700 1 -- 172.21.15.81:0/4195767646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb400cec30 msgr2=0x7fdb400cfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb44905700 1 --2- 172.21.15.81:0/4195767646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb400cec30 0x7fdb400cfe80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdb4008c490 tx=0x7fdb3800b330 comp rx=0 tx=0).stop 2024-05-08T23:31:13.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb44905700 1 -- 172.21.15.81:0/4195767646 shutdown_connections 2024-05-08T23:31:13.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb44905700 1 --2- 172.21.15.81:0/4195767646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400d1da0 0x7fdb400cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb44905700 1 --2- 172.21.15.81:0/4195767646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb400cec30 0x7fdb400cfe80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb44905700 1 --2- 172.21.15.81:0/4195767646 >> v2:172.21.15.117:3300/0 conn(0x7fdb40160920 0x7fdb400ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb44905700 1 -- 172.21.15.81:0/4195767646 >> 172.21.15.81:0/4195767646 conn(0x7fdb4009c340 msgr2=0x7fdb400d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb44905700 1 -- 172.21.15.81:0/4195767646 shutdown_connections 2024-05-08T23:31:13.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.017+0000 7fdb44905700 1 -- 172.21.15.81:0/4195767646 wait complete. 2024-05-08T23:31:13.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.021+0000 7fdb44905700 1 Processor -- start 2024-05-08T23:31:13.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.021+0000 7fdb44905700 1 -- start start 2024-05-08T23:31:13.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.021+0000 7fdb44905700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb400cec30 0x7fdb400a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.021+0000 7fdb44905700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400d1da0 0x7fdb400b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.021+0000 7fdb44905700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb40160920 0x7fdb400b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.021+0000 7fdb44905700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4015ee50 con 0x7fdb40160920 2024-05-08T23:31:13.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.021+0000 7fdb44905700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4014c560 con 0x7fdb400cec30 2024-05-08T23:31:13.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.021+0000 7fdb44905700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4014c840 con 0x7fdb400d1da0 2024-05-08T23:31:13.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.021+0000 7fdb3f59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb40160920 0x7fdb400b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.021+0000 7fdb3f59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb40160920 0x7fdb400b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58482/0 (socket says 172.21.15.81:58482) 2024-05-08T23:31:13.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.021+0000 7fdb3f59e700 1 -- 172.21.15.81:0/4271140609 learned_addr learned my addr 172.21.15.81:0/4271140609 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:13.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.021+0000 7fdb3ed9d700 1 --2- 172.21.15.81:0/4271140609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb400cec30 0x7fdb400a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.021+0000 7fdb37fff700 1 --2- 172.21.15.81:0/4271140609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400d1da0 0x7fdb400b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.025+0000 7fdb3f59e700 1 -- 172.21.15.81:0/4271140609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400d1da0 msgr2=0x7fdb400b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.025+0000 7fdb3f59e700 1 --2- 172.21.15.81:0/4271140609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400d1da0 0x7fdb400b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.025+0000 7fdb3f59e700 1 -- 172.21.15.81:0/4271140609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb400cec30 msgr2=0x7fdb400a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.025+0000 7fdb3f59e700 1 --2- 172.21.15.81:0/4271140609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb400cec30 0x7fdb400a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.025+0000 7fdb3f59e700 1 -- 172.21.15.81:0/4271140609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb38009b30 con 0x7fdb40160920 2024-05-08T23:31:13.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.025+0000 7fdb37fff700 1 --2- 172.21.15.81:0/4271140609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400d1da0 0x7fdb400b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:13.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.025+0000 7fdb3f59e700 1 --2- 172.21.15.81:0/4271140609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb40160920 0x7fdb400b28a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdb4008bfd0 tx=0x7fdb38004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.025+0000 7fdb377fe700 1 -- 172.21.15.81:0/4271140609 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380099b0 con 0x7fdb40160920 2024-05-08T23:31:13.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.025+0000 7fdb377fe700 1 -- 172.21.15.81:0/4271140609 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb38004250 con 0x7fdb40160920 2024-05-08T23:31:13.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.025+0000 7fdb377fe700 1 -- 172.21.15.81:0/4271140609 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380045e0 con 0x7fdb40160920 2024-05-08T23:31:13.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.025+0000 7fdb44905700 1 -- 172.21.15.81:0/4271140609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb40160e80 con 0x7fdb40160920 2024-05-08T23:31:13.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.025+0000 7fdb44905700 1 -- 172.21.15.81:0/4271140609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb400a0ba0 con 0x7fdb40160920 2024-05-08T23:31:13.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.025+0000 7fdb44905700 1 -- 172.21.15.81:0/4271140609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4014cd60 con 0x7fdb40160920 2024-05-08T23:31:13.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.029+0000 7fdb377fe700 1 -- 172.21.15.81:0/4271140609 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdb38004740 con 0x7fdb40160920 2024-05-08T23:31:13.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.029+0000 7fdb377fe700 1 --2- 172.21.15.81:0/4271140609 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdb24066730 0x7fdb24068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.029+0000 7fdb377fe700 1 -- 172.21.15.81:0/4271140609 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb380855f0 con 0x7fdb40160920 2024-05-08T23:31:13.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.029+0000 7fdb3ed9d700 1 --2- 172.21.15.81:0/4271140609 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdb24066730 0x7fdb24068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.029+0000 7fdb3ed9d700 1 --2- 172.21.15.81:0/4271140609 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdb24066730 0x7fdb24068bf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdb300044b0 tx=0x7fdb30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.037+0000 7fdb377fe700 1 -- 172.21.15.81:0/4271140609 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdb4014cd60 con 0x7fdb40160920 2024-05-08T23:31:13.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.201+0000 7fdb44905700 1 -- 172.21.15.81:0/4271140609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb4014cd60 con 0x7fdb40160920 2024-05-08T23:31:13.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.201+0000 7fdb377fe700 1 -- 172.21.15.81:0/4271140609 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb4014cd60 con 0x7fdb40160920 2024-05-08T23:31:13.207 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:13.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.205+0000 7fdb44905700 1 -- 172.21.15.81:0/4271140609 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdb24066730 msgr2=0x7fdb24068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.205+0000 7fdb44905700 1 --2- 172.21.15.81:0/4271140609 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdb24066730 0x7fdb24068bf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdb300044b0 tx=0x7fdb30008040 comp rx=0 tx=0).stop 2024-05-08T23:31:13.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.205+0000 7fdb44905700 1 -- 172.21.15.81:0/4271140609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb40160920 msgr2=0x7fdb400b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.205+0000 7fdb44905700 1 --2- 172.21.15.81:0/4271140609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb40160920 0x7fdb400b28a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdb4008bfd0 tx=0x7fdb38004af0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.205+0000 7fdb44905700 1 -- 172.21.15.81:0/4271140609 shutdown_connections 2024-05-08T23:31:13.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.205+0000 7fdb44905700 1 --2- 172.21.15.81:0/4271140609 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdb24066730 0x7fdb24068bf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.205+0000 7fdb44905700 1 --2- 172.21.15.81:0/4271140609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb40160920 0x7fdb400b28a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.205+0000 7fdb44905700 1 --2- 172.21.15.81:0/4271140609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb400d1da0 0x7fdb400b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.205+0000 7fdb44905700 1 --2- 172.21.15.81:0/4271140609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb400cec30 0x7fdb400a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.205+0000 7fdb44905700 1 -- 172.21.15.81:0/4271140609 >> 172.21.15.81:0/4271140609 conn(0x7fdb4009c340 msgr2=0x7fdb400cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.205+0000 7fdb44905700 1 -- 172.21.15.81:0/4271140609 shutdown_connections 2024-05-08T23:31:13.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.205+0000 7fdb44905700 1 -- 172.21.15.81:0/4271140609 wait complete. 2024-05-08T23:31:13.222 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:31:13.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb8a026700 1 Processor -- start 2024-05-08T23:31:13.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb8a026700 1 -- start start 2024-05-08T23:31:13.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb8a026700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbb840ce9d0 0x7fbb840cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb8a026700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb840cee40 0x7fbb840cf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb8a026700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb840cd4b0 con 0x7fbb8415b180 2024-05-08T23:31:13.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb8a026700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb840cc5c0 con 0x7fbb840cee40 2024-05-08T23:31:13.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb8a026700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb840cc8a0 con 0x7fbb840ce9d0 2024-05-08T23:31:13.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb89825700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb840cee40 0x7fbb840cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb89825700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb840cee40 0x7fbb840cf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58494/0 (socket says 172.21.15.81:58494) 2024-05-08T23:31:13.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb89825700 1 -- 172.21.15.81:0/4210483226 learned_addr learned my addr 172.21.15.81:0/4210483226 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:13.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb88823700 1 --2- 172.21.15.81:0/4210483226 >> v2:172.21.15.117:3300/0 conn(0x7fbb840ce9d0 0x7fbb840cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb89825700 1 -- 172.21.15.81:0/4210483226 >> v2:172.21.15.117:3300/0 conn(0x7fbb840ce9d0 msgr2=0x7fbb840cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb89825700 1 --2- 172.21.15.81:0/4210483226 >> v2:172.21.15.117:3300/0 conn(0x7fbb840ce9d0 0x7fbb840cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb89825700 1 -- 172.21.15.81:0/4210483226 >> v1:172.21.15.117:6789/0 conn(0x7fbb8415b180 legacy=0x7fbb840b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb89825700 1 -- 172.21.15.81:0/4210483226 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb840ccb80 con 0x7fbb840cee40 2024-05-08T23:31:13.340 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb89825700 1 --2- 172.21.15.81:0/4210483226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb840cee40 0x7fbb840cf920 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbb840c4480 tx=0x7fbb7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0df61886f7ed42b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb7bfff700 1 -- 172.21.15.81:0/4210483226 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c00e070 con 0x7fbb840cee40 2024-05-08T23:31:13.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb7bfff700 1 -- 172.21.15.81:0/4210483226 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7c00b910 con 0x7fbb840cee40 2024-05-08T23:31:13.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.333+0000 7fbb7bfff700 1 -- 172.21.15.81:0/4210483226 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c00bcc0 con 0x7fbb840cee40 2024-05-08T23:31:13.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.337+0000 7fbb8a026700 1 -- 172.21.15.81:0/4210483226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb840cee40 msgr2=0x7fbb840cf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.337+0000 7fbb8a026700 1 --2- 172.21.15.81:0/4210483226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb840cee40 0x7fbb840cf920 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbb840c4480 tx=0x7fbb7c00b330 comp rx=0 tx=0).stop 2024-05-08T23:31:13.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.337+0000 7fbb8a026700 1 -- 172.21.15.81:0/4210483226 shutdown_connections 2024-05-08T23:31:13.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.337+0000 7fbb8a026700 1 --2- 172.21.15.81:0/4210483226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb840cee40 0x7fbb840cf920 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.337+0000 7fbb8a026700 1 --2- 172.21.15.81:0/4210483226 >> v2:172.21.15.117:3300/0 conn(0x7fbb840ce9d0 0x7fbb840cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.337+0000 7fbb8a026700 1 -- 172.21.15.81:0/4210483226 >> 172.21.15.81:0/4210483226 conn(0x7fbb840b2970 msgr2=0x7fbb840b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.337+0000 7fbb8a026700 1 -- 172.21.15.81:0/4210483226 shutdown_connections 2024-05-08T23:31:13.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.337+0000 7fbb8a026700 1 -- 172.21.15.81:0/4210483226 wait complete. 2024-05-08T23:31:13.342 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.337+0000 7fbb8a026700 1 Processor -- start 2024-05-08T23:31:13.342 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.337+0000 7fbb8a026700 1 -- start start 2024-05-08T23:31:13.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.337+0000 7fbb8a026700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb840ce9d0 0x7fbb841f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.337+0000 7fbb8a026700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb840cee40 0x7fbb841ee050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb8a026700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb8415b180 0x7fbb8415db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb8a026700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb840b49c0 con 0x7fbb840ce9d0 2024-05-08T23:31:13.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb8a026700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8415b690 con 0x7fbb840cee40 2024-05-08T23:31:13.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb8a026700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8415b970 con 0x7fbb8415b180 2024-05-08T23:31:13.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb89024700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb840ce9d0 0x7fbb841f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb89024700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb840ce9d0 0x7fbb841f1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58500/0 (socket says 172.21.15.81:58500) 2024-05-08T23:31:13.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb88823700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb840cee40 0x7fbb841ee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb89024700 1 -- 172.21.15.81:0/573065382 learned_addr learned my addr 172.21.15.81:0/573065382 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:13.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb89825700 1 --2- 172.21.15.81:0/573065382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb8415b180 0x7fbb8415db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb89024700 1 -- 172.21.15.81:0/573065382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb8415b180 msgr2=0x7fbb8415db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb89024700 1 --2- 172.21.15.81:0/573065382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb8415b180 0x7fbb8415db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb89024700 1 -- 172.21.15.81:0/573065382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb840cee40 msgr2=0x7fbb841ee050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb89024700 1 --2- 172.21.15.81:0/573065382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb840cee40 0x7fbb841ee050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb89024700 1 -- 172.21.15.81:0/573065382 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c009b30 con 0x7fbb840ce9d0 2024-05-08T23:31:13.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb89024700 1 --2- 172.21.15.81:0/573065382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb840ce9d0 0x7fbb841f1da0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000c500 tx=0x7fbb8000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb79ffb700 1 -- 172.21.15.81:0/573065382 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb800052d0 con 0x7fbb840ce9d0 2024-05-08T23:31:13.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb79ffb700 1 -- 172.21.15.81:0/573065382 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb80005430 con 0x7fbb840ce9d0 2024-05-08T23:31:13.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb8a026700 1 -- 172.21.15.81:0/573065382 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8415bc50 con 0x7fbb840ce9d0 2024-05-08T23:31:13.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb79ffb700 1 -- 172.21.15.81:0/573065382 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb800055a0 con 0x7fbb840ce9d0 2024-05-08T23:31:13.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb8a026700 1 -- 172.21.15.81:0/573065382 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb841f22e0 con 0x7fbb840ce9d0 2024-05-08T23:31:13.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.341+0000 7fbb8a026700 1 -- 172.21.15.81:0/573065382 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8415c170 con 0x7fbb840ce9d0 2024-05-08T23:31:13.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.345+0000 7fbb79ffb700 1 -- 172.21.15.81:0/573065382 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbb800038e0 con 0x7fbb840ce9d0 2024-05-08T23:31:13.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.345+0000 7fbb79ffb700 1 --2- 172.21.15.81:0/573065382 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbb68066730 0x7fbb68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.345+0000 7fbb79ffb700 1 -- 172.21.15.81:0/573065382 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb80083480 con 0x7fbb840ce9d0 2024-05-08T23:31:13.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.345+0000 7fbb88823700 1 --2- 172.21.15.81:0/573065382 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbb68066730 0x7fbb68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.345+0000 7fbb88823700 1 --2- 172.21.15.81:0/573065382 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbb68066730 0x7fbb68068bf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009bf0 tx=0x7fbb74009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.353+0000 7fbb79ffb700 1 -- 172.21.15.81:0/573065382 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbb8415c170 con 0x7fbb840ce9d0 2024-05-08T23:31:13.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.533+0000 7fbb8a026700 1 -- 172.21.15.81:0/573065382 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbb8415c170 con 0x7fbb840ce9d0 2024-05-08T23:31:13.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.537+0000 7fbb79ffb700 1 -- 172.21.15.81:0/573065382 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbb8415c170 con 0x7fbb840ce9d0 2024-05-08T23:31:13.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.537+0000 7fbb8a026700 1 -- 172.21.15.81:0/573065382 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbb68066730 msgr2=0x7fbb68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.537+0000 7fbb8a026700 1 --2- 172.21.15.81:0/573065382 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbb68066730 0x7fbb68068bf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009bf0 tx=0x7fbb74009240 comp rx=0 tx=0).stop 2024-05-08T23:31:13.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.537+0000 7fbb8a026700 1 -- 172.21.15.81:0/573065382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb840ce9d0 msgr2=0x7fbb841f1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.537+0000 7fbb8a026700 1 --2- 172.21.15.81:0/573065382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb840ce9d0 0x7fbb841f1da0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000c500 tx=0x7fbb8000c810 comp rx=0 tx=0).stop 2024-05-08T23:31:13.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.537+0000 7fbb8a026700 1 -- 172.21.15.81:0/573065382 shutdown_connections 2024-05-08T23:31:13.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.537+0000 7fbb8a026700 1 --2- 172.21.15.81:0/573065382 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbb68066730 0x7fbb68068bf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.537+0000 7fbb8a026700 1 --2- 172.21.15.81:0/573065382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb8415b180 0x7fbb8415db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.537+0000 7fbb8a026700 1 --2- 172.21.15.81:0/573065382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb840cee40 0x7fbb841ee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.537+0000 7fbb8a026700 1 --2- 172.21.15.81:0/573065382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb840ce9d0 0x7fbb841f1da0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.537+0000 7fbb8a026700 1 -- 172.21.15.81:0/573065382 >> 172.21.15.81:0/573065382 conn(0x7fbb840b2970 msgr2=0x7fbb8406e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.537+0000 7fbb8a026700 1 -- 172.21.15.81:0/573065382 shutdown_connections 2024-05-08T23:31:13.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.537+0000 7fbb8a026700 1 -- 172.21.15.81:0/573065382 wait complete. 2024-05-08T23:31:13.544 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:31:13.554 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2024-05-08T23:31:13.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:13.554 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T23:31:13.576 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:31:13.576 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:31:13.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:13.576 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:31:13.611 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:31:13.612 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:31:13.619 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:31:13.620 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:31:13.627 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:13.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.741+0000 7fe6fbfff700 1 Processor -- start 2024-05-08T23:31:13.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.741+0000 7fe6fbfff700 1 -- start start 2024-05-08T23:31:13.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.741+0000 7fe6fbfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc0b3c20 0x7fe6fc1624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.741+0000 7fe6fbfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe6fc070110 0x7fe6fc0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fbfff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6fc158960 con 0x7fe6fc071e70 2024-05-08T23:31:13.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fbfff700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6fc0b1f60 con 0x7fe6fc070110 2024-05-08T23:31:13.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fbfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc0ae990 con 0x7fe6fc0b3c20 2024-05-08T23:31:13.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fb7fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe6fc070110 0x7fe6fc0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fb7fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe6fc070110 0x7fe6fc0705a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58506/0 (socket says 172.21.15.81:58506) 2024-05-08T23:31:13.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fb7fe700 1 -- 172.21.15.81:0/1118644614 learned_addr learned my addr 172.21.15.81:0/1118644614 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:13.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6faffd700 1 --2- 172.21.15.81:0/1118644614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc0b3c20 0x7fe6fc1624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fb7fe700 1 -- 172.21.15.81:0/1118644614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc0b3c20 msgr2=0x7fe6fc1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fb7fe700 1 --2- 172.21.15.81:0/1118644614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc0b3c20 0x7fe6fc1624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fb7fe700 1 -- 172.21.15.81:0/1118644614 >> v1:172.21.15.117:6789/0 conn(0x7fe6fc071e70 legacy=0x7fe6fc0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fb7fe700 1 -- 172.21.15.81:0/1118644614 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc0b2170 con 0x7fe6fc070110 2024-05-08T23:31:13.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6faffd700 1 --2- 172.21.15.81:0/1118644614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc0b3c20 0x7fe6fc1624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:13.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fb7fe700 1 --2- 172.21.15.81:0/1118644614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc070110 0x7fe6fc0705a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009e50 tx=0x7fe6f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384c44134ce2652f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6f9ffb700 1 -- 172.21.15.81:0/1118644614 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f400e070 con 0x7fe6fc070110 2024-05-08T23:31:13.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6f9ffb700 1 -- 172.21.15.81:0/1118644614 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f400b9c0 con 0x7fe6fc070110 2024-05-08T23:31:13.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6f9ffb700 1 -- 172.21.15.81:0/1118644614 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f400bd70 con 0x7fe6fc070110 2024-05-08T23:31:13.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fbfff700 1 -- 172.21.15.81:0/1118644614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc070110 msgr2=0x7fe6fc0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fbfff700 1 --2- 172.21.15.81:0/1118644614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc070110 0x7fe6fc0705a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009e50 tx=0x7fe6f400b330 comp rx=0 tx=0).stop 2024-05-08T23:31:13.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fbfff700 1 -- 172.21.15.81:0/1118644614 shutdown_connections 2024-05-08T23:31:13.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fbfff700 1 --2- 172.21.15.81:0/1118644614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc070110 0x7fe6fc0705a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fbfff700 1 --2- 172.21.15.81:0/1118644614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc0b3c20 0x7fe6fc1624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fbfff700 1 -- 172.21.15.81:0/1118644614 >> 172.21.15.81:0/1118644614 conn(0x7fe6fc06c330 msgr2=0x7fe6fc06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fbfff700 1 -- 172.21.15.81:0/1118644614 shutdown_connections 2024-05-08T23:31:13.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fbfff700 1 -- 172.21.15.81:0/1118644614 wait complete. 2024-05-08T23:31:13.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.745+0000 7fe6fbfff700 1 Processor -- start 2024-05-08T23:31:13.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6fbfff700 1 -- start start 2024-05-08T23:31:13.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6fbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc070110 0x7fe6fc140580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6fbfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc071e70 0x7fe6fc153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6fbfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6fc0b3c20 0x7fe6fc151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6fbfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc1544a0 con 0x7fe6fc070110 2024-05-08T23:31:13.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6fbfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc154780 con 0x7fe6fc0b3c20 2024-05-08T23:31:13.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6fbfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc154a60 con 0x7fe6fc071e70 2024-05-08T23:31:13.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6faffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc070110 0x7fe6fc140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6faffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc070110 0x7fe6fc140580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58508/0 (socket says 172.21.15.81:58508) 2024-05-08T23:31:13.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6faffd700 1 -- 172.21.15.81:0/2588108575 learned_addr learned my addr 172.21.15.81:0/2588108575 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:13.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6fa7fc700 1 --2- 172.21.15.81:0/2588108575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc071e70 0x7fe6fc153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6fb7fe700 1 --2- 172.21.15.81:0/2588108575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6fc0b3c20 0x7fe6fc151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6faffd700 1 -- 172.21.15.81:0/2588108575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc071e70 msgr2=0x7fe6fc153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6faffd700 1 --2- 172.21.15.81:0/2588108575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc071e70 0x7fe6fc153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6faffd700 1 -- 172.21.15.81:0/2588108575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6fc0b3c20 msgr2=0x7fe6fc151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6faffd700 1 --2- 172.21.15.81:0/2588108575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6fc0b3c20 0x7fe6fc151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6faffd700 1 -- 172.21.15.81:0/2588108575 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f4009b30 con 0x7fe6fc070110 2024-05-08T23:31:13.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6faffd700 1 --2- 172.21.15.81:0/2588108575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc070110 0x7fe6fc140580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000e760 tx=0x7fe6f000ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6e37fe700 1 -- 172.21.15.81:0/2588108575 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f000c070 con 0x7fe6fc070110 2024-05-08T23:31:13.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6e37fe700 1 -- 172.21.15.81:0/2588108575 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f0004250 con 0x7fe6fc070110 2024-05-08T23:31:13.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6fbfff700 1 -- 172.21.15.81:0/2588108575 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6fc154d40 con 0x7fe6fc070110 2024-05-08T23:31:13.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6e37fe700 1 -- 172.21.15.81:0/2588108575 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f00036c0 con 0x7fe6fc070110 2024-05-08T23:31:13.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.749+0000 7fe6fbfff700 1 -- 172.21.15.81:0/2588108575 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6fc159220 con 0x7fe6fc070110 2024-05-08T23:31:13.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.753+0000 7fe6fbfff700 1 -- 172.21.15.81:0/2588108575 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6fc0684b0 con 0x7fe6fc070110 2024-05-08T23:31:13.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.753+0000 7fe6e37fe700 1 -- 172.21.15.81:0/2588108575 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe6f0003820 con 0x7fe6fc070110 2024-05-08T23:31:13.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.753+0000 7fe6e37fe700 1 --2- 172.21.15.81:0/2588108575 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe6e8066730 0x7fe6e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.753+0000 7fe6fa7fc700 1 --2- 172.21.15.81:0/2588108575 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe6e8066730 0x7fe6e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.753+0000 7fe6e37fe700 1 -- 172.21.15.81:0/2588108575 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6f0083de0 con 0x7fe6fc070110 2024-05-08T23:31:13.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.753+0000 7fe6fa7fc700 1 --2- 172.21.15.81:0/2588108575 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe6e8066730 0x7fe6e8068bf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc165900 tx=0x7fe6ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.761+0000 7fe6e37fe700 1 -- 172.21.15.81:0/2588108575 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe6f0051150 con 0x7fe6fc070110 2024-05-08T23:31:13.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.929+0000 7fe6fbfff700 1 -- 172.21.15.81:0/2588108575 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6fc163cd0 con 0x7fe6fc070110 2024-05-08T23:31:13.934 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.929+0000 7fe6e37fe700 1 -- 172.21.15.81:0/2588108575 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fe6f0054770 con 0x7fe6fc070110 2024-05-08T23:31:13.935 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:13.935 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.933+0000 7fe6fbfff700 1 -- 172.21.15.81:0/2588108575 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe6e8066730 msgr2=0x7fe6e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.933+0000 7fe6fbfff700 1 --2- 172.21.15.81:0/2588108575 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe6e8066730 0x7fe6e8068bf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc165900 tx=0x7fe6ec009040 comp rx=0 tx=0).stop 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.933+0000 7fe6fbfff700 1 -- 172.21.15.81:0/2588108575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc070110 msgr2=0x7fe6fc140580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.933+0000 7fe6fbfff700 1 --2- 172.21.15.81:0/2588108575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc070110 0x7fe6fc140580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000e760 tx=0x7fe6f000ea70 comp rx=0 tx=0).stop 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.933+0000 7fe6fbfff700 1 -- 172.21.15.81:0/2588108575 shutdown_connections 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.933+0000 7fe6fbfff700 1 --2- 172.21.15.81:0/2588108575 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe6e8066730 0x7fe6e8068bf0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.933+0000 7fe6fbfff700 1 --2- 172.21.15.81:0/2588108575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6fc0b3c20 0x7fe6fc151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.933+0000 7fe6fbfff700 1 --2- 172.21.15.81:0/2588108575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc071e70 0x7fe6fc153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.933+0000 7fe6fbfff700 1 --2- 172.21.15.81:0/2588108575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc070110 0x7fe6fc140580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.933+0000 7fe6fbfff700 1 -- 172.21.15.81:0/2588108575 >> 172.21.15.81:0/2588108575 conn(0x7fe6fc06c330 msgr2=0x7fe6fc1580d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.933+0000 7fe6fbfff700 1 -- 172.21.15.81:0/2588108575 shutdown_connections 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:13.933+0000 7fe6fbfff700 1 -- 172.21.15.81:0/2588108575 wait complete. 2024-05-08T23:31:13.950 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:13.950 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:14.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.065+0000 7f71df59e700 1 Processor -- start 2024-05-08T23:31:14.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.065+0000 7f71df59e700 1 -- start start 2024-05-08T23:31:14.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.065+0000 7f71df59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f71e0160920 0x7f71e00ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.065+0000 7f71df59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71e00d1da0 0x7f71e00cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.065+0000 7f71df59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e00d3d40 con 0x7f71e00cec30 2024-05-08T23:31:14.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.065+0000 7f71df59e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e00d0d00 con 0x7f71e0160920 2024-05-08T23:31:14.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.065+0000 7f71df59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e00d0fe0 con 0x7f71e00d1da0 2024-05-08T23:31:14.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.065+0000 7f71de59c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f71e0160920 0x7f71e00ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.065+0000 7f71de59c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f71e0160920 0x7f71e00ce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58516/0 (socket says 172.21.15.81:58516) 2024-05-08T23:31:14.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.065+0000 7f71de59c700 1 -- 172.21.15.81:0/2960206493 learned_addr learned my addr 172.21.15.81:0/2960206493 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:14.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.065+0000 7f71ddd9b700 1 --2- 172.21.15.81:0/2960206493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71e00d1da0 0x7f71e00cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71de59c700 1 -- 172.21.15.81:0/2960206493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71e00d1da0 msgr2=0x7f71e00cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:14.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71de59c700 1 --2- 172.21.15.81:0/2960206493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71e00d1da0 0x7f71e00cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71de59c700 1 -- 172.21.15.81:0/2960206493 >> v1:172.21.15.117:6789/0 conn(0x7f71e00cec30 legacy=0x7f71e0161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71de59c700 1 -- 172.21.15.81:0/2960206493 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e00d12c0 con 0x7f71e0160920 2024-05-08T23:31:14.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71de59c700 1 --2- 172.21.15.81:0/2960206493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e0160920 0x7f71e00ce6f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009910 tx=0x7f71d4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb375a0e9118d4a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71dd59a700 1 -- 172.21.15.81:0/2960206493 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4011830 con 0x7f71e0160920 2024-05-08T23:31:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71dd59a700 1 -- 172.21.15.81:0/2960206493 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d4011990 con 0x7f71e0160920 2024-05-08T23:31:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71dd59a700 1 -- 172.21.15.81:0/2960206493 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4011d40 con 0x7f71e0160920 2024-05-08T23:31:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71df59e700 1 -- 172.21.15.81:0/2960206493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e0160920 msgr2=0x7f71e00ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71df59e700 1 --2- 172.21.15.81:0/2960206493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e0160920 0x7f71e00ce6f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009910 tx=0x7f71d4009c20 comp rx=0 tx=0).stop 2024-05-08T23:31:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71df59e700 1 -- 172.21.15.81:0/2960206493 shutdown_connections 2024-05-08T23:31:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71df59e700 1 --2- 172.21.15.81:0/2960206493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71e00d1da0 0x7f71e00cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71df59e700 1 --2- 172.21.15.81:0/2960206493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e0160920 0x7f71e00ce6f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71df59e700 1 -- 172.21.15.81:0/2960206493 >> 172.21.15.81:0/2960206493 conn(0x7f71e009c340 msgr2=0x7f71e00d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71df59e700 1 -- 172.21.15.81:0/2960206493 shutdown_connections 2024-05-08T23:31:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.069+0000 7f71df59e700 1 -- 172.21.15.81:0/2960206493 wait complete. 2024-05-08T23:31:14.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71df59e700 1 Processor -- start 2024-05-08T23:31:14.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71df59e700 1 -- start start 2024-05-08T23:31:14.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71df59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e00cec30 0x7f71e0150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71df59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71e00d1da0 0x7f71e009d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71df59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71e0160920 0x7f71e0159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71df59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e00cb320 con 0x7f71e00cec30 2024-05-08T23:31:14.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71df59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e00a0360 con 0x7f71e00d1da0 2024-05-08T23:31:14.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71df59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e00a0640 con 0x7f71e0160920 2024-05-08T23:31:14.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71de59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e00cec30 0x7f71e0150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71de59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e00cec30 0x7f71e0150500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58528/0 (socket says 172.21.15.81:58528) 2024-05-08T23:31:14.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71de59c700 1 -- 172.21.15.81:0/1326838056 learned_addr learned my addr 172.21.15.81:0/1326838056 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:14.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71ddd9b700 1 --2- 172.21.15.81:0/1326838056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71e00d1da0 0x7f71e009d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71ded9d700 1 --2- 172.21.15.81:0/1326838056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71e0160920 0x7f71e0159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71de59c700 1 -- 172.21.15.81:0/1326838056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71e0160920 msgr2=0x7f71e0159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71de59c700 1 --2- 172.21.15.81:0/1326838056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71e0160920 0x7f71e0159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71de59c700 1 -- 172.21.15.81:0/1326838056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71e00d1da0 msgr2=0x7f71e009d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71de59c700 1 --2- 172.21.15.81:0/1326838056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71e00d1da0 0x7f71e009d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71de59c700 1 -- 172.21.15.81:0/1326838056 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d4009570 con 0x7f71e00cec30 2024-05-08T23:31:14.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71ddd9b700 1 --2- 172.21.15.81:0/1326838056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71e00d1da0 0x7f71e009d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:14.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71de59c700 1 --2- 172.21.15.81:0/1326838056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e00cec30 0x7f71e0150500 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f71e008bfd0 tx=0x7f71d4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:14.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71ceffd700 1 -- 172.21.15.81:0/1326838056 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4004590 con 0x7f71e00cec30 2024-05-08T23:31:14.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71ceffd700 1 -- 172.21.15.81:0/1326838056 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d401e070 con 0x7f71e00cec30 2024-05-08T23:31:14.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71df59e700 1 -- 172.21.15.81:0/1326838056 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e0160e80 con 0x7f71e00cec30 2024-05-08T23:31:14.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71ceffd700 1 -- 172.21.15.81:0/1326838056 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4003900 con 0x7f71e00cec30 2024-05-08T23:31:14.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.073+0000 7f71df59e700 1 -- 172.21.15.81:0/1326838056 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e00a0c00 con 0x7f71e00cec30 2024-05-08T23:31:14.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.077+0000 7f71df59e700 1 -- 172.21.15.81:0/1326838056 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e00684b0 con 0x7f71e00cec30 2024-05-08T23:31:14.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.077+0000 7f71ceffd700 1 -- 172.21.15.81:0/1326838056 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f71d4003a60 con 0x7f71e00cec30 2024-05-08T23:31:14.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.077+0000 7f71ceffd700 1 --2- 172.21.15.81:0/1326838056 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f71c0066730 0x7f71c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.077+0000 7f71ceffd700 1 -- 172.21.15.81:0/1326838056 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71d4084c60 con 0x7f71e00cec30 2024-05-08T23:31:14.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.077+0000 7f71ddd9b700 1 --2- 172.21.15.81:0/1326838056 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f71c0066730 0x7f71c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.077+0000 7f71ddd9b700 1 --2- 172.21.15.81:0/1326838056 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f71c0066730 0x7f71c0068bf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f71e01f69a0 tx=0x7f71d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:14.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.081+0000 7f71ceffd700 1 -- 172.21.15.81:0/1326838056 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f71d4052180 con 0x7f71e00cec30 2024-05-08T23:31:14.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.249+0000 7f71df59e700 1 -- 172.21.15.81:0/1326838056 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71e00cf510 con 0x7f71e00cec30 2024-05-08T23:31:14.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.253+0000 7f71ceffd700 1 -- 172.21.15.81:0/1326838056 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71d40200a0 con 0x7f71e00cec30 2024-05-08T23:31:14.256 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:14.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.253+0000 7f71df59e700 1 -- 172.21.15.81:0/1326838056 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f71c0066730 msgr2=0x7f71c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.253+0000 7f71df59e700 1 --2- 172.21.15.81:0/1326838056 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f71c0066730 0x7f71c0068bf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f71e01f69a0 tx=0x7f71d0008040 comp rx=0 tx=0).stop 2024-05-08T23:31:14.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.253+0000 7f71df59e700 1 -- 172.21.15.81:0/1326838056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e00cec30 msgr2=0x7f71e0150500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.253+0000 7f71df59e700 1 --2- 172.21.15.81:0/1326838056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e00cec30 0x7f71e0150500 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f71e008bfd0 tx=0x7f71d4004770 comp rx=0 tx=0).stop 2024-05-08T23:31:14.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.253+0000 7f71df59e700 1 -- 172.21.15.81:0/1326838056 shutdown_connections 2024-05-08T23:31:14.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.253+0000 7f71df59e700 1 --2- 172.21.15.81:0/1326838056 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f71c0066730 0x7f71c0068bf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.253+0000 7f71df59e700 1 --2- 172.21.15.81:0/1326838056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71e0160920 0x7f71e0159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.253+0000 7f71df59e700 1 --2- 172.21.15.81:0/1326838056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71e00d1da0 0x7f71e009d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.253+0000 7f71df59e700 1 --2- 172.21.15.81:0/1326838056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71e00cec30 0x7f71e0150500 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.253+0000 7f71df59e700 1 -- 172.21.15.81:0/1326838056 >> 172.21.15.81:0/1326838056 conn(0x7f71e009c340 msgr2=0x7f71e00b74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:14.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.253+0000 7f71df59e700 1 -- 172.21.15.81:0/1326838056 shutdown_connections 2024-05-08T23:31:14.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.253+0000 7f71df59e700 1 -- 172.21.15.81:0/1326838056 wait complete. 2024-05-08T23:31:14.274 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:14.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 Processor -- start 2024-05-08T23:31:14.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 -- start start 2024-05-08T23:31:14.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f395015ae10 0x7f39500ac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f395015ead0 0x7f39500cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f39500ce4f0 0x7f39500b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39500ce9a0 con 0x7f39500ce4f0 2024-05-08T23:31:14.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395006c7d0 con 0x7f395015ead0 2024-05-08T23:31:14.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f395006c4d0 con 0x7f395015ae10 2024-05-08T23:31:14.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f39557db700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f39500ce4f0 0x7f39500b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f39557db700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f39500ce4f0 0x7f39500b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58536/0 (socket says 172.21.15.81:58536) 2024-05-08T23:31:14.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f39557db700 1 -- 172.21.15.81:0/744813313 learned_addr learned my addr 172.21.15.81:0/744813313 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:14.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f394ffff700 1 --2- 172.21.15.81:0/744813313 >> v2:172.21.15.117:3300/0 conn(0x7f395015ead0 0x7f39500cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3954fda700 1 --2- 172.21.15.81:0/744813313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f395015ae10 0x7f39500ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f39557db700 1 -- 172.21.15.81:0/744813313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f395015ae10 msgr2=0x7f39500ac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:14.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f39557db700 1 --2- 172.21.15.81:0/744813313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f395015ae10 0x7f39500ac420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f39557db700 1 -- 172.21.15.81:0/744813313 >> v2:172.21.15.117:3300/0 conn(0x7f395015ead0 msgr2=0x7f39500cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:14.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f39557db700 1 --2- 172.21.15.81:0/744813313 >> v2:172.21.15.117:3300/0 conn(0x7f395015ead0 0x7f39500cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f39557db700 1 -- 172.21.15.81:0/744813313 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39500acd90 con 0x7f39500ce4f0 2024-05-08T23:31:14.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f394ffff700 1 --2- 172.21.15.81:0/744813313 >> v2:172.21.15.117:3300/0 conn(0x7f395015ead0 0x7f39500cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:14.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f39557db700 1 --2- 172.21.15.81:0/744813313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39500ce4f0 0x7f39500b2dd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f39500b1900 tx=0x7f3940009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fac8b43d32b4bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:14.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f394f7fe700 1 -- 172.21.15.81:0/744813313 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394000b8a0 con 0x7f39500ce4f0 2024-05-08T23:31:14.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f394f7fe700 1 -- 172.21.15.81:0/744813313 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394000ba00 con 0x7f39500ce4f0 2024-05-08T23:31:14.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f394f7fe700 1 -- 172.21.15.81:0/744813313 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394000bd90 con 0x7f39500ce4f0 2024-05-08T23:31:14.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 -- 172.21.15.81:0/744813313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39500ce4f0 msgr2=0x7f39500b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 --2- 172.21.15.81:0/744813313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39500ce4f0 0x7f39500b2dd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f39500b1900 tx=0x7f3940009bf0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 -- 172.21.15.81:0/744813313 shutdown_connections 2024-05-08T23:31:14.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 --2- 172.21.15.81:0/744813313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39500ce4f0 0x7f39500b2dd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 --2- 172.21.15.81:0/744813313 >> v2:172.21.15.117:3300/0 conn(0x7f395015ead0 0x7f39500cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 --2- 172.21.15.81:0/744813313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f395015ae10 0x7f39500ac420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 -- 172.21.15.81:0/744813313 >> 172.21.15.81:0/744813313 conn(0x7f39500b0de0 msgr2=0x7f39500b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:14.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 -- 172.21.15.81:0/744813313 shutdown_connections 2024-05-08T23:31:14.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.393+0000 7f3955fdc700 1 -- 172.21.15.81:0/744813313 wait complete. 2024-05-08T23:31:14.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3955fdc700 1 Processor -- start 2024-05-08T23:31:14.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3955fdc700 1 -- start start 2024-05-08T23:31:14.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3955fdc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39500ce4f0 0x7f3950158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3955fdc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f395015ae10 0x7f395014c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3955fdc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f395015ead0 0x7f395014d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3955fdc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39501e4bf0 con 0x7f39500ce4f0 2024-05-08T23:31:14.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3955fdc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39501e3e60 con 0x7f395015ead0 2024-05-08T23:31:14.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3955fdc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39501595f0 con 0x7f395015ae10 2024-05-08T23:31:14.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3954fda700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39500ce4f0 0x7f3950158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3954fda700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39500ce4f0 0x7f3950158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58538/0 (socket says 172.21.15.81:58538) 2024-05-08T23:31:14.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3954fda700 1 -- 172.21.15.81:0/3267709788 learned_addr learned my addr 172.21.15.81:0/3267709788 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:14.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f39557db700 1 --2- 172.21.15.81:0/3267709788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f395015ead0 0x7f395014d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f394ffff700 1 --2- 172.21.15.81:0/3267709788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f395015ae10 0x7f395014c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3954fda700 1 -- 172.21.15.81:0/3267709788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f395015ae10 msgr2=0x7f395014c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3954fda700 1 --2- 172.21.15.81:0/3267709788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f395015ae10 0x7f395014c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3954fda700 1 -- 172.21.15.81:0/3267709788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f395015ead0 msgr2=0x7f395014d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3954fda700 1 --2- 172.21.15.81:0/3267709788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f395015ead0 0x7f395014d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3954fda700 1 -- 172.21.15.81:0/3267709788 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3940009570 con 0x7f39500ce4f0 2024-05-08T23:31:14.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f39557db700 1 --2- 172.21.15.81:0/3267709788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f395015ead0 0x7f395014d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:14.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3954fda700 1 --2- 172.21.15.81:0/3267709788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39500ce4f0 0x7f3950158f60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f394400e760 tx=0x7f394400ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:14.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f394d7fa700 1 -- 172.21.15.81:0/3267709788 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394400c070 con 0x7f39500ce4f0 2024-05-08T23:31:14.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f394d7fa700 1 -- 172.21.15.81:0/3267709788 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3944013610 con 0x7f39500ce4f0 2024-05-08T23:31:14.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f394d7fa700 1 -- 172.21.15.81:0/3267709788 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944013980 con 0x7f39500ce4f0 2024-05-08T23:31:14.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3955fdc700 1 -- 172.21.15.81:0/3267709788 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39501598d0 con 0x7f39500ce4f0 2024-05-08T23:31:14.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.397+0000 7f3955fdc700 1 -- 172.21.15.81:0/3267709788 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39501e5170 con 0x7f39500ce4f0 2024-05-08T23:31:14.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.401+0000 7f3955fdc700 1 -- 172.21.15.81:0/3267709788 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39500684b0 con 0x7f39500ce4f0 2024-05-08T23:31:14.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.401+0000 7f394d7fa700 1 -- 172.21.15.81:0/3267709788 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3944013ae0 con 0x7f39500ce4f0 2024-05-08T23:31:14.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.401+0000 7f394d7fa700 1 --2- 172.21.15.81:0/3267709788 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f393c066730 0x7f393c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.401+0000 7f394ffff700 1 --2- 172.21.15.81:0/3267709788 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f393c066730 0x7f393c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.401+0000 7f394d7fa700 1 -- 172.21.15.81:0/3267709788 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3944083f30 con 0x7f39500ce4f0 2024-05-08T23:31:14.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.401+0000 7f394ffff700 1 --2- 172.21.15.81:0/3267709788 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f393c066730 0x7f393c068bf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f395015eef0 tx=0x7f3948009870 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:14.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.409+0000 7f394d7fa700 1 -- 172.21.15.81:0/3267709788 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3944051290 con 0x7f39500ce4f0 2024-05-08T23:31:14.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.573+0000 7f3955fdc700 1 -- 172.21.15.81:0/3267709788 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f39500b3900 con 0x7f39500ce4f0 2024-05-08T23:31:14.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.573+0000 7f394d7fa700 1 -- 172.21.15.81:0/3267709788 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f39440548b0 con 0x7f39500ce4f0 2024-05-08T23:31:14.579 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:14.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.577+0000 7f3955fdc700 1 -- 172.21.15.81:0/3267709788 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f393c066730 msgr2=0x7f393c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.577+0000 7f3955fdc700 1 --2- 172.21.15.81:0/3267709788 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f393c066730 0x7f393c068bf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f395015eef0 tx=0x7f3948009870 comp rx=0 tx=0).stop 2024-05-08T23:31:14.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.577+0000 7f3955fdc700 1 -- 172.21.15.81:0/3267709788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39500ce4f0 msgr2=0x7f3950158f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.577+0000 7f3955fdc700 1 --2- 172.21.15.81:0/3267709788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39500ce4f0 0x7f3950158f60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f394400e760 tx=0x7f394400ea70 comp rx=0 tx=0).stop 2024-05-08T23:31:14.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.577+0000 7f3955fdc700 1 -- 172.21.15.81:0/3267709788 shutdown_connections 2024-05-08T23:31:14.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.577+0000 7f3955fdc700 1 --2- 172.21.15.81:0/3267709788 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f393c066730 0x7f393c068bf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.577+0000 7f3955fdc700 1 --2- 172.21.15.81:0/3267709788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f395015ead0 0x7f395014d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.577+0000 7f3955fdc700 1 --2- 172.21.15.81:0/3267709788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f395015ae10 0x7f395014c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.577+0000 7f3955fdc700 1 --2- 172.21.15.81:0/3267709788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39500ce4f0 0x7f3950158f60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.577+0000 7f3955fdc700 1 -- 172.21.15.81:0/3267709788 >> 172.21.15.81:0/3267709788 conn(0x7f39500b0de0 msgr2=0x7f39500ae830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:14.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.577+0000 7f3955fdc700 1 -- 172.21.15.81:0/3267709788 shutdown_connections 2024-05-08T23:31:14.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.577+0000 7f3955fdc700 1 -- 172.21.15.81:0/3267709788 wait complete. 2024-05-08T23:31:14.594 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:14.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.701+0000 7f7cba556700 1 Processor -- start 2024-05-08T23:31:14.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.701+0000 7f7cba556700 1 -- start start 2024-05-08T23:31:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.701+0000 7f7cba556700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7cb40ae7c0 0x7f7cb406b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.701+0000 7f7cba556700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7cb406b8b0 0x7f7cb406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.701+0000 7f7cba556700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb40b2d90 con 0x7f7cb415d090 2024-05-08T23:31:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.701+0000 7f7cba556700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb406bd20 con 0x7f7cb40ae7c0 2024-05-08T23:31:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.701+0000 7f7cba556700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40b2ff0 con 0x7f7cb406b8b0 2024-05-08T23:31:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cb8d53700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7cb40ae7c0 0x7f7cb406b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cb8d53700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7cb40ae7c0 0x7f7cb406b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58542/0 (socket says 172.21.15.81:58542) 2024-05-08T23:31:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cb8d53700 1 -- 172.21.15.81:0/3221482208 learned_addr learned my addr 172.21.15.81:0/3221482208 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cb9d55700 1 --2- 172.21.15.81:0/3221482208 >> v2:172.21.15.117:3300/0 conn(0x7f7cb406b8b0 0x7f7cb406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cb8d53700 1 -- 172.21.15.81:0/3221482208 >> v2:172.21.15.117:3300/0 conn(0x7f7cb406b8b0 msgr2=0x7f7cb406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cb8d53700 1 --2- 172.21.15.81:0/3221482208 >> v2:172.21.15.117:3300/0 conn(0x7f7cb406b8b0 0x7f7cb406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cb8d53700 1 -- 172.21.15.81:0/3221482208 >> v1:172.21.15.117:6789/0 conn(0x7f7cb415d090 legacy=0x7f7cb40b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cb8d53700 1 -- 172.21.15.81:0/3221482208 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40b32d0 con 0x7f7cb40ae7c0 2024-05-08T23:31:14.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cb8d53700 1 --2- 172.21.15.81:0/3221482208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb40ae7c0 0x7f7cb406b370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7cb406cf90 tx=0x7f7cb0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81c1137d4f0105a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:14.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cabfff700 1 -- 172.21.15.81:0/3221482208 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0011830 con 0x7f7cb40ae7c0 2024-05-08T23:31:14.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cabfff700 1 -- 172.21.15.81:0/3221482208 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb0011990 con 0x7f7cb40ae7c0 2024-05-08T23:31:14.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cabfff700 1 -- 172.21.15.81:0/3221482208 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0011d20 con 0x7f7cb40ae7c0 2024-05-08T23:31:14.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cba556700 1 -- 172.21.15.81:0/3221482208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb40ae7c0 msgr2=0x7f7cb406b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cba556700 1 --2- 172.21.15.81:0/3221482208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb40ae7c0 0x7f7cb406b370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7cb406cf90 tx=0x7f7cb0009bf0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cba556700 1 -- 172.21.15.81:0/3221482208 shutdown_connections 2024-05-08T23:31:14.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cba556700 1 --2- 172.21.15.81:0/3221482208 >> v2:172.21.15.117:3300/0 conn(0x7f7cb406b8b0 0x7f7cb406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cba556700 1 --2- 172.21.15.81:0/3221482208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb40ae7c0 0x7f7cb406b370 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cba556700 1 -- 172.21.15.81:0/3221482208 >> 172.21.15.81:0/3221482208 conn(0x7f7cb40708f0 msgr2=0x7f7cb4070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:14.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cba556700 1 -- 172.21.15.81:0/3221482208 shutdown_connections 2024-05-08T23:31:14.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.705+0000 7f7cba556700 1 -- 172.21.15.81:0/3221482208 wait complete. 2024-05-08T23:31:14.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.709+0000 7f7cba556700 1 Processor -- start 2024-05-08T23:31:14.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.709+0000 7f7cba556700 1 -- start start 2024-05-08T23:31:14.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.709+0000 7f7cba556700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cb406b8b0 0x7f7cb4152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.709+0000 7f7cba556700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb40ae7c0 0x7f7cb415e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.709+0000 7f7cba556700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb415d090 0x7f7cb41f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.709+0000 7f7cba556700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb415fa90 con 0x7f7cb415d090 2024-05-08T23:31:14.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.709+0000 7f7cba556700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb41f65b0 con 0x7f7cb40ae7c0 2024-05-08T23:31:14.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.709+0000 7f7cba556700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb415ee10 con 0x7f7cb406b8b0 2024-05-08T23:31:14.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.709+0000 7f7cb9d55700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb415d090 0x7f7cb41f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.709+0000 7f7cb9d55700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb415d090 0x7f7cb41f6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58548/0 (socket says 172.21.15.81:58548) 2024-05-08T23:31:14.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7cb9d55700 1 -- 172.21.15.81:0/2550727637 learned_addr learned my addr 172.21.15.81:0/2550727637 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:14.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7cb9554700 1 --2- 172.21.15.81:0/2550727637 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cb406b8b0 0x7f7cb4152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7cb8d53700 1 --2- 172.21.15.81:0/2550727637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb40ae7c0 0x7f7cb415e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7cb9d55700 1 -- 172.21.15.81:0/2550727637 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cb406b8b0 msgr2=0x7f7cb4152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7cb9d55700 1 --2- 172.21.15.81:0/2550727637 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cb406b8b0 0x7f7cb4152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7cb9d55700 1 -- 172.21.15.81:0/2550727637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb40ae7c0 msgr2=0x7f7cb415e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7cb9d55700 1 --2- 172.21.15.81:0/2550727637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb40ae7c0 0x7f7cb415e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7cb9d55700 1 -- 172.21.15.81:0/2550727637 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb0009570 con 0x7f7cb415d090 2024-05-08T23:31:14.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7cb9d55700 1 --2- 172.21.15.81:0/2550727637 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb415d090 0x7f7cb41f6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00efc0 tx=0x7f7cac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:14.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7ca9ffb700 1 -- 172.21.15.81:0/2550727637 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac009ea0 con 0x7f7cb415d090 2024-05-08T23:31:14.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7ca9ffb700 1 -- 172.21.15.81:0/2550727637 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac00f040 con 0x7f7cb415d090 2024-05-08T23:31:14.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7cba556700 1 -- 172.21.15.81:0/2550727637 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb415f0f0 con 0x7f7cb415d090 2024-05-08T23:31:14.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7ca9ffb700 1 -- 172.21.15.81:0/2550727637 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac00ced0 con 0x7f7cb415d090 2024-05-08T23:31:14.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7cba556700 1 -- 172.21.15.81:0/2550727637 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb415f6b0 con 0x7f7cb415d090 2024-05-08T23:31:14.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7cba556700 1 -- 172.21.15.81:0/2550727637 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb415fbd0 con 0x7f7cb415d090 2024-05-08T23:31:14.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.713+0000 7f7ca9ffb700 1 -- 172.21.15.81:0/2550727637 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7cac014480 con 0x7f7cb415d090 2024-05-08T23:31:14.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.717+0000 7f7ca9ffb700 1 --2- 172.21.15.81:0/2550727637 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7c98066730 0x7f7c98068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:14.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.717+0000 7f7cb9554700 1 --2- 172.21.15.81:0/2550727637 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7c98066730 0x7f7c98068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:14.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.717+0000 7f7ca9ffb700 1 -- 172.21.15.81:0/2550727637 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cac084c70 con 0x7f7cb415d090 2024-05-08T23:31:14.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.717+0000 7f7cb9554700 1 --2- 172.21.15.81:0/2550727637 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7c98066730 0x7f7c98068bf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4000f60 tx=0x7f7ca4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:14.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.721+0000 7f7ca9ffb700 1 -- 172.21.15.81:0/2550727637 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7cb415fbd0 con 0x7f7cb415d090 2024-05-08T23:31:14.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.893+0000 7f7cba556700 1 -- 172.21.15.81:0/2550727637 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7cb415fbd0 con 0x7f7cb415d090 2024-05-08T23:31:14.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.893+0000 7f7ca9ffb700 1 -- 172.21.15.81:0/2550727637 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7cb415fbd0 con 0x7f7cb415d090 2024-05-08T23:31:14.898 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:14.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.897+0000 7f7cba556700 1 -- 172.21.15.81:0/2550727637 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7c98066730 msgr2=0x7f7c98068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.897+0000 7f7cba556700 1 --2- 172.21.15.81:0/2550727637 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7c98066730 0x7f7c98068bf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4000f60 tx=0x7f7ca4009040 comp rx=0 tx=0).stop 2024-05-08T23:31:14.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.897+0000 7f7cba556700 1 -- 172.21.15.81:0/2550727637 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb415d090 msgr2=0x7f7cb41f6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:14.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.897+0000 7f7cba556700 1 --2- 172.21.15.81:0/2550727637 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb415d090 0x7f7cb41f6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00efc0 tx=0x7f7cac00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.897+0000 7f7cba556700 1 -- 172.21.15.81:0/2550727637 shutdown_connections 2024-05-08T23:31:14.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.897+0000 7f7cba556700 1 --2- 172.21.15.81:0/2550727637 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7c98066730 0x7f7c98068bf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.897+0000 7f7cba556700 1 --2- 172.21.15.81:0/2550727637 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb415d090 0x7f7cb41f6070 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.897+0000 7f7cba556700 1 --2- 172.21.15.81:0/2550727637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb40ae7c0 0x7f7cb415e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.897+0000 7f7cba556700 1 --2- 172.21.15.81:0/2550727637 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cb406b8b0 0x7f7cb4152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:14.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.897+0000 7f7cba556700 1 -- 172.21.15.81:0/2550727637 >> 172.21.15.81:0/2550727637 conn(0x7f7cb40708f0 msgr2=0x7f7cb40afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:14.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.897+0000 7f7cba556700 1 -- 172.21.15.81:0/2550727637 shutdown_connections 2024-05-08T23:31:14.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:14.897+0000 7f7cba556700 1 -- 172.21.15.81:0/2550727637 wait complete. 2024-05-08T23:31:14.913 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:31:15.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.021+0000 7fdf0b59e700 1 Processor -- start 2024-05-08T23:31:15.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.021+0000 7fdf0b59e700 1 -- start start 2024-05-08T23:31:15.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0b59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdf0c1606b0 0x7fdf0c0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0b59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdf0c06b470 0x7fdf0c06baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0b59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf0c06bfe0 0x7fdf0c160de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0b59e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c06c450 con 0x7fdf0c1606b0 2024-05-08T23:31:15.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0b59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c0d0420 con 0x7fdf0c06b470 2024-05-08T23:31:15.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0b59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c0d0180 con 0x7fdf0c06bfe0 2024-05-08T23:31:15.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0a59c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdf0c1606b0 0x7fdf0c0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0a59c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdf0c1606b0 0x7fdf0c0cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58552/0 (socket says 172.21.15.81:58552) 2024-05-08T23:31:15.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0a59c700 1 -- 172.21.15.81:0/2488002629 learned_addr learned my addr 172.21.15.81:0/2488002629 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:15.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf09d9b700 1 --2- 172.21.15.81:0/2488002629 >> v2:172.21.15.117:3300/0 conn(0x7fdf0c06b470 0x7fdf0c06baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0ad9d700 1 --2- 172.21.15.81:0/2488002629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf0c06bfe0 0x7fdf0c160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0a59c700 1 -- 172.21.15.81:0/2488002629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf0c06bfe0 msgr2=0x7fdf0c160de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:15.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0a59c700 1 --2- 172.21.15.81:0/2488002629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf0c06bfe0 0x7fdf0c160de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0a59c700 1 -- 172.21.15.81:0/2488002629 >> v2:172.21.15.117:3300/0 conn(0x7fdf0c06b470 msgr2=0x7fdf0c06baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:15.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0a59c700 1 --2- 172.21.15.81:0/2488002629 >> v2:172.21.15.117:3300/0 conn(0x7fdf0c06b470 0x7fdf0c06baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0a59c700 1 -- 172.21.15.81:0/2488002629 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c0b04a0 con 0x7fdf0c1606b0 2024-05-08T23:31:15.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0ad9d700 1 --2- 172.21.15.81:0/2488002629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf0c06bfe0 0x7fdf0c160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:15.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0a59c700 1 --2- 172.21.15.81:0/2488002629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf0c1606b0 0x7fdf0c0cfb50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0715c0 tx=0x7fdefc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2016443cab5b5333 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:15.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0959a700 1 -- 172.21.15.81:0/2488002629 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc011830 con 0x7fdf0c1606b0 2024-05-08T23:31:15.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0959a700 1 -- 172.21.15.81:0/2488002629 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc011990 con 0x7fdf0c1606b0 2024-05-08T23:31:15.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.025+0000 7fdf0959a700 1 -- 172.21.15.81:0/2488002629 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc011d20 con 0x7fdf0c1606b0 2024-05-08T23:31:15.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.029+0000 7fdf0b59e700 1 -- 172.21.15.81:0/2488002629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf0c1606b0 msgr2=0x7fdf0c0cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.029+0000 7fdf0b59e700 1 --2- 172.21.15.81:0/2488002629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf0c1606b0 0x7fdf0c0cfb50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0715c0 tx=0x7fdefc009bf0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.029+0000 7fdf0b59e700 1 -- 172.21.15.81:0/2488002629 shutdown_connections 2024-05-08T23:31:15.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.029+0000 7fdf0b59e700 1 --2- 172.21.15.81:0/2488002629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf0c06bfe0 0x7fdf0c160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.029+0000 7fdf0b59e700 1 --2- 172.21.15.81:0/2488002629 >> v2:172.21.15.117:3300/0 conn(0x7fdf0c06b470 0x7fdf0c06baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.029+0000 7fdf0b59e700 1 --2- 172.21.15.81:0/2488002629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf0c1606b0 0x7fdf0c0cfb50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.029+0000 7fdf0b59e700 1 -- 172.21.15.81:0/2488002629 >> 172.21.15.81:0/2488002629 conn(0x7fdf0c0b2970 msgr2=0x7fdf0c0b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:15.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.029+0000 7fdf0b59e700 1 -- 172.21.15.81:0/2488002629 shutdown_connections 2024-05-08T23:31:15.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.029+0000 7fdf0b59e700 1 -- 172.21.15.81:0/2488002629 wait complete. 2024-05-08T23:31:15.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.029+0000 7fdf0b59e700 1 Processor -- start 2024-05-08T23:31:15.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.029+0000 7fdf0b59e700 1 -- start start 2024-05-08T23:31:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf0b59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf0c06b470 0x7fdf0c1f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf0b59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf0c06bfe0 0x7fdf0c1f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf0b59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf0c1606b0 0x7fdf0c1f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf0b59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c158710 con 0x7fdf0c06bfe0 2024-05-08T23:31:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf0b59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1f64e0 con 0x7fdf0c06b470 2024-05-08T23:31:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf0b59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c157a90 con 0x7fdf0c1606b0 2024-05-08T23:31:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf09d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf0c06bfe0 0x7fdf0c1f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf09d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf0c06bfe0 0x7fdf0c1f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58562/0 (socket says 172.21.15.81:58562) 2024-05-08T23:31:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf09d9b700 1 -- 172.21.15.81:0/2801278030 learned_addr learned my addr 172.21.15.81:0/2801278030 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf0ad9d700 1 --2- 172.21.15.81:0/2801278030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf0c1606b0 0x7fdf0c1f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf0a59c700 1 --2- 172.21.15.81:0/2801278030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf0c06b470 0x7fdf0c1f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf09d9b700 1 -- 172.21.15.81:0/2801278030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf0c1606b0 msgr2=0x7fdf0c1f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf09d9b700 1 --2- 172.21.15.81:0/2801278030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf0c1606b0 0x7fdf0c1f33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf09d9b700 1 -- 172.21.15.81:0/2801278030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf0c06b470 msgr2=0x7fdf0c1f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf09d9b700 1 --2- 172.21.15.81:0/2801278030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf0c06b470 0x7fdf0c1f5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf09d9b700 1 -- 172.21.15.81:0/2801278030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc009570 con 0x7fdf0c06bfe0 2024-05-08T23:31:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf09d9b700 1 --2- 172.21.15.81:0/2801278030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf0c06bfe0 0x7fdf0c1f5fa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400eed0 tx=0x7fdf0400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf02ffd700 1 -- 172.21.15.81:0/2801278030 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04010040 con 0x7fdf0c06bfe0 2024-05-08T23:31:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf02ffd700 1 -- 172.21.15.81:0/2801278030 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0400bdb0 con 0x7fdf0c06bfe0 2024-05-08T23:31:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf0b59e700 1 -- 172.21.15.81:0/2801278030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c157d70 con 0x7fdf0c06bfe0 2024-05-08T23:31:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf02ffd700 1 -- 172.21.15.81:0/2801278030 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04014620 con 0x7fdf0c06bfe0 2024-05-08T23:31:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf0b59e700 1 -- 172.21.15.81:0/2801278030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c158330 con 0x7fdf0c06bfe0 2024-05-08T23:31:15.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf0b59e700 1 -- 172.21.15.81:0/2801278030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0c0672a0 con 0x7fdf0c06bfe0 2024-05-08T23:31:15.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf02ffd700 1 -- 172.21.15.81:0/2801278030 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf04014780 con 0x7fdf0c06bfe0 2024-05-08T23:31:15.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.033+0000 7fdf02ffd700 1 --2- 172.21.15.81:0/2801278030 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdef0066730 0x7fdef0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.037+0000 7fdf02ffd700 1 -- 172.21.15.81:0/2801278030 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf04084950 con 0x7fdf0c06bfe0 2024-05-08T23:31:15.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.037+0000 7fdf0a59c700 1 --2- 172.21.15.81:0/2801278030 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdef0066730 0x7fdef0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.037+0000 7fdf0a59c700 1 --2- 172.21.15.81:0/2801278030 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdef0066730 0x7fdef0068bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdefc000c00 tx=0x7fdefc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:15.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.045+0000 7fdf02ffd700 1 -- 172.21.15.81:0/2801278030 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdf04051cc0 con 0x7fdf0c06bfe0 2024-05-08T23:31:15.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.225+0000 7fdf0b59e700 1 -- 172.21.15.81:0/2801278030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdf0c0724f0 con 0x7fdf0c06bfe0 2024-05-08T23:31:15.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.225+0000 7fdf02ffd700 1 -- 172.21.15.81:0/2801278030 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdf040552e0 con 0x7fdf0c06bfe0 2024-05-08T23:31:15.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.229+0000 7fdf0b59e700 1 -- 172.21.15.81:0/2801278030 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdef0066730 msgr2=0x7fdef0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.229+0000 7fdf0b59e700 1 --2- 172.21.15.81:0/2801278030 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdef0066730 0x7fdef0068bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdefc000c00 tx=0x7fdefc008030 comp rx=0 tx=0).stop 2024-05-08T23:31:15.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.229+0000 7fdf0b59e700 1 -- 172.21.15.81:0/2801278030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf0c06bfe0 msgr2=0x7fdf0c1f5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.229+0000 7fdf0b59e700 1 --2- 172.21.15.81:0/2801278030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf0c06bfe0 0x7fdf0c1f5fa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400eed0 tx=0x7fdf0400b390 comp rx=0 tx=0).stop 2024-05-08T23:31:15.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.229+0000 7fdf0b59e700 1 -- 172.21.15.81:0/2801278030 shutdown_connections 2024-05-08T23:31:15.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.229+0000 7fdf0b59e700 1 --2- 172.21.15.81:0/2801278030 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdef0066730 0x7fdef0068bf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.229+0000 7fdf0b59e700 1 --2- 172.21.15.81:0/2801278030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf0c1606b0 0x7fdf0c1f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.229+0000 7fdf0b59e700 1 --2- 172.21.15.81:0/2801278030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf0c06bfe0 0x7fdf0c1f5fa0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.229+0000 7fdf0b59e700 1 --2- 172.21.15.81:0/2801278030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf0c06b470 0x7fdf0c1f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.229+0000 7fdf0b59e700 1 -- 172.21.15.81:0/2801278030 >> 172.21.15.81:0/2801278030 conn(0x7fdf0c0b2970 msgr2=0x7fdf0c06f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:15.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.229+0000 7fdf0b59e700 1 -- 172.21.15.81:0/2801278030 shutdown_connections 2024-05-08T23:31:15.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.229+0000 7fdf0b59e700 1 -- 172.21.15.81:0/2801278030 wait complete. 2024-05-08T23:31:15.234 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:31:15.245 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:31:15.245 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:31:15.254 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:15.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.357+0000 7f056c90d700 1 Processor -- start 2024-05-08T23:31:15.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.357+0000 7f056c90d700 1 -- start start 2024-05-08T23:31:15.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f056c90d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f05680b4580 0x7f0568153d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f056c90d700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05680cd3a0 con 0x7f056806e400 2024-05-08T23:31:15.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f056c90d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05680b35b0 con 0x7f05680b41a0 2024-05-08T23:31:15.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f056c90d700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05680cd5d0 con 0x7f05680b4580 2024-05-08T23:31:15.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f0567fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f05680b4580 0x7f0568153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f0567fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f05680b4580 0x7f0568153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58576/0 (socket says 172.21.15.81:58576) 2024-05-08T23:31:15.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f0566ffd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f056806e400 0x7f0568153630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46818/0 (socket says 172.21.15.81:46818) 2024-05-08T23:31:15.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f0567fff700 1 -- 172.21.15.81:0/2712675347 learned_addr learned my addr 172.21.15.81:0/2712675347 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:15.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f0567fff700 1 -- 172.21.15.81:0/2712675347 >> v1:172.21.15.117:6789/0 conn(0x7f05680b41a0 legacy=0x7f056815a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f0567fff700 1 -- 172.21.15.81:0/2712675347 >> v1:172.21.15.81:6789/0 conn(0x7f056806e400 legacy=0x7f0568153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f0567fff700 1 -- 172.21.15.81:0/2712675347 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05680cd8b0 con 0x7f05680b4580 2024-05-08T23:31:15.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f05667fc700 1 -- 172.21.15.81:0/2712675347 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191869048 0 0) 0x7f05680cd3a0 con 0x7f056806e400 2024-05-08T23:31:15.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f0567fff700 1 --2- 172.21.15.81:0/2712675347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05680b4580 0x7f0568153d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f05680b37a0 tx=0x7f056000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42ae838093f94c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:15.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f05667fc700 1 -- 172.21.15.81:0/2712675347 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056000f070 con 0x7f05680b4580 2024-05-08T23:31:15.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f05667fc700 1 -- 172.21.15.81:0/2712675347 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056000a2e0 con 0x7f05680b4580 2024-05-08T23:31:15.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f05667fc700 1 -- 172.21.15.81:0/2712675347 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056000ccc0 con 0x7f05680b4580 2024-05-08T23:31:15.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f056c90d700 1 -- 172.21.15.81:0/2712675347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05680b4580 msgr2=0x7f0568153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f056c90d700 1 --2- 172.21.15.81:0/2712675347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05680b4580 0x7f0568153d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f05680b37a0 tx=0x7f056000c330 comp rx=0 tx=0).stop 2024-05-08T23:31:15.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f056c90d700 1 -- 172.21.15.81:0/2712675347 shutdown_connections 2024-05-08T23:31:15.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f056c90d700 1 --2- 172.21.15.81:0/2712675347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05680b4580 0x7f0568153d40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f056c90d700 1 -- 172.21.15.81:0/2712675347 >> 172.21.15.81:0/2712675347 conn(0x7f05680b16c0 msgr2=0x7f05680b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:15.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f056c90d700 1 -- 172.21.15.81:0/2712675347 shutdown_connections 2024-05-08T23:31:15.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.361+0000 7f056c90d700 1 -- 172.21.15.81:0/2712675347 wait complete. 2024-05-08T23:31:15.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f056c90d700 1 Processor -- start 2024-05-08T23:31:15.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f056c90d700 1 -- start start 2024-05-08T23:31:15.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f056c90d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f056806e400 0x7f0568146160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f056c90d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05680b41a0 0x7f05680c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f056c90d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05680b4580 0x7f0568151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f056c90d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05681516a0 con 0x7f05680b4580 2024-05-08T23:31:15.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f056c90d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056814fa20 con 0x7f05680b41a0 2024-05-08T23:31:15.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f056c90d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056814fd00 con 0x7f056806e400 2024-05-08T23:31:15.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f0567fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05680b4580 0x7f0568151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f0567fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05680b4580 0x7f0568151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58578/0 (socket says 172.21.15.81:58578) 2024-05-08T23:31:15.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f0567fff700 1 -- 172.21.15.81:0/529094304 learned_addr learned my addr 172.21.15.81:0/529094304 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:15.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f05677fe700 1 --2- 172.21.15.81:0/529094304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f056806e400 0x7f0568146160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f0566ffd700 1 --2- 172.21.15.81:0/529094304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05680b41a0 0x7f05680c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f0567fff700 1 -- 172.21.15.81:0/529094304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f056806e400 msgr2=0x7f0568146160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f0567fff700 1 --2- 172.21.15.81:0/529094304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f056806e400 0x7f0568146160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f0567fff700 1 -- 172.21.15.81:0/529094304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05680b41a0 msgr2=0x7f05680c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f0567fff700 1 --2- 172.21.15.81:0/529094304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05680b41a0 0x7f05680c46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f0567fff700 1 -- 172.21.15.81:0/529094304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056000ac40 con 0x7f05680b4580 2024-05-08T23:31:15.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.365+0000 7f0567fff700 1 --2- 172.21.15.81:0/529094304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05680b4580 0x7f0568151160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0568159c00 tx=0x7f0560004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:15.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.369+0000 7f054ffff700 1 -- 172.21.15.81:0/529094304 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560004440 con 0x7f05680b4580 2024-05-08T23:31:15.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.369+0000 7f054ffff700 1 -- 172.21.15.81:0/529094304 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05600045a0 con 0x7f05680b4580 2024-05-08T23:31:15.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.369+0000 7f056c90d700 1 -- 172.21.15.81:0/529094304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056814ffe0 con 0x7f05680b4580 2024-05-08T23:31:15.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.369+0000 7f054ffff700 1 -- 172.21.15.81:0/529094304 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560004950 con 0x7f05680b4580 2024-05-08T23:31:15.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.369+0000 7f056c90d700 1 -- 172.21.15.81:0/529094304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05681507e0 con 0x7f05680b4580 2024-05-08T23:31:15.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.369+0000 7f056c90d700 1 -- 172.21.15.81:0/529094304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05680684b0 con 0x7f05680b4580 2024-05-08T23:31:15.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.369+0000 7f054ffff700 1 -- 172.21.15.81:0/529094304 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f056001a040 con 0x7f05680b4580 2024-05-08T23:31:15.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.369+0000 7f054ffff700 1 --2- 172.21.15.81:0/529094304 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0554066730 0x7f0554068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.369+0000 7f054ffff700 1 -- 172.21.15.81:0/529094304 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05600854b0 con 0x7f05680b4580 2024-05-08T23:31:15.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.369+0000 7f05677fe700 1 --2- 172.21.15.81:0/529094304 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0554066730 0x7f0554068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.369+0000 7f05677fe700 1 --2- 172.21.15.81:0/529094304 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0554066730 0x7f0554068bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0558000f60 tx=0x7f0558009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:15.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.377+0000 7f054ffff700 1 -- 172.21.15.81:0/529094304 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f05600529d0 con 0x7f05680b4580 2024-05-08T23:31:15.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.545+0000 7f056c90d700 1 -- 172.21.15.81:0/529094304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f056806bca0 con 0x7f05680b4580 2024-05-08T23:31:15.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.545+0000 7f054ffff700 1 -- 172.21.15.81:0/529094304 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f0560055ff0 con 0x7f05680b4580 2024-05-08T23:31:15.549 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:15.550 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:15.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.545+0000 7f056c90d700 1 -- 172.21.15.81:0/529094304 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0554066730 msgr2=0x7f0554068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.545+0000 7f056c90d700 1 --2- 172.21.15.81:0/529094304 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0554066730 0x7f0554068bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0558000f60 tx=0x7f0558009040 comp rx=0 tx=0).stop 2024-05-08T23:31:15.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.545+0000 7f056c90d700 1 -- 172.21.15.81:0/529094304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05680b4580 msgr2=0x7f0568151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.545+0000 7f056c90d700 1 --2- 172.21.15.81:0/529094304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05680b4580 0x7f0568151160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0568159c00 tx=0x7f0560004af0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.545+0000 7f056c90d700 1 -- 172.21.15.81:0/529094304 shutdown_connections 2024-05-08T23:31:15.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.545+0000 7f056c90d700 1 --2- 172.21.15.81:0/529094304 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0554066730 0x7f0554068bf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.545+0000 7f056c90d700 1 --2- 172.21.15.81:0/529094304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05680b4580 0x7f0568151160 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.545+0000 7f056c90d700 1 --2- 172.21.15.81:0/529094304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05680b41a0 0x7f05680c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.545+0000 7f056c90d700 1 --2- 172.21.15.81:0/529094304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f056806e400 0x7f0568146160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.545+0000 7f056c90d700 1 -- 172.21.15.81:0/529094304 >> 172.21.15.81:0/529094304 conn(0x7f05680b16c0 msgr2=0x7f05680b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:15.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.549+0000 7f056c90d700 1 -- 172.21.15.81:0/529094304 shutdown_connections 2024-05-08T23:31:15.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.549+0000 7f056c90d700 1 -- 172.21.15.81:0/529094304 wait complete. 2024-05-08T23:31:15.564 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:15.565 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:15.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9feca3700 1 Processor -- start 2024-05-08T23:31:15.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9feca3700 1 -- start start 2024-05-08T23:31:15.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9feca3700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa9f8160a30 0x7fa9f80d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9feca3700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f80b66d0 con 0x7fa9f80d3c50 2024-05-08T23:31:15.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9feca3700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f80d11c0 con 0x7fa9f80d1e90 2024-05-08T23:31:15.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9feca3700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f80d14a0 con 0x7fa9f8160a30 2024-05-08T23:31:15.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9fca3f700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa9f8160a30 0x7fa9f80d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9fca3f700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa9f8160a30 0x7fa9f80d0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58586/0 (socket says 172.21.15.81:58586) 2024-05-08T23:31:15.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9fca3f700 1 -- 172.21.15.81:0/617517037 learned_addr learned my addr 172.21.15.81:0/617517037 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:15.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9fca3f700 1 -- 172.21.15.81:0/617517037 >> v1:172.21.15.117:6789/0 conn(0x7fa9f80d1e90 legacy=0x7fa9f80b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9fca3f700 1 -- 172.21.15.81:0/617517037 >> v1:172.21.15.81:6789/0 conn(0x7fa9f80d3c50 legacy=0x7fa9f80d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9fca3f700 1 -- 172.21.15.81:0/617517037 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f80b6080 con 0x7fa9f8160a30 2024-05-08T23:31:15.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9fca3f700 1 --2- 172.21.15.81:0/617517037 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9f8160a30 0x7fa9f80d0480 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009910 tx=0x7fa9ec009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5054cff89f1c73a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:15.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9f77fe700 1 -- 172.21.15.81:0/617517037 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec00ba60 con 0x7fa9f8160a30 2024-05-08T23:31:15.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9f77fe700 1 -- 172.21.15.81:0/617517037 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec00bbc0 con 0x7fa9f8160a30 2024-05-08T23:31:15.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.681+0000 7fa9f77fe700 1 -- 172.21.15.81:0/617517037 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec009e60 con 0x7fa9f8160a30 2024-05-08T23:31:15.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9feca3700 1 -- 172.21.15.81:0/617517037 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9f8160a30 msgr2=0x7fa9f80d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9feca3700 1 --2- 172.21.15.81:0/617517037 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9f8160a30 0x7fa9f80d0480 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009910 tx=0x7fa9ec009c20 comp rx=0 tx=0).stop 2024-05-08T23:31:15.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9feca3700 1 -- 172.21.15.81:0/617517037 shutdown_connections 2024-05-08T23:31:15.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9feca3700 1 --2- 172.21.15.81:0/617517037 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9f8160a30 0x7fa9f80d0480 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9feca3700 1 -- 172.21.15.81:0/617517037 >> 172.21.15.81:0/617517037 conn(0x7fa9f809c500 msgr2=0x7fa9f80d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:15.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9feca3700 1 -- 172.21.15.81:0/617517037 shutdown_connections 2024-05-08T23:31:15.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9feca3700 1 -- 172.21.15.81:0/617517037 wait complete. 2024-05-08T23:31:15.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9feca3700 1 Processor -- start 2024-05-08T23:31:15.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9feca3700 1 -- start start 2024-05-08T23:31:15.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9feca3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9f80d1e90 0x7fa9f80ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9feca3700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9f80d3c50 0x7fa9f8152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9feca3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9f8160a30 0x7fa9f809d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9feca3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f80a1080 con 0x7fa9f80d1e90 2024-05-08T23:31:15.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9feca3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f81f4020 con 0x7fa9f8160a30 2024-05-08T23:31:15.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9feca3700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f80cd330 con 0x7fa9f80d3c50 2024-05-08T23:31:15.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9fca3f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9f80d1e90 0x7fa9f80ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9fca3f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9f80d1e90 0x7fa9f80ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58600/0 (socket says 172.21.15.81:58600) 2024-05-08T23:31:15.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.685+0000 7fa9fca3f700 1 -- 172.21.15.81:0/2463709288 learned_addr learned my addr 172.21.15.81:0/2463709288 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:15.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9fd240700 1 --2- 172.21.15.81:0/2463709288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9f8160a30 0x7fa9f809d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9f7fff700 1 --2- 172.21.15.81:0/2463709288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9f80d3c50 0x7fa9f8152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9fca3f700 1 -- 172.21.15.81:0/2463709288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9f80d3c50 msgr2=0x7fa9f8152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9fca3f700 1 --2- 172.21.15.81:0/2463709288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9f80d3c50 0x7fa9f8152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9fca3f700 1 -- 172.21.15.81:0/2463709288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9f8160a30 msgr2=0x7fa9f809d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9fca3f700 1 --2- 172.21.15.81:0/2463709288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9f8160a30 0x7fa9f809d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9fca3f700 1 -- 172.21.15.81:0/2463709288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec009570 con 0x7fa9f80d1e90 2024-05-08T23:31:15.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9fd240700 1 --2- 172.21.15.81:0/2463709288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9f8160a30 0x7fa9f809d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:15.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9fca3f700 1 --2- 172.21.15.81:0/2463709288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9f80d1e90 0x7fa9f80ccdf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec014040 tx=0x7fa9ec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:15.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9f57fa700 1 -- 172.21.15.81:0/2463709288 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0038c0 con 0x7fa9f80d1e90 2024-05-08T23:31:15.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9f57fa700 1 -- 172.21.15.81:0/2463709288 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec01e070 con 0x7fa9f80d1e90 2024-05-08T23:31:15.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9f57fa700 1 -- 172.21.15.81:0/2463709288 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec003c60 con 0x7fa9f80d1e90 2024-05-08T23:31:15.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9feca3700 1 -- 172.21.15.81:0/2463709288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f80a0ac0 con 0x7fa9f80d1e90 2024-05-08T23:31:15.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9feca3700 1 -- 172.21.15.81:0/2463709288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f81f3cb0 con 0x7fa9f80d1e90 2024-05-08T23:31:15.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9feca3700 1 -- 172.21.15.81:0/2463709288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f81f3ae0 con 0x7fa9f80d1e90 2024-05-08T23:31:15.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9f57fa700 1 -- 172.21.15.81:0/2463709288 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa9ec025480 con 0x7fa9f80d1e90 2024-05-08T23:31:15.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9f57fa700 1 --2- 172.21.15.81:0/2463709288 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa9dc066730 0x7fa9dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:15.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9f57fa700 1 -- 172.21.15.81:0/2463709288 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9ec020070 con 0x7fa9f80d1e90 2024-05-08T23:31:15.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.689+0000 7fa9f7fff700 1 --2- 172.21.15.81:0/2463709288 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa9dc066730 0x7fa9dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:15.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.693+0000 7fa9f7fff700 1 --2- 172.21.15.81:0/2463709288 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa9dc066730 0x7fa9dc068bf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa9f809d290 tx=0x7fa9e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:15.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.697+0000 7fa9f57fa700 1 -- 172.21.15.81:0/2463709288 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa9ec052b70 con 0x7fa9f80d1e90 2024-05-08T23:31:15.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.865+0000 7fa9feca3700 1 -- 172.21.15.81:0/2463709288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9f80684b0 con 0x7fa9f80d1e90 2024-05-08T23:31:15.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.869+0000 7fa9f57fa700 1 -- 172.21.15.81:0/2463709288 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9ec02a0a0 con 0x7fa9f80d1e90 2024-05-08T23:31:15.872 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:15.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.869+0000 7fa9feca3700 1 -- 172.21.15.81:0/2463709288 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa9dc066730 msgr2=0x7fa9dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.869+0000 7fa9feca3700 1 --2- 172.21.15.81:0/2463709288 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa9dc066730 0x7fa9dc068bf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa9f809d290 tx=0x7fa9e4009040 comp rx=0 tx=0).stop 2024-05-08T23:31:15.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.869+0000 7fa9feca3700 1 -- 172.21.15.81:0/2463709288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9f80d1e90 msgr2=0x7fa9f80ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:15.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.869+0000 7fa9feca3700 1 --2- 172.21.15.81:0/2463709288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9f80d1e90 0x7fa9f80ccdf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec014040 tx=0x7fa9ec004770 comp rx=0 tx=0).stop 2024-05-08T23:31:15.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.869+0000 7fa9feca3700 1 -- 172.21.15.81:0/2463709288 shutdown_connections 2024-05-08T23:31:15.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.869+0000 7fa9feca3700 1 --2- 172.21.15.81:0/2463709288 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa9dc066730 0x7fa9dc068bf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.869+0000 7fa9feca3700 1 --2- 172.21.15.81:0/2463709288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9f8160a30 0x7fa9f809d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.869+0000 7fa9feca3700 1 --2- 172.21.15.81:0/2463709288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9f80d3c50 0x7fa9f8152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.869+0000 7fa9feca3700 1 --2- 172.21.15.81:0/2463709288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9f80d1e90 0x7fa9f80ccdf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:15.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.869+0000 7fa9feca3700 1 -- 172.21.15.81:0/2463709288 >> 172.21.15.81:0/2463709288 conn(0x7fa9f809c500 msgr2=0x7fa9f80cf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:15.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.869+0000 7fa9feca3700 1 -- 172.21.15.81:0/2463709288 shutdown_connections 2024-05-08T23:31:15.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:15.869+0000 7fa9feca3700 1 -- 172.21.15.81:0/2463709288 wait complete. 2024-05-08T23:31:15.887 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:16.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.005+0000 7f2027fff700 1 Processor -- start 2024-05-08T23:31:16.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.005+0000 7f2027fff700 1 -- start start 2024-05-08T23:31:16.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.005+0000 7f2027fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f202815b080 0x7f2028099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.005+0000 7f2027fff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202809a660 con 0x7f20280b53c0 2024-05-08T23:31:16.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.005+0000 7f2027fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202815b460 con 0x7f20280b2380 2024-05-08T23:31:16.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.005+0000 7f2027fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202815b700 con 0x7f202815b080 2024-05-08T23:31:16.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.005+0000 7f201e7fc700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f20280b53c0 0x7f20280b1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34408/0 (socket says 172.21.15.81:34408) 2024-05-08T23:31:16.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.005+0000 7f201e7fc700 1 -- 172.21.15.81:0/1469158555 learned_addr learned my addr 172.21.15.81:0/1469158555 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:16.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f2026ffd700 1 --2- 172.21.15.81:0/1469158555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f202815b080 0x7f2028099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889173340 0 0) 0x7f202809a660 con 0x7f20280b53c0 2024-05-08T23:31:16.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20140035d0 con 0x7f20280b53c0 2024-05-08T23:31:16.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864795689 0 0) 0x7f202815b460 con 0x7f20280b2380 2024-05-08T23:31:16.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f202809a660 con 0x7f20280b2380 2024-05-08T23:31:16.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2267365994 0 0) 0x7f20140035d0 con 0x7f20280b53c0 2024-05-08T23:31:16.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f202815b080 msgr2=0x7f2028099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:16.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 --2- 172.21.15.81:0/1469158555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f202815b080 0x7f2028099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 >> v1:172.21.15.117:6789/0 conn(0x7f20280b2380 legacy=0x7f202815c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202809a380 con 0x7f20280b53c0 2024-05-08T23:31:16.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f2010001670 con 0x7f20280b53c0 2024-05-08T23:31:16.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 >> v1:172.21.15.81:6789/0 conn(0x7f20280b53c0 legacy=0x7f20280b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 --2- 172.21.15.81:0/1469158555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f202815b080 0x7f2014003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 --2- 172.21.15.81:0/1469158555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2014004140 0x7f2014004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 --2- 172.21.15.81:0/1469158555 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2014004db0 0x7f2014005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202815b700 con 0x7f2014004db0 2024-05-08T23:31:16.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201400bb10 con 0x7f2014004140 2024-05-08T23:31:16.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201400bcd0 con 0x7f202815b080 2024-05-08T23:31:16.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20277fe700 1 --2- 172.21.15.81:0/1469158555 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2014004db0 0x7f2014005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f2026ffd700 1 --2- 172.21.15.81:0/1469158555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f202815b080 0x7f2014003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f201e7fc700 1 --2- 172.21.15.81:0/1469158555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2014004140 0x7f2014004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20277fe700 1 -- 172.21.15.81:0/1469158555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f202815b080 msgr2=0x7f2014003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:16.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20277fe700 1 --2- 172.21.15.81:0/1469158555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f202815b080 0x7f2014003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20277fe700 1 -- 172.21.15.81:0/1469158555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2014004140 msgr2=0x7f2014004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:16.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20277fe700 1 --2- 172.21.15.81:0/1469158555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2014004140 0x7f2014004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20277fe700 1 -- 172.21.15.81:0/1469158555 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20140035d0 con 0x7f2014004db0 2024-05-08T23:31:16.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f2026ffd700 1 --2- 172.21.15.81:0/1469158555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f202815b080 0x7f2014003be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:16.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20277fe700 1 --2- 172.21.15.81:0/1469158555 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2014004db0 0x7f2014005230 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2020002230 tx=0x7f20140058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538243b3cc4e0a4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:16.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020001f30 con 0x7f2014004db0 2024-05-08T23:31:16.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202000ce60 con 0x7f2014004db0 2024-05-08T23:31:16.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20267fc700 1 -- 172.21.15.81:0/1469158555 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020003c60 con 0x7f2014004db0 2024-05-08T23:31:16.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f2027fff700 1 -- 172.21.15.81:0/1469158555 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2014004db0 msgr2=0x7f2014005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f2027fff700 1 --2- 172.21.15.81:0/1469158555 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2014004db0 0x7f2014005230 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2020002230 tx=0x7f20140058a0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f20277fe700 1 -- 172.21.15.81:0/1469158555 reap_dead start 2024-05-08T23:31:16.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f2027fff700 1 -- 172.21.15.81:0/1469158555 shutdown_connections 2024-05-08T23:31:16.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f2027fff700 1 -- 172.21.15.81:0/1469158555 >> 172.21.15.81:0/1469158555 conn(0x7f20280b2ae0 msgr2=0x7f20280b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:16.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f2027fff700 1 -- 172.21.15.81:0/1469158555 shutdown_connections 2024-05-08T23:31:16.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.009+0000 7f2027fff700 1 -- 172.21.15.81:0/1469158555 wait complete. 2024-05-08T23:31:16.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f2027fff700 1 Processor -- start 2024-05-08T23:31:16.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f2027fff700 1 -- start start 2024-05-08T23:31:16.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f2027fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2028143f50 0x7f202809e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f2027fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2028144330 0x7f202809e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f2027fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2028159280 0x7f2028159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f2027fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2028145230 con 0x7f2028159280 2024-05-08T23:31:16.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f2027fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2028144900 con 0x7f2028143f50 2024-05-08T23:31:16.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f2027fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2028144be0 con 0x7f2028144330 2024-05-08T23:31:16.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f20277fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2028159280 0x7f2028159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f20277fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2028159280 0x7f2028159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57146/0 (socket says 172.21.15.81:57146) 2024-05-08T23:31:16.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f20277fe700 1 -- 172.21.15.81:0/36780368 learned_addr learned my addr 172.21.15.81:0/36780368 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:16.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f201e7fc700 1 --2- 172.21.15.81:0/36780368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2028144330 0x7f202809e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f2026ffd700 1 --2- 172.21.15.81:0/36780368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2028143f50 0x7f202809e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f20277fe700 1 -- 172.21.15.81:0/36780368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2028144330 msgr2=0x7f202809e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f20277fe700 1 --2- 172.21.15.81:0/36780368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2028144330 0x7f202809e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f20277fe700 1 -- 172.21.15.81:0/36780368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2028143f50 msgr2=0x7f202809e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f20277fe700 1 --2- 172.21.15.81:0/36780368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2028143f50 0x7f202809e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.013+0000 7f20277fe700 1 -- 172.21.15.81:0/36780368 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2028144ec0 con 0x7f2028159280 2024-05-08T23:31:16.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.017+0000 7f20277fe700 1 --2- 172.21.15.81:0/36780368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2028159280 0x7f2028159730 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2020015040 tx=0x7f2020001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:16.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.017+0000 7f2024ff9700 1 -- 172.21.15.81:0/36780368 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200050d0 con 0x7f2028159280 2024-05-08T23:31:16.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.017+0000 7f2024ff9700 1 -- 172.21.15.81:0/36780368 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2020005250 con 0x7f2028159280 2024-05-08T23:31:16.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.017+0000 7f2027fff700 1 -- 172.21.15.81:0/36780368 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202809b9b0 con 0x7f2028159280 2024-05-08T23:31:16.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.017+0000 7f2024ff9700 1 -- 172.21.15.81:0/36780368 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020005600 con 0x7f2028159280 2024-05-08T23:31:16.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.017+0000 7f2027fff700 1 -- 172.21.15.81:0/36780368 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2028159fb0 con 0x7f2028159280 2024-05-08T23:31:16.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.017+0000 7f2027fff700 1 -- 172.21.15.81:0/36780368 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20280672a0 con 0x7f2028159280 2024-05-08T23:31:16.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.017+0000 7f2024ff9700 1 -- 172.21.15.81:0/36780368 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2020019040 con 0x7f2028159280 2024-05-08T23:31:16.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.017+0000 7f2024ff9700 1 --2- 172.21.15.81:0/36780368 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2008066760 0x7f2008068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.017+0000 7f2024ff9700 1 -- 172.21.15.81:0/36780368 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2020084e70 con 0x7f2028159280 2024-05-08T23:31:16.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.017+0000 7f2026ffd700 1 --2- 172.21.15.81:0/36780368 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2008066760 0x7f2008068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.021+0000 7f2026ffd700 1 --2- 172.21.15.81:0/36780368 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2008066760 0x7f2008068c20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2018009c70 tx=0x7f2018009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:16.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.025+0000 7f2024ff9700 1 -- 172.21.15.81:0/36780368 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2020052290 con 0x7f2028159280 2024-05-08T23:31:16.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.193+0000 7f2027fff700 1 -- 172.21.15.81:0/36780368 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2028145050 con 0x7f2028159280 2024-05-08T23:31:16.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.193+0000 7f2024ff9700 1 -- 172.21.15.81:0/36780368 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f20200011c0 con 0x7f2028159280 2024-05-08T23:31:16.199 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:16.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.197+0000 7f2027fff700 1 -- 172.21.15.81:0/36780368 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2008066760 msgr2=0x7f2008068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.197+0000 7f2027fff700 1 --2- 172.21.15.81:0/36780368 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2008066760 0x7f2008068c20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2018009c70 tx=0x7f2018009240 comp rx=0 tx=0).stop 2024-05-08T23:31:16.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.197+0000 7f2027fff700 1 -- 172.21.15.81:0/36780368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2028159280 msgr2=0x7f2028159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.197+0000 7f2027fff700 1 --2- 172.21.15.81:0/36780368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2028159280 0x7f2028159730 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2020015040 tx=0x7f2020001650 comp rx=0 tx=0).stop 2024-05-08T23:31:16.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.197+0000 7f2027fff700 1 -- 172.21.15.81:0/36780368 shutdown_connections 2024-05-08T23:31:16.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.197+0000 7f2027fff700 1 --2- 172.21.15.81:0/36780368 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2008066760 0x7f2008068c20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.197+0000 7f2027fff700 1 --2- 172.21.15.81:0/36780368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2028159280 0x7f2028159730 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.197+0000 7f2027fff700 1 --2- 172.21.15.81:0/36780368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2028144330 0x7f202809e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.197+0000 7f2027fff700 1 --2- 172.21.15.81:0/36780368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2028143f50 0x7f202809e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.197+0000 7f2027fff700 1 -- 172.21.15.81:0/36780368 >> 172.21.15.81:0/36780368 conn(0x7f20280b2ae0 msgr2=0x7f20280b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:16.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.197+0000 7f2027fff700 1 -- 172.21.15.81:0/36780368 shutdown_connections 2024-05-08T23:31:16.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.197+0000 7f2027fff700 1 -- 172.21.15.81:0/36780368 wait complete. 2024-05-08T23:31:16.215 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:16.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.329+0000 7f59e624a700 1 Processor -- start 2024-05-08T23:31:16.335 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.329+0000 7f59e624a700 1 -- start start 2024-05-08T23:31:16.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.329+0000 7f59e624a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f59e0160920 0x7f59e00ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e624a700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e00d3d40 con 0x7f59e00d1da0 2024-05-08T23:31:16.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e624a700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e00d0d00 con 0x7f59e00d0050 2024-05-08T23:31:16.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e624a700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e00d0fe0 con 0x7f59e0160920 2024-05-08T23:31:16.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e5248700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f59e0160920 0x7f59e00ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e4a47700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f59e00d1da0 0x7f59e00cf940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34416/0 (socket says 172.21.15.81:34416) 2024-05-08T23:31:16.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e4a47700 1 -- 172.21.15.81:0/562793984 learned_addr learned my addr 172.21.15.81:0/562793984 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:16.337 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59d3fff700 1 -- 172.21.15.81:0/562793984 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164990072 0 0) 0x7f59e00d3d40 con 0x7f59e00d1da0 2024-05-08T23:31:16.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59d3fff700 1 -- 172.21.15.81:0/562793984 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59cc0035d0 con 0x7f59e00d1da0 2024-05-08T23:31:16.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e5248700 1 -- 172.21.15.81:0/562793984 >> v1:172.21.15.117:6789/0 conn(0x7f59e00d0050 legacy=0x7f59e00b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e5248700 1 -- 172.21.15.81:0/562793984 >> v1:172.21.15.81:6789/0 conn(0x7f59e00d1da0 legacy=0x7f59e00cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e5248700 1 -- 172.21.15.81:0/562793984 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e00d12c0 con 0x7f59e0160920 2024-05-08T23:31:16.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e5248700 1 --2- 172.21.15.81:0/562793984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e0160920 0x7f59e00ce6f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009850 tx=0x7f59d4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43676a86e768ea1d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:16.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59d3fff700 1 -- 172.21.15.81:0/562793984 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4011830 con 0x7f59e0160920 2024-05-08T23:31:16.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59d3fff700 1 -- 172.21.15.81:0/562793984 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d4011990 con 0x7f59e0160920 2024-05-08T23:31:16.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59d3fff700 1 -- 172.21.15.81:0/562793984 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4011d00 con 0x7f59e0160920 2024-05-08T23:31:16.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e624a700 1 -- 172.21.15.81:0/562793984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e0160920 msgr2=0x7f59e00ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e624a700 1 --2- 172.21.15.81:0/562793984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e0160920 0x7f59e00ce6f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009850 tx=0x7f59d4009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:16.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e624a700 1 -- 172.21.15.81:0/562793984 shutdown_connections 2024-05-08T23:31:16.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e624a700 1 --2- 172.21.15.81:0/562793984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e0160920 0x7f59e00ce6f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e624a700 1 -- 172.21.15.81:0/562793984 >> 172.21.15.81:0/562793984 conn(0x7f59e009c340 msgr2=0x7f59e00d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:16.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e624a700 1 -- 172.21.15.81:0/562793984 shutdown_connections 2024-05-08T23:31:16.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.333+0000 7f59e624a700 1 -- 172.21.15.81:0/562793984 wait complete. 2024-05-08T23:31:16.340 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e624a700 1 Processor -- start 2024-05-08T23:31:16.340 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e624a700 1 -- start start 2024-05-08T23:31:16.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e624a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e00d0050 0x7f59e0158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e624a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e00d1da0 0x7f59e015e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e624a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e0160920 0x7f59e00b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e624a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e015f4f0 con 0x7f59e00d0050 2024-05-08T23:31:16.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e624a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e015f740 con 0x7f59e0160920 2024-05-08T23:31:16.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e624a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e015fa20 con 0x7f59e00d1da0 2024-05-08T23:31:16.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e5248700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e00d0050 0x7f59e0158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e5248700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e00d0050 0x7f59e0158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57168/0 (socket says 172.21.15.81:57168) 2024-05-08T23:31:16.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e5248700 1 -- 172.21.15.81:0/163454533 learned_addr learned my addr 172.21.15.81:0/163454533 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:16.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e4a47700 1 --2- 172.21.15.81:0/163454533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e00d1da0 0x7f59e015e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e5a49700 1 --2- 172.21.15.81:0/163454533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e0160920 0x7f59e00b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e5248700 1 -- 172.21.15.81:0/163454533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e00d1da0 msgr2=0x7f59e015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e5248700 1 --2- 172.21.15.81:0/163454533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e00d1da0 0x7f59e015e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e5248700 1 -- 172.21.15.81:0/163454533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e0160920 msgr2=0x7f59e00b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e5248700 1 --2- 172.21.15.81:0/163454533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e0160920 0x7f59e00b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e5248700 1 -- 172.21.15.81:0/163454533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d40094b0 con 0x7f59e00d0050 2024-05-08T23:31:16.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e4a47700 1 --2- 172.21.15.81:0/163454533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e00d1da0 0x7f59e015e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:16.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e5248700 1 --2- 172.21.15.81:0/163454533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e00d0050 0x7f59e0158c60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009ba0 tx=0x7f59d4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:16.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59d1ffb700 1 -- 172.21.15.81:0/163454533 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40036c0 con 0x7f59e00d0050 2024-05-08T23:31:16.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59d1ffb700 1 -- 172.21.15.81:0/163454533 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d401e070 con 0x7f59e00d0050 2024-05-08T23:31:16.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e624a700 1 -- 172.21.15.81:0/163454533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e0160e80 con 0x7f59e00d0050 2024-05-08T23:31:16.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59e624a700 1 -- 172.21.15.81:0/163454533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e015ffe0 con 0x7f59e00d0050 2024-05-08T23:31:16.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.337+0000 7f59d1ffb700 1 -- 172.21.15.81:0/163454533 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4003a80 con 0x7f59e00d0050 2024-05-08T23:31:16.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.341+0000 7f59e624a700 1 -- 172.21.15.81:0/163454533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e00684b0 con 0x7f59e00d0050 2024-05-08T23:31:16.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.341+0000 7f59d1ffb700 1 -- 172.21.15.81:0/163454533 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59d4003be0 con 0x7f59e00d0050 2024-05-08T23:31:16.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.341+0000 7f59d1ffb700 1 --2- 172.21.15.81:0/163454533 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f59c4066730 0x7f59c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.341+0000 7f59e4a47700 1 --2- 172.21.15.81:0/163454533 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f59c4066730 0x7f59c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.341+0000 7f59d1ffb700 1 -- 172.21.15.81:0/163454533 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59d40850d0 con 0x7f59e00d0050 2024-05-08T23:31:16.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.341+0000 7f59e4a47700 1 --2- 172.21.15.81:0/163454533 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f59c4066730 0x7f59c4068bf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f59e00c9a30 tx=0x7f59dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:16.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.345+0000 7f59d1ffb700 1 -- 172.21.15.81:0/163454533 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f59d4052430 con 0x7f59e00d0050 2024-05-08T23:31:16.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.517+0000 7f59e624a700 1 -- 172.21.15.81:0/163454533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f59e00cf510 con 0x7f59e00d0050 2024-05-08T23:31:16.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.521+0000 7f59d1ffb700 1 -- 172.21.15.81:0/163454533 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f59d4055a50 con 0x7f59e00d0050 2024-05-08T23:31:16.524 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:16.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.521+0000 7f59e624a700 1 -- 172.21.15.81:0/163454533 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f59c4066730 msgr2=0x7f59c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.521+0000 7f59e624a700 1 --2- 172.21.15.81:0/163454533 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f59c4066730 0x7f59c4068bf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f59e00c9a30 tx=0x7f59dc009040 comp rx=0 tx=0).stop 2024-05-08T23:31:16.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.521+0000 7f59e624a700 1 -- 172.21.15.81:0/163454533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e00d0050 msgr2=0x7f59e0158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.521+0000 7f59e624a700 1 --2- 172.21.15.81:0/163454533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e00d0050 0x7f59e0158c60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009ba0 tx=0x7f59d4004770 comp rx=0 tx=0).stop 2024-05-08T23:31:16.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.521+0000 7f59e624a700 1 -- 172.21.15.81:0/163454533 shutdown_connections 2024-05-08T23:31:16.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.521+0000 7f59e624a700 1 --2- 172.21.15.81:0/163454533 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f59c4066730 0x7f59c4068bf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.521+0000 7f59e624a700 1 --2- 172.21.15.81:0/163454533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e0160920 0x7f59e00b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.521+0000 7f59e624a700 1 --2- 172.21.15.81:0/163454533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e00d1da0 0x7f59e015e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.521+0000 7f59e624a700 1 --2- 172.21.15.81:0/163454533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e00d0050 0x7f59e0158c60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.521+0000 7f59e624a700 1 -- 172.21.15.81:0/163454533 >> 172.21.15.81:0/163454533 conn(0x7f59e009c340 msgr2=0x7f59e00d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:16.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.521+0000 7f59e624a700 1 -- 172.21.15.81:0/163454533 shutdown_connections 2024-05-08T23:31:16.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.521+0000 7f59e624a700 1 -- 172.21.15.81:0/163454533 wait complete. 2024-05-08T23:31:16.539 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:31:16.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.653+0000 7fd4ee469700 1 Processor -- start 2024-05-08T23:31:16.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.653+0000 7fd4ee469700 1 -- start start 2024-05-08T23:31:16.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4ee469700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd4e8153ea0 0x7fd4e80ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4ee469700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e815a820 0x7fd4e8071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4ee469700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd4e8072010 0x7fd4e815d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4ee469700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e80ce620 con 0x7fd4e8153ea0 2024-05-08T23:31:16.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4ee469700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e8072440 con 0x7fd4e8072010 2024-05-08T23:31:16.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4ee469700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8154280 con 0x7fd4e815a820 2024-05-08T23:31:16.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4e7fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd4e8153ea0 0x7fd4e80ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4e7fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd4e8153ea0 0x7fd4e80ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57178/0 (socket says 172.21.15.81:57178) 2024-05-08T23:31:16.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4e7fff700 1 -- 172.21.15.81:0/1699375318 learned_addr learned my addr 172.21.15.81:0/1699375318 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:16.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4e77fe700 1 --2- 172.21.15.81:0/1699375318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e815a820 0x7fd4e8071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4eca06700 1 --2- 172.21.15.81:0/1699375318 >> v2:172.21.15.117:3300/0 conn(0x7fd4e8072010 0x7fd4e815d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4e7fff700 1 -- 172.21.15.81:0/1699375318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e815a820 msgr2=0x7fd4e8071ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:16.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4e7fff700 1 --2- 172.21.15.81:0/1699375318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e815a820 0x7fd4e8071ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4e7fff700 1 -- 172.21.15.81:0/1699375318 >> v2:172.21.15.117:3300/0 conn(0x7fd4e8072010 msgr2=0x7fd4e815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:16.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4e7fff700 1 --2- 172.21.15.81:0/1699375318 >> v2:172.21.15.117:3300/0 conn(0x7fd4e8072010 0x7fd4e815d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4e7fff700 1 -- 172.21.15.81:0/1699375318 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e80cd580 con 0x7fd4e8153ea0 2024-05-08T23:31:16.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4eca06700 1 --2- 172.21.15.81:0/1699375318 >> v2:172.21.15.117:3300/0 conn(0x7fd4e8072010 0x7fd4e815d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:16.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4e77fe700 1 --2- 172.21.15.81:0/1699375318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e815a820 0x7fd4e8071ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:16.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4e7fff700 1 --2- 172.21.15.81:0/1699375318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4e8153ea0 0x7fd4e80ccf90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80ce920 tx=0x7fd4e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900a9fa792c73c36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:16.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4e6ffd700 1 -- 172.21.15.81:0/1699375318 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000e070 con 0x7fd4e8153ea0 2024-05-08T23:31:16.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4e6ffd700 1 -- 172.21.15.81:0/1699375318 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e000b9c0 con 0x7fd4e8153ea0 2024-05-08T23:31:16.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4e6ffd700 1 -- 172.21.15.81:0/1699375318 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000bd70 con 0x7fd4e8153ea0 2024-05-08T23:31:16.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4ee469700 1 -- 172.21.15.81:0/1699375318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4e8153ea0 msgr2=0x7fd4e80ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4ee469700 1 --2- 172.21.15.81:0/1699375318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4e8153ea0 0x7fd4e80ccf90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80ce920 tx=0x7fd4e000b330 comp rx=0 tx=0).stop 2024-05-08T23:31:16.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4ee469700 1 -- 172.21.15.81:0/1699375318 shutdown_connections 2024-05-08T23:31:16.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4ee469700 1 --2- 172.21.15.81:0/1699375318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e815a820 0x7fd4e8071ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4ee469700 1 --2- 172.21.15.81:0/1699375318 >> v2:172.21.15.117:3300/0 conn(0x7fd4e8072010 0x7fd4e815d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4ee469700 1 --2- 172.21.15.81:0/1699375318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4e8153ea0 0x7fd4e80ccf90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4ee469700 1 -- 172.21.15.81:0/1699375318 >> 172.21.15.81:0/1699375318 conn(0x7fd4e806a360 msgr2=0x7fd4e806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:16.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4ee469700 1 -- 172.21.15.81:0/1699375318 shutdown_connections 2024-05-08T23:31:16.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.657+0000 7fd4ee469700 1 -- 172.21.15.81:0/1699375318 wait complete. 2024-05-08T23:31:16.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.661+0000 7fd4ee469700 1 Processor -- start 2024-05-08T23:31:16.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.661+0000 7fd4ee469700 1 -- start start 2024-05-08T23:31:16.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.661+0000 7fd4ee469700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e8072010 0x7fd4e8154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.661+0000 7fd4ee469700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4e8153ea0 0x7fd4e81577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.661+0000 7fd4ee469700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e815a820 0x7fd4e81414f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.661+0000 7fd4ee469700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8142630 con 0x7fd4e8153ea0 2024-05-08T23:31:16.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.661+0000 7fd4ee469700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8142910 con 0x7fd4e815a820 2024-05-08T23:31:16.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.661+0000 7fd4ee469700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8142bf0 con 0x7fd4e8072010 2024-05-08T23:31:16.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.661+0000 7fd4e77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4e8153ea0 0x7fd4e81577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.661+0000 7fd4e77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4e8153ea0 0x7fd4e81577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57182/0 (socket says 172.21.15.81:57182) 2024-05-08T23:31:16.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.661+0000 7fd4e77fe700 1 -- 172.21.15.81:0/330809376 learned_addr learned my addr 172.21.15.81:0/330809376 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:16.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.661+0000 7fd4e7fff700 1 --2- 172.21.15.81:0/330809376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e8072010 0x7fd4e8154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.661+0000 7fd4eca06700 1 --2- 172.21.15.81:0/330809376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e815a820 0x7fd4e81414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4e77fe700 1 -- 172.21.15.81:0/330809376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e8072010 msgr2=0x7fd4e8154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4e77fe700 1 --2- 172.21.15.81:0/330809376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e8072010 0x7fd4e8154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4e77fe700 1 -- 172.21.15.81:0/330809376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e815a820 msgr2=0x7fd4e81414f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4e77fe700 1 --2- 172.21.15.81:0/330809376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e815a820 0x7fd4e81414f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4e77fe700 1 -- 172.21.15.81:0/330809376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e0009b30 con 0x7fd4e8153ea0 2024-05-08T23:31:16.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4e77fe700 1 --2- 172.21.15.81:0/330809376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4e8153ea0 0x7fd4e81577d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000e760 tx=0x7fd4d000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:16.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4e4ff9700 1 -- 172.21.15.81:0/330809376 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0004230 con 0x7fd4e8153ea0 2024-05-08T23:31:16.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4e4ff9700 1 -- 172.21.15.81:0/330809376 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d00043b0 con 0x7fd4e8153ea0 2024-05-08T23:31:16.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4ee469700 1 -- 172.21.15.81:0/330809376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e8142ed0 con 0x7fd4e8153ea0 2024-05-08T23:31:16.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4e4ff9700 1 -- 172.21.15.81:0/330809376 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d00036c0 con 0x7fd4e8153ea0 2024-05-08T23:31:16.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4ee469700 1 -- 172.21.15.81:0/330809376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e8159730 con 0x7fd4e8153ea0 2024-05-08T23:31:16.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4ee469700 1 -- 172.21.15.81:0/330809376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e80684b0 con 0x7fd4e8153ea0 2024-05-08T23:31:16.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4e4ff9700 1 -- 172.21.15.81:0/330809376 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd4d0003820 con 0x7fd4e8153ea0 2024-05-08T23:31:16.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4e4ff9700 1 --2- 172.21.15.81:0/330809376 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd4cc066730 0x7fd4cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:16.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4e4ff9700 1 -- 172.21.15.81:0/330809376 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4d0083d70 con 0x7fd4e8153ea0 2024-05-08T23:31:16.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4e7fff700 1 --2- 172.21.15.81:0/330809376 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd4cc066730 0x7fd4cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:16.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.665+0000 7fd4e7fff700 1 --2- 172.21.15.81:0/330809376 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd4cc066730 0x7fd4cc068bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000b370 tx=0x7fd4e0009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:16.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.673+0000 7fd4e4ff9700 1 -- 172.21.15.81:0/330809376 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd4d0051040 con 0x7fd4e8153ea0 2024-05-08T23:31:16.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.857+0000 7fd4ee469700 1 -- 172.21.15.81:0/330809376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd4e80cc9a0 con 0x7fd4e8153ea0 2024-05-08T23:31:16.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.857+0000 7fd4e4ff9700 1 -- 172.21.15.81:0/330809376 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd4d0020080 con 0x7fd4e8153ea0 2024-05-08T23:31:16.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.857+0000 7fd4ee469700 1 -- 172.21.15.81:0/330809376 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd4cc066730 msgr2=0x7fd4cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.857+0000 7fd4ee469700 1 --2- 172.21.15.81:0/330809376 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd4cc066730 0x7fd4cc068bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000b370 tx=0x7fd4e0009000 comp rx=0 tx=0).stop 2024-05-08T23:31:16.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.857+0000 7fd4ee469700 1 -- 172.21.15.81:0/330809376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4e8153ea0 msgr2=0x7fd4e81577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:16.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.857+0000 7fd4ee469700 1 --2- 172.21.15.81:0/330809376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4e8153ea0 0x7fd4e81577d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000e760 tx=0x7fd4d000eb20 comp rx=0 tx=0).stop 2024-05-08T23:31:16.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.861+0000 7fd4ee469700 1 -- 172.21.15.81:0/330809376 shutdown_connections 2024-05-08T23:31:16.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.861+0000 7fd4ee469700 1 --2- 172.21.15.81:0/330809376 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd4cc066730 0x7fd4cc068bf0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.861+0000 7fd4ee469700 1 --2- 172.21.15.81:0/330809376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e815a820 0x7fd4e81414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.861+0000 7fd4ee469700 1 --2- 172.21.15.81:0/330809376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4e8153ea0 0x7fd4e81577d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.861+0000 7fd4ee469700 1 --2- 172.21.15.81:0/330809376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4e8072010 0x7fd4e8154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:16.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.861+0000 7fd4ee469700 1 -- 172.21.15.81:0/330809376 >> 172.21.15.81:0/330809376 conn(0x7fd4e806a360 msgr2=0x7fd4e80701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:16.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.861+0000 7fd4ee469700 1 -- 172.21.15.81:0/330809376 shutdown_connections 2024-05-08T23:31:16.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:16.861+0000 7fd4ee469700 1 -- 172.21.15.81:0/330809376 wait complete. 2024-05-08T23:31:16.865 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:31:16.877 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi117.front.sepia.ceph.com... 2024-05-08T23:31:16.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:16.877 DEBUG:teuthology.orchestra.run.smithi117:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-05-08T23:31:16.910 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:31:16.910 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:31:16.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:16.910 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T23:31:16.947 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:31:16.947 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:31:16.955 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:31:16.956 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:31:16.963 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:17.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa82a5b7700 1 Processor -- start 2024-05-08T23:31:17.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa82a5b7700 1 -- start start 2024-05-08T23:31:17.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa82a5b7700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa8240cb970 0x7fa8240cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa82a5b7700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa82406c350 0x7fa82406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa82a5b7700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8240cd470 con 0x7fa82415aed0 2024-05-08T23:31:17.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa82a5b7700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82406cd00 con 0x7fa8240cb970 2024-05-08T23:31:17.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa82a5b7700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8240ced50 con 0x7fa82406c350 2024-05-08T23:31:17.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa8237fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa8240cb970 0x7fa8240cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa8237fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa8240cb970 0x7fa8240cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57186/0 (socket says 172.21.15.81:57186) 2024-05-08T23:31:17.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa8237fe700 1 -- 172.21.15.81:0/1592641325 learned_addr learned my addr 172.21.15.81:0/1592641325 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:17.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa828b54700 1 --2- 172.21.15.81:0/1592641325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa82406c350 0x7fa82406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa8237fe700 1 -- 172.21.15.81:0/1592641325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa82406c350 msgr2=0x7fa82406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:17.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa8237fe700 1 --2- 172.21.15.81:0/1592641325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa82406c350 0x7fa82406c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa8237fe700 1 -- 172.21.15.81:0/1592641325 >> v1:172.21.15.117:6789/0 conn(0x7fa82415aed0 legacy=0x7fa8240b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa8237fe700 1 -- 172.21.15.81:0/1592641325 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8240cceb0 con 0x7fa8240cb970 2024-05-08T23:31:17.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa8237fe700 1 --2- 172.21.15.81:0/1592641325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8240cb970 0x7fa8240cc7c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa818009910 tx=0x7fa818009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0bb7eab8807228c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:17.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa822ffd700 1 -- 172.21.15.81:0/1592641325 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81800ba60 con 0x7fa8240cb970 2024-05-08T23:31:17.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa822ffd700 1 -- 172.21.15.81:0/1592641325 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81800bbc0 con 0x7fa8240cb970 2024-05-08T23:31:17.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa822ffd700 1 -- 172.21.15.81:0/1592641325 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818009e60 con 0x7fa8240cb970 2024-05-08T23:31:17.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa82a5b7700 1 -- 172.21.15.81:0/1592641325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8240cb970 msgr2=0x7fa8240cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa82a5b7700 1 --2- 172.21.15.81:0/1592641325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8240cb970 0x7fa8240cc7c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa818009910 tx=0x7fa818009c20 comp rx=0 tx=0).stop 2024-05-08T23:31:17.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa82a5b7700 1 -- 172.21.15.81:0/1592641325 shutdown_connections 2024-05-08T23:31:17.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa82a5b7700 1 --2- 172.21.15.81:0/1592641325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa82406c350 0x7fa82406c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa82a5b7700 1 --2- 172.21.15.81:0/1592641325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8240cb970 0x7fa8240cc7c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa82a5b7700 1 -- 172.21.15.81:0/1592641325 >> 172.21.15.81:0/1592641325 conn(0x7fa82406a360 msgr2=0x7fa82406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:17.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa82a5b7700 1 -- 172.21.15.81:0/1592641325 shutdown_connections 2024-05-08T23:31:17.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.120+0000 7fa82a5b7700 1 -- 172.21.15.81:0/1592641325 wait complete. 2024-05-08T23:31:17.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa82a5b7700 1 Processor -- start 2024-05-08T23:31:17.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa82a5b7700 1 -- start start 2024-05-08T23:31:17.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa82a5b7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa82406c350 0x7fa82414ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa82a5b7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8240cb970 0x7fa82414c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa82a5b7700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa82415aed0 0x7fa824158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa82a5b7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82414b750 con 0x7fa82406c350 2024-05-08T23:31:17.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa82a5b7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82414b220 con 0x7fa8240cb970 2024-05-08T23:31:17.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa82a5b7700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82414b470 con 0x7fa82415aed0 2024-05-08T23:31:17.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa823fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa82406c350 0x7fa82414ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa828b54700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa82415aed0 0x7fa824158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa823fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa82406c350 0x7fa82414ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57202/0 (socket says 172.21.15.81:57202) 2024-05-08T23:31:17.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa8237fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8240cb970 0x7fa82414c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa823fff700 1 -- 172.21.15.81:0/3639728032 learned_addr learned my addr 172.21.15.81:0/3639728032 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:17.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa823fff700 1 -- 172.21.15.81:0/3639728032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa82415aed0 msgr2=0x7fa824158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa823fff700 1 --2- 172.21.15.81:0/3639728032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa82415aed0 0x7fa824158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa823fff700 1 -- 172.21.15.81:0/3639728032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8240cb970 msgr2=0x7fa82414c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa823fff700 1 --2- 172.21.15.81:0/3639728032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8240cb970 0x7fa82414c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa823fff700 1 -- 172.21.15.81:0/3639728032 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa818009570 con 0x7fa82406c350 2024-05-08T23:31:17.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa823fff700 1 --2- 172.21.15.81:0/3639728032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa82406c350 0x7fa82414ace0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa81400c500 tx=0x7fa81400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:17.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa820ff9700 1 -- 172.21.15.81:0/3639728032 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8140052d0 con 0x7fa82406c350 2024-05-08T23:31:17.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa820ff9700 1 -- 172.21.15.81:0/3639728032 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa814005430 con 0x7fa82406c350 2024-05-08T23:31:17.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.124+0000 7fa82a5b7700 1 -- 172.21.15.81:0/3639728032 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8240b0020 con 0x7fa82406c350 2024-05-08T23:31:17.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.128+0000 7fa820ff9700 1 -- 172.21.15.81:0/3639728032 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8140055a0 con 0x7fa82406c350 2024-05-08T23:31:17.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.128+0000 7fa82a5b7700 1 -- 172.21.15.81:0/3639728032 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8240b05e0 con 0x7fa82406c350 2024-05-08T23:31:17.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.128+0000 7fa820ff9700 1 -- 172.21.15.81:0/3639728032 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa8140038e0 con 0x7fa82406c350 2024-05-08T23:31:17.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.128+0000 7fa82a5b7700 1 -- 172.21.15.81:0/3639728032 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa824067670 con 0x7fa82406c350 2024-05-08T23:31:17.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.128+0000 7fa820ff9700 1 --2- 172.21.15.81:0/3639728032 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa810066730 0x7fa810068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.128+0000 7fa820ff9700 1 -- 172.21.15.81:0/3639728032 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa814083220 con 0x7fa82406c350 2024-05-08T23:31:17.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.128+0000 7fa8237fe700 1 --2- 172.21.15.81:0/3639728032 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa810066730 0x7fa810068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.128+0000 7fa8237fe700 1 --2- 172.21.15.81:0/3639728032 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa810066730 0x7fa810068bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa818000c00 tx=0x7fa818008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:17.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.136+0000 7fa820ff9700 1 -- 172.21.15.81:0/3639728032 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa814088060 con 0x7fa82406c350 2024-05-08T23:31:17.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.304+0000 7fa82a5b7700 1 -- 172.21.15.81:0/3639728032 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa824066960 con 0x7fa82406c350 2024-05-08T23:31:17.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.304+0000 7fa820ff9700 1 -- 172.21.15.81:0/3639728032 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fa814050590 con 0x7fa82406c350 2024-05-08T23:31:17.310 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:17.310 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:17.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.304+0000 7fa82a5b7700 1 -- 172.21.15.81:0/3639728032 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa810066730 msgr2=0x7fa810068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.304+0000 7fa82a5b7700 1 --2- 172.21.15.81:0/3639728032 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa810066730 0x7fa810068bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa818000c00 tx=0x7fa818008030 comp rx=0 tx=0).stop 2024-05-08T23:31:17.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.304+0000 7fa82a5b7700 1 -- 172.21.15.81:0/3639728032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa82406c350 msgr2=0x7fa82414ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.304+0000 7fa82a5b7700 1 --2- 172.21.15.81:0/3639728032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa82406c350 0x7fa82414ace0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa81400c500 tx=0x7fa81400c810 comp rx=0 tx=0).stop 2024-05-08T23:31:17.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.308+0000 7fa82a5b7700 1 -- 172.21.15.81:0/3639728032 shutdown_connections 2024-05-08T23:31:17.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.308+0000 7fa82a5b7700 1 --2- 172.21.15.81:0/3639728032 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa810066730 0x7fa810068bf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.308+0000 7fa82a5b7700 1 --2- 172.21.15.81:0/3639728032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa82415aed0 0x7fa824158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.308+0000 7fa82a5b7700 1 --2- 172.21.15.81:0/3639728032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8240cb970 0x7fa82414c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.308+0000 7fa82a5b7700 1 --2- 172.21.15.81:0/3639728032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa82406c350 0x7fa82414ace0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.308+0000 7fa82a5b7700 1 -- 172.21.15.81:0/3639728032 >> 172.21.15.81:0/3639728032 conn(0x7fa82406a360 msgr2=0x7fa8241576f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:17.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.308+0000 7fa82a5b7700 1 -- 172.21.15.81:0/3639728032 shutdown_connections 2024-05-08T23:31:17.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.308+0000 7fa82a5b7700 1 -- 172.21.15.81:0/3639728032 wait complete. 2024-05-08T23:31:17.325 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:17.325 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:17.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.432+0000 7f1b6e79d700 1 Processor -- start 2024-05-08T23:31:17.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.432+0000 7f1b6e79d700 1 -- start start 2024-05-08T23:31:17.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.432+0000 7f1b6e79d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1b6815d560 0x7f1b680d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.432+0000 7f1b6e79d700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b680ce480 con 0x7f1b6815d180 2024-05-08T23:31:17.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.432+0000 7f1b6e79d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b680ce760 con 0x7f1b680d2f60 2024-05-08T23:31:17.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.432+0000 7f1b6e79d700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b680cea40 con 0x7f1b6815d560 2024-05-08T23:31:17.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.432+0000 7f1b6cf9a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1b6815d560 0x7f1b680d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.432+0000 7f1b6d79b700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f1b6815d180 0x7f1b680cdcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34432/0 (socket says 172.21.15.81:34432) 2024-05-08T23:31:17.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.432+0000 7f1b6cf9a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1b6815d560 0x7f1b680d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57218/0 (socket says 172.21.15.81:57218) 2024-05-08T23:31:17.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.432+0000 7f1b6d79b700 1 -- 172.21.15.81:0/4152566975 learned_addr learned my addr 172.21.15.81:0/4152566975 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:17.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b5bfff700 1 -- 172.21.15.81:0/4152566975 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462205576 0 0) 0x7f1b680ce760 con 0x7f1b680d2f60 2024-05-08T23:31:17.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b5bfff700 1 -- 172.21.15.81:0/4152566975 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b540035d0 con 0x7f1b680d2f60 2024-05-08T23:31:17.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b5bfff700 1 -- 172.21.15.81:0/4152566975 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981275226 0 0) 0x7f1b680ce480 con 0x7f1b6815d180 2024-05-08T23:31:17.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6cf9a700 1 -- 172.21.15.81:0/4152566975 >> v1:172.21.15.117:6789/0 conn(0x7f1b680d2f60 legacy=0x7f1b680b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6cf9a700 1 -- 172.21.15.81:0/4152566975 >> v1:172.21.15.81:6789/0 conn(0x7f1b6815d180 legacy=0x7f1b680cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6cf9a700 1 -- 172.21.15.81:0/4152566975 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b680ced20 con 0x7f1b6815d560 2024-05-08T23:31:17.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6cf9a700 1 --2- 172.21.15.81:0/4152566975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b6815d560 0x7f1b680d1bf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009850 tx=0x7f1b5c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710d5947828d2160 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:17.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b5bfff700 1 -- 172.21.15.81:0/4152566975 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c011830 con 0x7f1b6815d560 2024-05-08T23:31:17.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b5bfff700 1 -- 172.21.15.81:0/4152566975 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5c011990 con 0x7f1b6815d560 2024-05-08T23:31:17.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b5bfff700 1 -- 172.21.15.81:0/4152566975 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c011d40 con 0x7f1b6815d560 2024-05-08T23:31:17.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6e79d700 1 -- 172.21.15.81:0/4152566975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b6815d560 msgr2=0x7f1b680d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6e79d700 1 --2- 172.21.15.81:0/4152566975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b6815d560 0x7f1b680d1bf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009850 tx=0x7f1b5c009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:17.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6e79d700 1 -- 172.21.15.81:0/4152566975 shutdown_connections 2024-05-08T23:31:17.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6e79d700 1 --2- 172.21.15.81:0/4152566975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b6815d560 0x7f1b680d1bf0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6e79d700 1 -- 172.21.15.81:0/4152566975 >> 172.21.15.81:0/4152566975 conn(0x7f1b6808c530 msgr2=0x7f1b6809c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:17.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6e79d700 1 -- 172.21.15.81:0/4152566975 shutdown_connections 2024-05-08T23:31:17.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6e79d700 1 -- 172.21.15.81:0/4152566975 wait complete. 2024-05-08T23:31:17.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6e79d700 1 Processor -- start 2024-05-08T23:31:17.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6e79d700 1 -- start start 2024-05-08T23:31:17.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6e79d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b680d2f60 0x7f1b680caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6e79d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b6815d180 0x7f1b680cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6e79d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b6815d560 0x7f1b680cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6e79d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b680ccd40 con 0x7f1b6815d560 2024-05-08T23:31:17.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6e79d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6814dcf0 con 0x7f1b6815d180 2024-05-08T23:31:17.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.436+0000 7f1b6e79d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6815eff0 con 0x7f1b680d2f60 2024-05-08T23:31:17.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b6df9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b6815d560 0x7f1b680cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b6df9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b6815d560 0x7f1b680cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57232/0 (socket says 172.21.15.81:57232) 2024-05-08T23:31:17.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b6df9c700 1 -- 172.21.15.81:0/3275505850 learned_addr learned my addr 172.21.15.81:0/3275505850 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:17.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b6d79b700 1 --2- 172.21.15.81:0/3275505850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b680d2f60 0x7f1b680caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b6cf9a700 1 --2- 172.21.15.81:0/3275505850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b6815d180 0x7f1b680cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b6df9c700 1 -- 172.21.15.81:0/3275505850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b680d2f60 msgr2=0x7f1b680caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b6df9c700 1 --2- 172.21.15.81:0/3275505850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b680d2f60 0x7f1b680caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b6df9c700 1 -- 172.21.15.81:0/3275505850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b6815d180 msgr2=0x7f1b680cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b6df9c700 1 --2- 172.21.15.81:0/3275505850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b6815d180 0x7f1b680cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b6df9c700 1 -- 172.21.15.81:0/3275505850 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c0094b0 con 0x7f1b6815d560 2024-05-08T23:31:17.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b6df9c700 1 --2- 172.21.15.81:0/3275505850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b6815d560 0x7f1b680cc800 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000cbc0 tx=0x7f1b6000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:17.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b59ffb700 1 -- 172.21.15.81:0/3275505850 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60003c20 con 0x7f1b6815d560 2024-05-08T23:31:17.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b59ffb700 1 -- 172.21.15.81:0/3275505850 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b60003d80 con 0x7f1b6815d560 2024-05-08T23:31:17.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b6e79d700 1 -- 172.21.15.81:0/3275505850 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6815f2d0 con 0x7f1b6815d560 2024-05-08T23:31:17.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b59ffb700 1 -- 172.21.15.81:0/3275505850 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6000aae0 con 0x7f1b6815d560 2024-05-08T23:31:17.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b6e79d700 1 -- 172.21.15.81:0/3275505850 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6815f890 con 0x7f1b6815d560 2024-05-08T23:31:17.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b6e79d700 1 -- 172.21.15.81:0/3275505850 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b680672a0 con 0x7f1b6815d560 2024-05-08T23:31:17.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.440+0000 7f1b59ffb700 1 -- 172.21.15.81:0/3275505850 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1b6000ac40 con 0x7f1b6815d560 2024-05-08T23:31:17.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.444+0000 7f1b59ffb700 1 --2- 172.21.15.81:0/3275505850 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1b4c066730 0x7f1b4c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.444+0000 7f1b59ffb700 1 -- 172.21.15.81:0/3275505850 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b600840f0 con 0x7f1b6815d560 2024-05-08T23:31:17.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.444+0000 7f1b6d79b700 1 --2- 172.21.15.81:0/3275505850 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1b4c066730 0x7f1b4c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.444+0000 7f1b6d79b700 1 --2- 172.21.15.81:0/3275505850 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1b4c066730 0x7f1b4c068bf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1b64000f60 tx=0x7f1b64007e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:17.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.452+0000 7f1b59ffb700 1 -- 172.21.15.81:0/3275505850 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1b60051460 con 0x7f1b6815d560 2024-05-08T23:31:17.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.620+0000 7f1b6e79d700 1 -- 172.21.15.81:0/3275505850 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b680d0470 con 0x7f1b6815d560 2024-05-08T23:31:17.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.620+0000 7f1b59ffb700 1 -- 172.21.15.81:0/3275505850 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b60054a80 con 0x7f1b6815d560 2024-05-08T23:31:17.627 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:17.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.624+0000 7f1b6e79d700 1 -- 172.21.15.81:0/3275505850 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1b4c066730 msgr2=0x7f1b4c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.624+0000 7f1b6e79d700 1 --2- 172.21.15.81:0/3275505850 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1b4c066730 0x7f1b4c068bf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1b64000f60 tx=0x7f1b64007e00 comp rx=0 tx=0).stop 2024-05-08T23:31:17.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.624+0000 7f1b6e79d700 1 -- 172.21.15.81:0/3275505850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b6815d560 msgr2=0x7f1b680cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.624+0000 7f1b6e79d700 1 --2- 172.21.15.81:0/3275505850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b6815d560 0x7f1b680cc800 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000cbc0 tx=0x7f1b6000ced0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.624+0000 7f1b6e79d700 1 -- 172.21.15.81:0/3275505850 shutdown_connections 2024-05-08T23:31:17.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.624+0000 7f1b6e79d700 1 --2- 172.21.15.81:0/3275505850 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1b4c066730 0x7f1b4c068bf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.624+0000 7f1b6e79d700 1 --2- 172.21.15.81:0/3275505850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b6815d560 0x7f1b680cc800 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.624+0000 7f1b6e79d700 1 --2- 172.21.15.81:0/3275505850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b6815d180 0x7f1b680cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.630 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.624+0000 7f1b6e79d700 1 --2- 172.21.15.81:0/3275505850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b680d2f60 0x7f1b680caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.630 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.624+0000 7f1b6e79d700 1 -- 172.21.15.81:0/3275505850 >> 172.21.15.81:0/3275505850 conn(0x7f1b6808c530 msgr2=0x7f1b6815c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:17.630 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.624+0000 7f1b6e79d700 1 -- 172.21.15.81:0/3275505850 shutdown_connections 2024-05-08T23:31:17.630 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.624+0000 7f1b6e79d700 1 -- 172.21.15.81:0/3275505850 wait complete. 2024-05-08T23:31:17.641 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:17.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2ec247700 1 Processor -- start 2024-05-08T23:31:17.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2ec247700 1 -- start start 2024-05-08T23:31:17.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2ec247700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2e4160a30 0x7ff2e40d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2ec247700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff2e40d3c50 0x7ff2e40d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2ec247700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e40b66d0 con 0x7ff2e40d1cc0 2024-05-08T23:31:17.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2ec247700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e40d2170 con 0x7ff2e40d3c50 2024-05-08T23:31:17.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2ec247700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e40d11c0 con 0x7ff2e4160a30 2024-05-08T23:31:17.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2ea7e4700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff2e40d1cc0 0x7ff2e40cddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34444/0 (socket says 172.21.15.81:34444) 2024-05-08T23:31:17.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2ea7e4700 1 -- 172.21.15.81:0/2681985103 learned_addr learned my addr 172.21.15.81:0/2681985103 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:17.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2e97e2700 1 --2- 172.21.15.81:0/2681985103 >> v2:172.21.15.117:3300/0 conn(0x7ff2e40d3c50 0x7ff2e40d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2e9fe3700 1 --2- 172.21.15.81:0/2681985103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2e4160a30 0x7ff2e40d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2e8fe1700 1 -- 172.21.15.81:0/2681985103 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224861933 0 0) 0x7ff2e40b66d0 con 0x7ff2e40d1cc0 2024-05-08T23:31:17.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2e8fe1700 1 -- 172.21.15.81:0/2681985103 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2d00035d0 con 0x7ff2e40d1cc0 2024-05-08T23:31:17.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2e9fe3700 1 -- 172.21.15.81:0/2681985103 >> v2:172.21.15.117:3300/0 conn(0x7ff2e40d3c50 msgr2=0x7ff2e40d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:17.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2e9fe3700 1 --2- 172.21.15.81:0/2681985103 >> v2:172.21.15.117:3300/0 conn(0x7ff2e40d3c50 0x7ff2e40d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2e9fe3700 1 -- 172.21.15.81:0/2681985103 >> v1:172.21.15.81:6789/0 conn(0x7ff2e40d1cc0 legacy=0x7ff2e40cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2e9fe3700 1 -- 172.21.15.81:0/2681985103 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e40d14a0 con 0x7ff2e4160a30 2024-05-08T23:31:17.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2e97e2700 1 --2- 172.21.15.81:0/2681985103 >> v2:172.21.15.117:3300/0 conn(0x7ff2e40d3c50 0x7ff2e40d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:17.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2e9fe3700 1 --2- 172.21.15.81:0/2681985103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2e4160a30 0x7ff2e40d0480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009850 tx=0x7ff2d8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f96c059ce01cc85d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:17.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.756+0000 7ff2e8fe1700 1 -- 172.21.15.81:0/2681985103 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8011830 con 0x7ff2e4160a30 2024-05-08T23:31:17.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2e8fe1700 1 -- 172.21.15.81:0/2681985103 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d8011990 con 0x7ff2e4160a30 2024-05-08T23:31:17.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2e8fe1700 1 -- 172.21.15.81:0/2681985103 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8011d40 con 0x7ff2e4160a30 2024-05-08T23:31:17.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 -- 172.21.15.81:0/2681985103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2e4160a30 msgr2=0x7ff2e40d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 --2- 172.21.15.81:0/2681985103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2e4160a30 0x7ff2e40d0480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009850 tx=0x7ff2d8009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:17.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 -- 172.21.15.81:0/2681985103 shutdown_connections 2024-05-08T23:31:17.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 --2- 172.21.15.81:0/2681985103 >> v2:172.21.15.117:3300/0 conn(0x7ff2e40d3c50 0x7ff2e40d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 --2- 172.21.15.81:0/2681985103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2e4160a30 0x7ff2e40d0480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 -- 172.21.15.81:0/2681985103 >> 172.21.15.81:0/2681985103 conn(0x7ff2e409c500 msgr2=0x7ff2e40d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:17.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 -- 172.21.15.81:0/2681985103 shutdown_connections 2024-05-08T23:31:17.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 -- 172.21.15.81:0/2681985103 wait complete. 2024-05-08T23:31:17.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 Processor -- start 2024-05-08T23:31:17.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 -- start start 2024-05-08T23:31:17.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2e40d1cc0 0x7ff2e414f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e40d3c50 0x7ff2e40a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2e4160a30 0x7ff2e40cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e41f33c0 con 0x7ff2e4160a30 2024-05-08T23:31:17.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e41f2a20 con 0x7ff2e40d3c50 2024-05-08T23:31:17.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.760+0000 7ff2ec247700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e41f2d00 con 0x7ff2e40d1cc0 2024-05-08T23:31:17.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2ea7e4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2e4160a30 0x7ff2e40cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2ea7e4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2e4160a30 0x7ff2e40cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57248/0 (socket says 172.21.15.81:57248) 2024-05-08T23:31:17.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2ea7e4700 1 -- 172.21.15.81:0/3308229800 learned_addr learned my addr 172.21.15.81:0/3308229800 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:17.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2e97e2700 1 --2- 172.21.15.81:0/3308229800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e40d3c50 0x7ff2e40a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2e9fe3700 1 --2- 172.21.15.81:0/3308229800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2e40d1cc0 0x7ff2e414f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2ea7e4700 1 -- 172.21.15.81:0/3308229800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2e40d1cc0 msgr2=0x7ff2e414f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2ea7e4700 1 --2- 172.21.15.81:0/3308229800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2e40d1cc0 0x7ff2e414f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2ea7e4700 1 -- 172.21.15.81:0/3308229800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e40d3c50 msgr2=0x7ff2e40a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2ea7e4700 1 --2- 172.21.15.81:0/3308229800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e40d3c50 0x7ff2e40a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2ea7e4700 1 -- 172.21.15.81:0/3308229800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e00094b0 con 0x7ff2e4160a30 2024-05-08T23:31:17.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2ea7e4700 1 --2- 172.21.15.81:0/3308229800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2e4160a30 0x7ff2e40cc4d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00cb40 tx=0x7ff2dc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:17.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2d67fc700 1 -- 172.21.15.81:0/3308229800 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc007500 con 0x7ff2e4160a30 2024-05-08T23:31:17.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2d67fc700 1 -- 172.21.15.81:0/3308229800 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2dc007660 con 0x7ff2e4160a30 2024-05-08T23:31:17.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2ec247700 1 -- 172.21.15.81:0/3308229800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d80094b0 con 0x7ff2e4160a30 2024-05-08T23:31:17.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2d67fc700 1 -- 172.21.15.81:0/3308229800 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc0079f0 con 0x7ff2e4160a30 2024-05-08T23:31:17.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2ec247700 1 -- 172.21.15.81:0/3308229800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e41f2710 con 0x7ff2e4160a30 2024-05-08T23:31:17.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2d67fc700 1 -- 172.21.15.81:0/3308229800 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff2dc01b3b0 con 0x7ff2e4160a30 2024-05-08T23:31:17.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2ec247700 1 -- 172.21.15.81:0/3308229800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e41f3500 con 0x7ff2e4160a30 2024-05-08T23:31:17.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2d67fc700 1 --2- 172.21.15.81:0/3308229800 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff2c8066730 0x7ff2c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:17.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2d67fc700 1 -- 172.21.15.81:0/3308229800 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2dc083af0 con 0x7ff2e4160a30 2024-05-08T23:31:17.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.764+0000 7ff2e9fe3700 1 --2- 172.21.15.81:0/3308229800 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff2c8066730 0x7ff2c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:17.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.768+0000 7ff2e9fe3700 1 --2- 172.21.15.81:0/3308229800 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff2c8066730 0x7ff2c8068bf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80090b0 tx=0x7ff2d801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:17.781 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.776+0000 7ff2d67fc700 1 -- 172.21.15.81:0/3308229800 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff2e41f3500 con 0x7ff2e4160a30 2024-05-08T23:31:17.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.944+0000 7ff2ec247700 1 -- 172.21.15.81:0/3308229800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2e41f36e0 con 0x7ff2e4160a30 2024-05-08T23:31:17.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.944+0000 7ff2d67fc700 1 -- 172.21.15.81:0/3308229800 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff2e41f36e0 con 0x7ff2e4160a30 2024-05-08T23:31:17.951 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:17.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.948+0000 7ff2ec247700 1 -- 172.21.15.81:0/3308229800 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff2c8066730 msgr2=0x7ff2c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.948+0000 7ff2ec247700 1 --2- 172.21.15.81:0/3308229800 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff2c8066730 0x7ff2c8068bf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80090b0 tx=0x7ff2d801c040 comp rx=0 tx=0).stop 2024-05-08T23:31:17.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.948+0000 7ff2ec247700 1 -- 172.21.15.81:0/3308229800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2e4160a30 msgr2=0x7ff2e40cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:17.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.948+0000 7ff2ec247700 1 --2- 172.21.15.81:0/3308229800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2e4160a30 0x7ff2e40cc4d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00cb40 tx=0x7ff2dc00cb70 comp rx=0 tx=0).stop 2024-05-08T23:31:17.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.948+0000 7ff2ec247700 1 -- 172.21.15.81:0/3308229800 shutdown_connections 2024-05-08T23:31:17.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.948+0000 7ff2ec247700 1 --2- 172.21.15.81:0/3308229800 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff2c8066730 0x7ff2c8068bf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.948+0000 7ff2ec247700 1 --2- 172.21.15.81:0/3308229800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2e4160a30 0x7ff2e40cc4d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.948+0000 7ff2ec247700 1 --2- 172.21.15.81:0/3308229800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e40d3c50 0x7ff2e40a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.948+0000 7ff2ec247700 1 --2- 172.21.15.81:0/3308229800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2e40d1cc0 0x7ff2e414f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:17.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.948+0000 7ff2ec247700 1 -- 172.21.15.81:0/3308229800 >> 172.21.15.81:0/3308229800 conn(0x7ff2e409c500 msgr2=0x7ff2e40b6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:17.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.948+0000 7ff2ec247700 1 -- 172.21.15.81:0/3308229800 shutdown_connections 2024-05-08T23:31:17.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:17.948+0000 7ff2ec247700 1 -- 172.21.15.81:0/3308229800 wait complete. 2024-05-08T23:31:17.966 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:18.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f070295d700 1 Processor -- start 2024-05-08T23:31:18.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f070295d700 1 -- start start 2024-05-08T23:31:18.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f070295d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f06fc0ac8e0 0x7f06fc15e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f070295d700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06fc0b4970 con 0x7f06fc0ae660 2024-05-08T23:31:18.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f070295d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06fc154700 con 0x7f06fc154320 2024-05-08T23:31:18.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f070295d700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06fc15e860 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f070215c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f06fc0ac8e0 0x7f06fc15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f070215c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f06fc0ac8e0 0x7f06fc15e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57260/0 (socket says 172.21.15.81:57260) 2024-05-08T23:31:18.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f070215c700 1 -- 172.21.15.81:0/4034869664 learned_addr learned my addr 172.21.15.81:0/4034869664 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:18.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f070215c700 1 -- 172.21.15.81:0/4034869664 >> v1:172.21.15.117:6789/0 conn(0x7f06fc154320 legacy=0x7f06fc0adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f070215c700 1 -- 172.21.15.81:0/4034869664 >> v1:172.21.15.81:6789/0 conn(0x7f06fc0ae660 legacy=0x7f06fc06dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f070215c700 1 -- 172.21.15.81:0/4034869664 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc0acdc0 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f070215c700 1 --2- 172.21.15.81:0/4034869664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06fc0ac8e0 0x7f06fc15e320 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009e50 tx=0x7f06f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37198ea80163f061 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:18.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f0700959700 1 -- 172.21.15.81:0/4034869664 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f400e070 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f0700959700 1 -- 172.21.15.81:0/4034869664 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f400b9c0 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.076+0000 7f0700959700 1 -- 172.21.15.81:0/4034869664 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f400bd70 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.080+0000 7f070295d700 1 -- 172.21.15.81:0/4034869664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06fc0ac8e0 msgr2=0x7f06fc15e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.080+0000 7f070295d700 1 --2- 172.21.15.81:0/4034869664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06fc0ac8e0 0x7f06fc15e320 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009e50 tx=0x7f06f400b330 comp rx=0 tx=0).stop 2024-05-08T23:31:18.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.080+0000 7f070295d700 1 -- 172.21.15.81:0/4034869664 shutdown_connections 2024-05-08T23:31:18.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.080+0000 7f070295d700 1 --2- 172.21.15.81:0/4034869664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06fc0ac8e0 0x7f06fc15e320 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.080+0000 7f070295d700 1 -- 172.21.15.81:0/4034869664 >> 172.21.15.81:0/4034869664 conn(0x7f06fc0717b0 msgr2=0x7f06fc06a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:18.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.080+0000 7f070295d700 1 -- 172.21.15.81:0/4034869664 shutdown_connections 2024-05-08T23:31:18.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.080+0000 7f070295d700 1 -- 172.21.15.81:0/4034869664 wait complete. 2024-05-08T23:31:18.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070295d700 1 Processor -- start 2024-05-08T23:31:18.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070295d700 1 -- start start 2024-05-08T23:31:18.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070295d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06fc0ac8e0 0x7f06fc1f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070295d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06fc0ae660 0x7f06fc155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070295d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06fc154320 0x7f06fc160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070295d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc1f5d40 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070295d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc155e70 con 0x7f06fc154320 2024-05-08T23:31:18.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070295d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc1614e0 con 0x7f06fc0ae660 2024-05-08T23:31:18.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070195b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06fc0ac8e0 0x7f06fc1f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070195b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06fc0ac8e0 0x7f06fc1f5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57268/0 (socket says 172.21.15.81:57268) 2024-05-08T23:31:18.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070195b700 1 -- 172.21.15.81:0/365130023 learned_addr learned my addr 172.21.15.81:0/365130023 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:18.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070215c700 1 --2- 172.21.15.81:0/365130023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06fc154320 0x7f06fc160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070115a700 1 --2- 172.21.15.81:0/365130023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06fc0ae660 0x7f06fc155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070195b700 1 -- 172.21.15.81:0/365130023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06fc0ae660 msgr2=0x7f06fc155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070195b700 1 --2- 172.21.15.81:0/365130023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06fc0ae660 0x7f06fc155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070195b700 1 -- 172.21.15.81:0/365130023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06fc154320 msgr2=0x7f06fc160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070195b700 1 --2- 172.21.15.81:0/365130023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06fc154320 0x7f06fc160cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070195b700 1 -- 172.21.15.81:0/365130023 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f4009b30 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070215c700 1 --2- 172.21.15.81:0/365130023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06fc154320 0x7f06fc160cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:18.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070195b700 1 --2- 172.21.15.81:0/365130023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06fc0ac8e0 0x7f06fc1f5800 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f06f800c500 tx=0x7f06f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:18.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f06f27fc700 1 -- 172.21.15.81:0/365130023 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f80052d0 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f06f27fc700 1 -- 172.21.15.81:0/365130023 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f8005430 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070295d700 1 -- 172.21.15.81:0/365130023 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06fc1617c0 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f06f27fc700 1 -- 172.21.15.81:0/365130023 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f80055a0 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.084+0000 7f070295d700 1 -- 172.21.15.81:0/365130023 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06fc15ecb0 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.088+0000 7f070295d700 1 -- 172.21.15.81:0/365130023 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06fc0672a0 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.088+0000 7f06f27fc700 1 -- 172.21.15.81:0/365130023 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f06f80038e0 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.088+0000 7f06f27fc700 1 --2- 172.21.15.81:0/365130023 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f06e0066730 0x7f06e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.088+0000 7f06f27fc700 1 -- 172.21.15.81:0/365130023 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06f8083660 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.088+0000 7f070115a700 1 --2- 172.21.15.81:0/365130023 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f06e0066730 0x7f06e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.088+0000 7f070115a700 1 --2- 172.21.15.81:0/365130023 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f06e0066730 0x7f06e0068bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f06fc06e430 tx=0x7f06ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:18.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.096+0000 7f06f27fc700 1 -- 172.21.15.81:0/365130023 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f06f8050930 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.260+0000 7f070295d700 1 -- 172.21.15.81:0/365130023 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f06fc161200 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.260+0000 7f06f27fc700 1 -- 172.21.15.81:0/365130023 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f06f8053f50 con 0x7f06fc0ac8e0 2024-05-08T23:31:18.265 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:18.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.260+0000 7f070295d700 1 -- 172.21.15.81:0/365130023 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f06e0066730 msgr2=0x7f06e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.260+0000 7f070295d700 1 --2- 172.21.15.81:0/365130023 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f06e0066730 0x7f06e0068bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f06fc06e430 tx=0x7f06ec009040 comp rx=0 tx=0).stop 2024-05-08T23:31:18.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.260+0000 7f070295d700 1 -- 172.21.15.81:0/365130023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06fc0ac8e0 msgr2=0x7f06fc1f5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.260+0000 7f070295d700 1 --2- 172.21.15.81:0/365130023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06fc0ac8e0 0x7f06fc1f5800 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f06f800c500 tx=0x7f06f800c8c0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.260+0000 7f070295d700 1 -- 172.21.15.81:0/365130023 shutdown_connections 2024-05-08T23:31:18.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.260+0000 7f070295d700 1 --2- 172.21.15.81:0/365130023 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f06e0066730 0x7f06e0068bf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.260+0000 7f070295d700 1 --2- 172.21.15.81:0/365130023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06fc154320 0x7f06fc160cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.260+0000 7f070295d700 1 --2- 172.21.15.81:0/365130023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06fc0ae660 0x7f06fc155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.260+0000 7f070295d700 1 --2- 172.21.15.81:0/365130023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06fc0ac8e0 0x7f06fc1f5800 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.264+0000 7f070295d700 1 -- 172.21.15.81:0/365130023 >> 172.21.15.81:0/365130023 conn(0x7f06fc0717b0 msgr2=0x7f06fc06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:18.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.264+0000 7f070295d700 1 -- 172.21.15.81:0/365130023 shutdown_connections 2024-05-08T23:31:18.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.264+0000 7f070295d700 1 -- 172.21.15.81:0/365130023 wait complete. 2024-05-08T23:31:18.280 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:31:18.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.392+0000 7fb95c96e700 1 Processor -- start 2024-05-08T23:31:18.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.392+0000 7fb95c96e700 1 -- start start 2024-05-08T23:31:18.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb95c96e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb9580cf280 0x7fb95815de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb95c96e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9580b4330 0x7fb9580b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb95c96e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb958071160 0x7fb9580715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb95c96e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9580b37e0 con 0x7fb9580cf280 2024-05-08T23:31:18.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb95c96e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb958071b30 con 0x7fb958071160 2024-05-08T23:31:18.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb95c96e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580b4790 con 0x7fb9580b4330 2024-05-08T23:31:18.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb95659c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb9580cf280 0x7fb95815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb95659c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb9580cf280 0x7fb95815de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57284/0 (socket says 172.21.15.81:57284) 2024-05-08T23:31:18.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb95659c700 1 -- 172.21.15.81:0/798771584 learned_addr learned my addr 172.21.15.81:0/798771584 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:18.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb94dd9b700 1 --2- 172.21.15.81:0/798771584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9580b4330 0x7fb9580b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb956d9d700 1 --2- 172.21.15.81:0/798771584 >> v2:172.21.15.117:3300/0 conn(0x7fb958071160 0x7fb9580715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb95659c700 1 -- 172.21.15.81:0/798771584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9580b4330 msgr2=0x7fb9580b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:18.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb95659c700 1 --2- 172.21.15.81:0/798771584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9580b4330 0x7fb9580b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb95659c700 1 -- 172.21.15.81:0/798771584 >> v2:172.21.15.117:3300/0 conn(0x7fb958071160 msgr2=0x7fb9580715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:18.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb95659c700 1 --2- 172.21.15.81:0/798771584 >> v2:172.21.15.117:3300/0 conn(0x7fb958071160 0x7fb9580715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb95659c700 1 -- 172.21.15.81:0/798771584 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9580b49f0 con 0x7fb9580cf280 2024-05-08T23:31:18.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb95659c700 1 --2- 172.21.15.81:0/798771584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9580cf280 0x7fb95815de50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb9580cd560 tx=0x7fb94800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4335cdc175d8c9f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:18.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb955d9b700 1 -- 172.21.15.81:0/798771584 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94800dbf0 con 0x7fb9580cf280 2024-05-08T23:31:18.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb955d9b700 1 -- 172.21.15.81:0/798771584 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94800dd50 con 0x7fb9580cf280 2024-05-08T23:31:18.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.396+0000 7fb955d9b700 1 -- 172.21.15.81:0/798771584 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948004250 con 0x7fb9580cf280 2024-05-08T23:31:18.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 -- 172.21.15.81:0/798771584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9580cf280 msgr2=0x7fb95815de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 --2- 172.21.15.81:0/798771584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9580cf280 0x7fb95815de50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb9580cd560 tx=0x7fb94800bea0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 -- 172.21.15.81:0/798771584 shutdown_connections 2024-05-08T23:31:18.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 --2- 172.21.15.81:0/798771584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9580b4330 0x7fb9580b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 --2- 172.21.15.81:0/798771584 >> v2:172.21.15.117:3300/0 conn(0x7fb958071160 0x7fb9580715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 --2- 172.21.15.81:0/798771584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9580cf280 0x7fb95815de50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 -- 172.21.15.81:0/798771584 >> 172.21.15.81:0/798771584 conn(0x7fb9580b1bb0 msgr2=0x7fb958070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:18.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 -- 172.21.15.81:0/798771584 shutdown_connections 2024-05-08T23:31:18.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 -- 172.21.15.81:0/798771584 wait complete. 2024-05-08T23:31:18.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 Processor -- start 2024-05-08T23:31:18.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 -- start start 2024-05-08T23:31:18.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb958071160 0x7fb95814e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9580b4330 0x7fb9580adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9580cf280 0x7fb9580af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95814e680 con 0x7fb9580b4330 2024-05-08T23:31:18.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580ae460 con 0x7fb9580cf280 2024-05-08T23:31:18.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95c96e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95815aeb0 con 0x7fb958071160 2024-05-08T23:31:18.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb94dd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9580b4330 0x7fb9580adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb94dd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9580b4330 0x7fb9580adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57300/0 (socket says 172.21.15.81:57300) 2024-05-08T23:31:18.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb94dd9b700 1 -- 172.21.15.81:0/3339176263 learned_addr learned my addr 172.21.15.81:0/3339176263 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:18.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb956d9d700 1 --2- 172.21.15.81:0/3339176263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9580cf280 0x7fb9580af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb95659c700 1 --2- 172.21.15.81:0/3339176263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb958071160 0x7fb95814e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb94dd9b700 1 -- 172.21.15.81:0/3339176263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb958071160 msgr2=0x7fb95814e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb94dd9b700 1 --2- 172.21.15.81:0/3339176263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb958071160 0x7fb95814e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb94dd9b700 1 -- 172.21.15.81:0/3339176263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9580cf280 msgr2=0x7fb9580af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb94dd9b700 1 --2- 172.21.15.81:0/3339176263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9580cf280 0x7fb9580af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.400+0000 7fb94dd9b700 1 -- 172.21.15.81:0/3339176263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94800b820 con 0x7fb9580b4330 2024-05-08T23:31:18.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.404+0000 7fb94dd9b700 1 --2- 172.21.15.81:0/3339176263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9580b4330 0x7fb9580adf20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb94000e9d0 tx=0x7fb94000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:18.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.404+0000 7fb94f7fe700 1 -- 172.21.15.81:0/3339176263 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940009720 con 0x7fb9580b4330 2024-05-08T23:31:18.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.404+0000 7fb94f7fe700 1 -- 172.21.15.81:0/3339176263 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94000ee70 con 0x7fb9580b4330 2024-05-08T23:31:18.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.404+0000 7fb95c96e700 1 -- 172.21.15.81:0/3339176263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95815b190 con 0x7fb9580b4330 2024-05-08T23:31:18.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.404+0000 7fb94f7fe700 1 -- 172.21.15.81:0/3339176263 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94000ca50 con 0x7fb9580b4330 2024-05-08T23:31:18.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.404+0000 7fb95c96e700 1 -- 172.21.15.81:0/3339176263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95815b750 con 0x7fb9580b4330 2024-05-08T23:31:18.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.404+0000 7fb95c96e700 1 -- 172.21.15.81:0/3339176263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9580672a0 con 0x7fb9580b4330 2024-05-08T23:31:18.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.404+0000 7fb94f7fe700 1 -- 172.21.15.81:0/3339176263 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb94000cbb0 con 0x7fb9580b4330 2024-05-08T23:31:18.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.404+0000 7fb94f7fe700 1 --2- 172.21.15.81:0/3339176263 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb938066730 0x7fb938068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.404+0000 7fb94f7fe700 1 -- 172.21.15.81:0/3339176263 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb940084140 con 0x7fb9580b4330 2024-05-08T23:31:18.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.404+0000 7fb95659c700 1 --2- 172.21.15.81:0/3339176263 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb938066730 0x7fb938068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.404+0000 7fb95659c700 1 --2- 172.21.15.81:0/3339176263 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb938066730 0x7fb938068bf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb948019040 tx=0x7fb948014000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:18.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.412+0000 7fb94f7fe700 1 -- 172.21.15.81:0/3339176263 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb940051430 con 0x7fb9580b4330 2024-05-08T23:31:18.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.592+0000 7fb95c96e700 1 -- 172.21.15.81:0/3339176263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb95806dd50 con 0x7fb9580b4330 2024-05-08T23:31:18.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.596+0000 7fb94f7fe700 1 -- 172.21.15.81:0/3339176263 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb94000ce60 con 0x7fb9580b4330 2024-05-08T23:31:18.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.600+0000 7fb95c96e700 1 -- 172.21.15.81:0/3339176263 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb938066730 msgr2=0x7fb938068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.600+0000 7fb95c96e700 1 --2- 172.21.15.81:0/3339176263 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb938066730 0x7fb938068bf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb948019040 tx=0x7fb948014000 comp rx=0 tx=0).stop 2024-05-08T23:31:18.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.600+0000 7fb95c96e700 1 -- 172.21.15.81:0/3339176263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9580b4330 msgr2=0x7fb9580adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.600+0000 7fb95c96e700 1 --2- 172.21.15.81:0/3339176263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9580b4330 0x7fb9580adf20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb94000e9d0 tx=0x7fb94000ece0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.600+0000 7fb95c96e700 1 -- 172.21.15.81:0/3339176263 shutdown_connections 2024-05-08T23:31:18.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.600+0000 7fb95c96e700 1 --2- 172.21.15.81:0/3339176263 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb938066730 0x7fb938068bf0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.600+0000 7fb95c96e700 1 --2- 172.21.15.81:0/3339176263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9580cf280 0x7fb9580af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.600+0000 7fb95c96e700 1 --2- 172.21.15.81:0/3339176263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9580b4330 0x7fb9580adf20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.600+0000 7fb95c96e700 1 --2- 172.21.15.81:0/3339176263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb958071160 0x7fb95814e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.600+0000 7fb95c96e700 1 -- 172.21.15.81:0/3339176263 >> 172.21.15.81:0/3339176263 conn(0x7fb9580b1bb0 msgr2=0x7fb9580cd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:18.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.600+0000 7fb95c96e700 1 -- 172.21.15.81:0/3339176263 shutdown_connections 2024-05-08T23:31:18.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.600+0000 7fb95c96e700 1 -- 172.21.15.81:0/3339176263 wait complete. 2024-05-08T23:31:18.607 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:31:18.622 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:31:18.622 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:31:18.628 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:18.741 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda9079700 1 Processor -- start 2024-05-08T23:31:18.741 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda9079700 1 -- start start 2024-05-08T23:31:18.742 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda9079700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd9c0e33a0 0x7fbd9c058950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.743 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda9079700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9c0578b0 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.743 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda9079700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9c057b90 con 0x7fbd9c007280 2024-05-08T23:31:18.743 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda9079700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0576c0 con 0x7fbd9c0e33a0 2024-05-08T23:31:18.743 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda37fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fbd9c03ebd0 0x7fbd9c056fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34464/0 (socket says 172.21.15.81:34464) 2024-05-08T23:31:18.743 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda37fe700 1 -- 172.21.15.81:0/249480463 learned_addr learned my addr 172.21.15.81:0/249480463 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:18.743 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda3fff700 1 --2- 172.21.15.81:0/249480463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd9c0e33a0 0x7fbd9c058950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.743 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda2ffd700 1 -- 172.21.15.81:0/249480463 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2779811831 0 0) 0x7fbd9c0578b0 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.743 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda2ffd700 1 -- 172.21.15.81:0/249480463 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd8c0035d0 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.743 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda2ffd700 1 -- 172.21.15.81:0/249480463 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404388110 0 0) 0x7fbd9c057b90 con 0x7fbd9c007280 2024-05-08T23:31:18.743 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda2ffd700 1 -- 172.21.15.81:0/249480463 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd9c0578b0 con 0x7fbd9c007280 2024-05-08T23:31:18.743 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda3fff700 1 -- 172.21.15.81:0/249480463 >> v1:172.21.15.117:6789/0 conn(0x7fbd9c007280 legacy=0x7fbd9c03c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda2ffd700 1 -- 172.21.15.81:0/249480463 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2272036522 0 0) 0x7fbd8c0035d0 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda3fff700 1 -- 172.21.15.81:0/249480463 >> v1:172.21.15.81:6789/0 conn(0x7fbd9c03ebd0 legacy=0x7fbd9c056fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda3fff700 1 -- 172.21.15.81:0/249480463 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c057e70 con 0x7fbd9c0e33a0 2024-05-08T23:31:18.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda3fff700 1 --2- 172.21.15.81:0/249480463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd9c0e33a0 0x7fbd9c058950 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009880 tx=0x7fbd98009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a49ca97fc01069d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:18.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda2ffd700 1 -- 172.21.15.81:0/249480463 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94001670 con 0x7fbd9c0e33a0 2024-05-08T23:31:18.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda2ffd700 1 -- 172.21.15.81:0/249480463 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd98004030 con 0x7fbd9c0e33a0 2024-05-08T23:31:18.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.736+0000 7fbda2ffd700 1 -- 172.21.15.81:0/249480463 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd980043a0 con 0x7fbd9c0e33a0 2024-05-08T23:31:18.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda9079700 1 -- 172.21.15.81:0/249480463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd9c0e33a0 msgr2=0x7fbd9c058950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda9079700 1 --2- 172.21.15.81:0/249480463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd9c0e33a0 0x7fbd9c058950 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009880 tx=0x7fbd98009b90 comp rx=0 tx=0).stop 2024-05-08T23:31:18.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda9079700 1 -- 172.21.15.81:0/249480463 shutdown_connections 2024-05-08T23:31:18.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda9079700 1 --2- 172.21.15.81:0/249480463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd9c0e33a0 0x7fbd9c058950 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda9079700 1 -- 172.21.15.81:0/249480463 >> 172.21.15.81:0/249480463 conn(0x7fbd9c055a10 msgr2=0x7fbd9c055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:18.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda9079700 1 -- 172.21.15.81:0/249480463 shutdown_connections 2024-05-08T23:31:18.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda9079700 1 -- 172.21.15.81:0/249480463 wait complete. 2024-05-08T23:31:18.748 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda9079700 1 Processor -- start 2024-05-08T23:31:18.748 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda9079700 1 -- start start 2024-05-08T23:31:18.748 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda9079700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd9c007280 0x7fbd9c052b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.748 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda9079700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd9c03ebd0 0x7fbd9c0d7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.748 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda9079700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd9c0e33a0 0x7fbd9c03a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda9079700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c052020 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda9079700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0d8510 con 0x7fbd9c007280 2024-05-08T23:31:18.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda9079700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c051d40 con 0x7fbd9c0e33a0 2024-05-08T23:31:18.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd9c03ebd0 0x7fbd9c0d7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda8878700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd9c0e33a0 0x7fbd9c03a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd9c03ebd0 0x7fbd9c0d7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57314/0 (socket says 172.21.15.81:57314) 2024-05-08T23:31:18.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda37fe700 1 -- 172.21.15.81:0/1855888614 learned_addr learned my addr 172.21.15.81:0/1855888614 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:18.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda3fff700 1 --2- 172.21.15.81:0/1855888614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd9c007280 0x7fbd9c052b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda37fe700 1 -- 172.21.15.81:0/1855888614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd9c0e33a0 msgr2=0x7fbd9c03a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda37fe700 1 --2- 172.21.15.81:0/1855888614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd9c0e33a0 0x7fbd9c03a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda37fe700 1 -- 172.21.15.81:0/1855888614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd9c007280 msgr2=0x7fbd9c052b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda37fe700 1 --2- 172.21.15.81:0/1855888614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd9c007280 0x7fbd9c052b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.740+0000 7fbda37fe700 1 -- 172.21.15.81:0/1855888614 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd980094b0 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.744+0000 7fbda37fe700 1 --2- 172.21.15.81:0/1855888614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd9c03ebd0 0x7fbd9c0d7fd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400c780 tx=0x7fbd9400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:18.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.744+0000 7fbda0ff9700 1 -- 172.21.15.81:0/1855888614 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94004ca0 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.744+0000 7fbda0ff9700 1 -- 172.21.15.81:0/1855888614 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd94004e20 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.744+0000 7fbda9079700 1 -- 172.21.15.81:0/1855888614 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9c039530 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.744+0000 7fbda0ff9700 1 -- 172.21.15.81:0/1855888614 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94007310 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.744+0000 7fbda9079700 1 -- 172.21.15.81:0/1855888614 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9c039af0 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.744+0000 7fbda9079700 1 -- 172.21.15.81:0/1855888614 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9c039ce0 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.748+0000 7fbda0ff9700 1 -- 172.21.15.81:0/1855888614 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbd94007470 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.748+0000 7fbda0ff9700 1 --2- 172.21.15.81:0/1855888614 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbd90066730 0x7fbd90068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:18.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.748+0000 7fbda0ff9700 1 -- 172.21.15.81:0/1855888614 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd94083e90 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.748+0000 7fbda3fff700 1 --2- 172.21.15.81:0/1855888614 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbd90066730 0x7fbd90068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:18.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.748+0000 7fbda3fff700 1 --2- 172.21.15.81:0/1855888614 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbd90066730 0x7fbd90068bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbd980090b0 tx=0x7fbd98010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:18.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.756+0000 7fbda0ff9700 1 -- 172.21.15.81:0/1855888614 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbd94051170 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.928+0000 7fbda9079700 1 -- 172.21.15.81:0/1855888614 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd9c005440 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.932+0000 7fbda0ff9700 1 -- 172.21.15.81:0/1855888614 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fbd94007720 con 0x7fbd9c03ebd0 2024-05-08T23:31:18.938 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:18.939 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:30:39.667988+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:18.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.936+0000 7fbda9079700 1 -- 172.21.15.81:0/1855888614 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbd90066730 msgr2=0x7fbd90068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.936+0000 7fbda9079700 1 --2- 172.21.15.81:0/1855888614 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbd90066730 0x7fbd90068bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbd980090b0 tx=0x7fbd98010000 comp rx=0 tx=0).stop 2024-05-08T23:31:18.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.936+0000 7fbda9079700 1 -- 172.21.15.81:0/1855888614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd9c03ebd0 msgr2=0x7fbd9c0d7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:18.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.936+0000 7fbda9079700 1 --2- 172.21.15.81:0/1855888614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd9c03ebd0 0x7fbd9c0d7fd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400c780 tx=0x7fbd9400ca90 comp rx=0 tx=0).stop 2024-05-08T23:31:18.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.936+0000 7fbda9079700 1 -- 172.21.15.81:0/1855888614 shutdown_connections 2024-05-08T23:31:18.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.936+0000 7fbda9079700 1 --2- 172.21.15.81:0/1855888614 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbd90066730 0x7fbd90068bf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.936+0000 7fbda9079700 1 --2- 172.21.15.81:0/1855888614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd9c0e33a0 0x7fbd9c03a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.936+0000 7fbda9079700 1 --2- 172.21.15.81:0/1855888614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd9c03ebd0 0x7fbd9c0d7fd0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.936+0000 7fbda9079700 1 --2- 172.21.15.81:0/1855888614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd9c007280 0x7fbd9c052b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:18.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.936+0000 7fbda9079700 1 -- 172.21.15.81:0/1855888614 >> 172.21.15.81:0/1855888614 conn(0x7fbd9c055a10 msgr2=0x7fbd9c04cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:18.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.936+0000 7fbda9079700 1 -- 172.21.15.81:0/1855888614 shutdown_connections 2024-05-08T23:31:18.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:18.936+0000 7fbda9079700 1 -- 172.21.15.81:0/1855888614 wait complete. 2024-05-08T23:31:18.953 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:18.954 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:19.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.072+0000 7f40a72d4700 1 Processor -- start 2024-05-08T23:31:19.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.072+0000 7f40a72d4700 1 -- start start 2024-05-08T23:31:19.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.072+0000 7f40a72d4700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f40a006a360 0x7f40a00af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.072+0000 7f40a72d4700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a006eb90 con 0x7f40a00d07b0 2024-05-08T23:31:19.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.072+0000 7f40a72d4700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a006ede0 con 0x7f40a0071150 2024-05-08T23:31:19.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.072+0000 7f40a72d4700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a006f0c0 con 0x7f40a006a360 2024-05-08T23:31:19.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.072+0000 7f40a62d2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f40a006a360 0x7f40a00af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.072+0000 7f40a5ad1700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f40a00d07b0 0x7f40a006a740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34470/0 (socket says 172.21.15.81:34470) 2024-05-08T23:31:19.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.072+0000 7f40a5ad1700 1 -- 172.21.15.81:0/481141283 learned_addr learned my addr 172.21.15.81:0/481141283 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:19.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.072+0000 7f40a52d0700 1 -- 172.21.15.81:0/481141283 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396089691 0 0) 0x7f40a006eb90 con 0x7f40a00d07b0 2024-05-08T23:31:19.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.072+0000 7f40a52d0700 1 -- 172.21.15.81:0/481141283 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40880035d0 con 0x7f40a00d07b0 2024-05-08T23:31:19.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a62d2700 1 -- 172.21.15.81:0/481141283 >> v1:172.21.15.117:6789/0 conn(0x7f40a0071150 legacy=0x7f40a00b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a62d2700 1 -- 172.21.15.81:0/481141283 >> v1:172.21.15.81:6789/0 conn(0x7f40a00d07b0 legacy=0x7f40a006a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a52d0700 1 -- 172.21.15.81:0/481141283 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3933236944 0 0) 0x7f40880035d0 con 0x7f40a00d07b0 2024-05-08T23:31:19.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a62d2700 1 -- 172.21.15.81:0/481141283 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a00b2e70 con 0x7f40a006a360 2024-05-08T23:31:19.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a62d2700 1 --2- 172.21.15.81:0/481141283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a006a360 0x7f40a00af290 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4090009850 tx=0x7f4090009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea587521019bd08 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:19.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a52d0700 1 -- 172.21.15.81:0/481141283 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c0019f0 con 0x7f40a006a360 2024-05-08T23:31:19.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a52d0700 1 -- 172.21.15.81:0/481141283 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4090004030 con 0x7f40a006a360 2024-05-08T23:31:19.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a52d0700 1 -- 172.21.15.81:0/481141283 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40900043c0 con 0x7f40a006a360 2024-05-08T23:31:19.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a72d4700 1 -- 172.21.15.81:0/481141283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a006a360 msgr2=0x7f40a00af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a72d4700 1 --2- 172.21.15.81:0/481141283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a006a360 0x7f40a00af290 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4090009850 tx=0x7f4090009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:19.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a72d4700 1 -- 172.21.15.81:0/481141283 shutdown_connections 2024-05-08T23:31:19.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a72d4700 1 --2- 172.21.15.81:0/481141283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a006a360 0x7f40a00af290 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a72d4700 1 -- 172.21.15.81:0/481141283 >> 172.21.15.81:0/481141283 conn(0x7f40a00cfcc0 msgr2=0x7f40a00d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:19.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a72d4700 1 -- 172.21.15.81:0/481141283 shutdown_connections 2024-05-08T23:31:19.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a72d4700 1 -- 172.21.15.81:0/481141283 wait complete. 2024-05-08T23:31:19.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a72d4700 1 Processor -- start 2024-05-08T23:31:19.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.076+0000 7f40a72d4700 1 -- start start 2024-05-08T23:31:19.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a72d4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a006a360 0x7f40a01f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a72d4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a0071150 0x7f40a00cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a72d4700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40a00d07b0 0x7f40a00ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a72d4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a01570f0 con 0x7f40a006a360 2024-05-08T23:31:19.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a72d4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00cf420 con 0x7f40a0071150 2024-05-08T23:31:19.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a72d4700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00ccdb0 con 0x7f40a00d07b0 2024-05-08T23:31:19.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a62d2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a006a360 0x7f40a01f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a62d2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a006a360 0x7f40a01f2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57328/0 (socket says 172.21.15.81:57328) 2024-05-08T23:31:19.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a62d2700 1 -- 172.21.15.81:0/1827268335 learned_addr learned my addr 172.21.15.81:0/1827268335 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:19.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a5ad1700 1 --2- 172.21.15.81:0/1827268335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a0071150 0x7f40a00cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a6ad3700 1 --2- 172.21.15.81:0/1827268335 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40a00d07b0 0x7f40a00ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a62d2700 1 -- 172.21.15.81:0/1827268335 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40a00d07b0 msgr2=0x7f40a00ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a62d2700 1 --2- 172.21.15.81:0/1827268335 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40a00d07b0 0x7f40a00ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a62d2700 1 -- 172.21.15.81:0/1827268335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a0071150 msgr2=0x7f40a00cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a62d2700 1 --2- 172.21.15.81:0/1827268335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a0071150 0x7f40a00cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a62d2700 1 -- 172.21.15.81:0/1827268335 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40900094b0 con 0x7f40a006a360 2024-05-08T23:31:19.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a62d2700 1 --2- 172.21.15.81:0/1827268335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a006a360 0x7f40a01f2950 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f40a0066ad0 tx=0x7f4090004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:19.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f4096ffd700 1 -- 172.21.15.81:0/1827268335 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090003e50 con 0x7f40a006a360 2024-05-08T23:31:19.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f4096ffd700 1 -- 172.21.15.81:0/1827268335 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409001f070 con 0x7f40a006a360 2024-05-08T23:31:19.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a72d4700 1 -- 172.21.15.81:0/1827268335 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a00cd090 con 0x7f40a006a360 2024-05-08T23:31:19.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f4096ffd700 1 -- 172.21.15.81:0/1827268335 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090011870 con 0x7f40a006a360 2024-05-08T23:31:19.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a72d4700 1 -- 172.21.15.81:0/1827268335 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a00cd650 con 0x7f40a006a360 2024-05-08T23:31:19.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f40a72d4700 1 -- 172.21.15.81:0/1827268335 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a00cf590 con 0x7f40a006a360 2024-05-08T23:31:19.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.080+0000 7f4096ffd700 1 -- 172.21.15.81:0/1827268335 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4090021030 con 0x7f40a006a360 2024-05-08T23:31:19.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.084+0000 7f4096ffd700 1 --2- 172.21.15.81:0/1827268335 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f408c066730 0x7f408c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.084+0000 7f40a5ad1700 1 --2- 172.21.15.81:0/1827268335 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f408c066730 0x7f408c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.084+0000 7f4096ffd700 1 -- 172.21.15.81:0/1827268335 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4090085740 con 0x7f40a006a360 2024-05-08T23:31:19.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.084+0000 7f40a5ad1700 1 --2- 172.21.15.81:0/1827268335 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f408c066730 0x7f408c068bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f409c002230 tx=0x7f409c002000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:19.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.088+0000 7f4096ffd700 1 -- 172.21.15.81:0/1827268335 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4090052b40 con 0x7f40a006a360 2024-05-08T23:31:19.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.256+0000 7f40a72d4700 1 -- 172.21.15.81:0/1827268335 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40a00672a0 con 0x7f40a006a360 2024-05-08T23:31:19.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.256+0000 7f4096ffd700 1 -- 172.21.15.81:0/1827268335 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4090056160 con 0x7f40a006a360 2024-05-08T23:31:19.262 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:19.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.260+0000 7f40a72d4700 1 -- 172.21.15.81:0/1827268335 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f408c066730 msgr2=0x7f408c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.260+0000 7f40a72d4700 1 --2- 172.21.15.81:0/1827268335 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f408c066730 0x7f408c068bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f409c002230 tx=0x7f409c002000 comp rx=0 tx=0).stop 2024-05-08T23:31:19.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.260+0000 7f40a72d4700 1 -- 172.21.15.81:0/1827268335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a006a360 msgr2=0x7f40a01f2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.260+0000 7f40a72d4700 1 --2- 172.21.15.81:0/1827268335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a006a360 0x7f40a01f2950 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f40a0066ad0 tx=0x7f4090004770 comp rx=0 tx=0).stop 2024-05-08T23:31:19.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.260+0000 7f40a72d4700 1 -- 172.21.15.81:0/1827268335 shutdown_connections 2024-05-08T23:31:19.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.260+0000 7f40a72d4700 1 --2- 172.21.15.81:0/1827268335 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f408c066730 0x7f408c068bf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.260+0000 7f40a72d4700 1 --2- 172.21.15.81:0/1827268335 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40a00d07b0 0x7f40a00ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.260+0000 7f40a72d4700 1 --2- 172.21.15.81:0/1827268335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a0071150 0x7f40a00cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.260+0000 7f40a72d4700 1 --2- 172.21.15.81:0/1827268335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a006a360 0x7f40a01f2950 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.260+0000 7f40a72d4700 1 -- 172.21.15.81:0/1827268335 >> 172.21.15.81:0/1827268335 conn(0x7f40a00cfcc0 msgr2=0x7f40a015ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:19.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.260+0000 7f40a72d4700 1 -- 172.21.15.81:0/1827268335 shutdown_connections 2024-05-08T23:31:19.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.260+0000 7f40a72d4700 1 -- 172.21.15.81:0/1827268335 wait complete. 2024-05-08T23:31:19.277 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:19.392 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.384+0000 7ff3469d8700 1 Processor -- start 2024-05-08T23:31:19.392 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.384+0000 7ff3469d8700 1 -- start start 2024-05-08T23:31:19.393 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3469d8700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff340071c60 0x7ff340072040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.393 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3469d8700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff34015d3f0 0x7ff3400acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.393 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3469d8700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3400ae180 con 0x7ff3400acb00 2024-05-08T23:31:19.393 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3469d8700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3400ad570 con 0x7ff340071c60 2024-05-08T23:31:19.393 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3469d8700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3400ad850 con 0x7ff34015d3f0 2024-05-08T23:31:19.393 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3451d5700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff3400acb00 0x7ff34015cce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34472/0 (socket says 172.21.15.81:34472) 2024-05-08T23:31:19.393 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3451d5700 1 -- 172.21.15.81:0/786167207 learned_addr learned my addr 172.21.15.81:0/786167207 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:19.393 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3461d7700 1 --2- 172.21.15.81:0/786167207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff34015d3f0 0x7ff3400acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3459d6700 1 --2- 172.21.15.81:0/786167207 >> v2:172.21.15.117:3300/0 conn(0x7ff340071c60 0x7ff340072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3449d4700 1 -- 172.21.15.81:0/786167207 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375199055 0 0) 0x7ff3400ae180 con 0x7ff3400acb00 2024-05-08T23:31:19.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3449d4700 1 -- 172.21.15.81:0/786167207 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff32c0035d0 con 0x7ff3400acb00 2024-05-08T23:31:19.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3449d4700 1 -- 172.21.15.81:0/786167207 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 5703539 0 0) 0x7ff32c0035d0 con 0x7ff3400acb00 2024-05-08T23:31:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3461d7700 1 -- 172.21.15.81:0/786167207 >> v2:172.21.15.117:3300/0 conn(0x7ff340071c60 msgr2=0x7ff340072040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3461d7700 1 --2- 172.21.15.81:0/786167207 >> v2:172.21.15.117:3300/0 conn(0x7ff340071c60 0x7ff340072040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3461d7700 1 -- 172.21.15.81:0/786167207 >> v1:172.21.15.81:6789/0 conn(0x7ff3400acb00 legacy=0x7ff34015cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3461d7700 1 -- 172.21.15.81:0/786167207 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3400adb30 con 0x7ff34015d3f0 2024-05-08T23:31:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3461d7700 1 --2- 172.21.15.81:0/786167207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff34015d3f0 0x7ff3400acf80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff33800c2e0 tx=0x7ff33800c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=830cfd34875560a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3449d4700 1 -- 172.21.15.81:0/786167207 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33800f070 con 0x7ff34015d3f0 2024-05-08T23:31:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3449d4700 1 -- 172.21.15.81:0/786167207 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33800ce00 con 0x7ff34015d3f0 2024-05-08T23:31:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3449d4700 1 -- 172.21.15.81:0/786167207 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c001670 con 0x7ff34015d3f0 2024-05-08T23:31:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3469d8700 1 -- 172.21.15.81:0/786167207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff34015d3f0 msgr2=0x7ff3400acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3469d8700 1 --2- 172.21.15.81:0/786167207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff34015d3f0 0x7ff3400acf80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff33800c2e0 tx=0x7ff33800c5f0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3469d8700 1 -- 172.21.15.81:0/786167207 shutdown_connections 2024-05-08T23:31:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3469d8700 1 --2- 172.21.15.81:0/786167207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff34015d3f0 0x7ff3400acf80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3469d8700 1 --2- 172.21.15.81:0/786167207 >> v2:172.21.15.117:3300/0 conn(0x7ff340071c60 0x7ff340072040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3469d8700 1 -- 172.21.15.81:0/786167207 >> 172.21.15.81:0/786167207 conn(0x7ff3400b0de0 msgr2=0x7ff3400b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:19.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3469d8700 1 -- 172.21.15.81:0/786167207 shutdown_connections 2024-05-08T23:31:19.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.388+0000 7ff3469d8700 1 -- 172.21.15.81:0/786167207 wait complete. 2024-05-08T23:31:19.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3469d8700 1 Processor -- start 2024-05-08T23:31:19.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3469d8700 1 -- start start 2024-05-08T23:31:19.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3469d8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff340071c60 0x7ff3401588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3469d8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3400acb00 0x7ff340142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3469d8700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff34015d3f0 0x7ff340153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3469d8700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff340158e30 con 0x7ff340071c60 2024-05-08T23:31:19.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3469d8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff340159c70 con 0x7ff3400acb00 2024-05-08T23:31:19.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3469d8700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff340142c80 con 0x7ff34015d3f0 2024-05-08T23:31:19.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3459d6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff340071c60 0x7ff3401588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3459d6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff340071c60 0x7ff3401588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57334/0 (socket says 172.21.15.81:57334) 2024-05-08T23:31:19.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3459d6700 1 -- 172.21.15.81:0/135899876 learned_addr learned my addr 172.21.15.81:0/135899876 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:19.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3461d7700 1 --2- 172.21.15.81:0/135899876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff34015d3f0 0x7ff340153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3451d5700 1 --2- 172.21.15.81:0/135899876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3400acb00 0x7ff340142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3459d6700 1 -- 172.21.15.81:0/135899876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff34015d3f0 msgr2=0x7ff340153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3459d6700 1 --2- 172.21.15.81:0/135899876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff34015d3f0 0x7ff340153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3459d6700 1 -- 172.21.15.81:0/135899876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3400acb00 msgr2=0x7ff340142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3459d6700 1 --2- 172.21.15.81:0/135899876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3400acb00 0x7ff340142740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.392+0000 7ff3459d6700 1 -- 172.21.15.81:0/135899876 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33800c050 con 0x7ff340071c60 2024-05-08T23:31:19.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.396+0000 7ff3459d6700 1 --2- 172.21.15.81:0/135899876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff340071c60 0x7ff3401588f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff33000e9d0 tx=0x7ff33000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:19.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.396+0000 7ff3367fc700 1 -- 172.21.15.81:0/135899876 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330009720 con 0x7ff340071c60 2024-05-08T23:31:19.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.396+0000 7ff3367fc700 1 -- 172.21.15.81:0/135899876 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33000ee70 con 0x7ff340071c60 2024-05-08T23:31:19.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.396+0000 7ff3469d8700 1 -- 172.21.15.81:0/135899876 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff340159060 con 0x7ff340071c60 2024-05-08T23:31:19.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.396+0000 7ff3367fc700 1 -- 172.21.15.81:0/135899876 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33000ca50 con 0x7ff340071c60 2024-05-08T23:31:19.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.396+0000 7ff3469d8700 1 -- 172.21.15.81:0/135899876 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff340159620 con 0x7ff340071c60 2024-05-08T23:31:19.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.396+0000 7ff3367fc700 1 -- 172.21.15.81:0/135899876 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff33000cbb0 con 0x7ff340071c60 2024-05-08T23:31:19.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.396+0000 7ff3469d8700 1 -- 172.21.15.81:0/135899876 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3400abe00 con 0x7ff340071c60 2024-05-08T23:31:19.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.396+0000 7ff3367fc700 1 --2- 172.21.15.81:0/135899876 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff324066760 0x7ff324068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.396+0000 7ff3451d5700 1 --2- 172.21.15.81:0/135899876 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff324066760 0x7ff324068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.396+0000 7ff3367fc700 1 -- 172.21.15.81:0/135899876 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3300841a0 con 0x7ff340071c60 2024-05-08T23:31:19.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.396+0000 7ff3451d5700 1 --2- 172.21.15.81:0/135899876 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff324066760 0x7ff324068c20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff3400ad290 tx=0x7ff33c001de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:19.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.404+0000 7ff3367fc700 1 -- 172.21.15.81:0/135899876 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff330051480 con 0x7ff340071c60 2024-05-08T23:31:19.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.576+0000 7ff3469d8700 1 -- 172.21.15.81:0/135899876 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff3400684b0 con 0x7ff340071c60 2024-05-08T23:31:19.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.576+0000 7ff3367fc700 1 -- 172.21.15.81:0/135899876 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff33000ce60 con 0x7ff340071c60 2024-05-08T23:31:19.581 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:19.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.580+0000 7ff3469d8700 1 -- 172.21.15.81:0/135899876 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff324066760 msgr2=0x7ff324068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.580+0000 7ff3469d8700 1 --2- 172.21.15.81:0/135899876 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff324066760 0x7ff324068c20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff3400ad290 tx=0x7ff33c001de0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.580+0000 7ff3469d8700 1 -- 172.21.15.81:0/135899876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff340071c60 msgr2=0x7ff3401588f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.580+0000 7ff3469d8700 1 --2- 172.21.15.81:0/135899876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff340071c60 0x7ff3401588f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff33000e9d0 tx=0x7ff33000ece0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.580+0000 7ff3469d8700 1 -- 172.21.15.81:0/135899876 shutdown_connections 2024-05-08T23:31:19.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.580+0000 7ff3469d8700 1 --2- 172.21.15.81:0/135899876 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff324066760 0x7ff324068c20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.580+0000 7ff3469d8700 1 --2- 172.21.15.81:0/135899876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff34015d3f0 0x7ff340153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.580+0000 7ff3469d8700 1 --2- 172.21.15.81:0/135899876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3400acb00 0x7ff340142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.580+0000 7ff3469d8700 1 --2- 172.21.15.81:0/135899876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff340071c60 0x7ff3401588f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.580+0000 7ff3469d8700 1 -- 172.21.15.81:0/135899876 >> 172.21.15.81:0/135899876 conn(0x7ff3400b0de0 msgr2=0x7ff340151dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:19.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.580+0000 7ff3469d8700 1 -- 172.21.15.81:0/135899876 shutdown_connections 2024-05-08T23:31:19.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.580+0000 7ff3469d8700 1 -- 172.21.15.81:0/135899876 wait complete. 2024-05-08T23:31:19.601 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:19.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f907759e700 1 Processor -- start 2024-05-08T23:31:19.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f907759e700 1 -- start start 2024-05-08T23:31:19.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f907759e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9078160920 0x7f90780ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f907759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90780d1da0 0x7f90780cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f907759e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90780d3d40 con 0x7f90780cec30 2024-05-08T23:31:19.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f907759e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90780d0d00 con 0x7f9078160920 2024-05-08T23:31:19.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f907759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90780d0fe0 con 0x7f90780d1da0 2024-05-08T23:31:19.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f907659c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9078160920 0x7f90780ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f9076d9d700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f90780cec30 0x7f9078161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34484/0 (socket says 172.21.15.81:34484) 2024-05-08T23:31:19.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f907659c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9078160920 0x7f90780ce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57342/0 (socket says 172.21.15.81:57342) 2024-05-08T23:31:19.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f9076d9d700 1 -- 172.21.15.81:0/629590150 learned_addr learned my addr 172.21.15.81:0/629590150 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:19.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f906dd9b700 1 --2- 172.21.15.81:0/629590150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90780d1da0 0x7f90780cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f907659c700 1 -- 172.21.15.81:0/629590150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90780d1da0 msgr2=0x7f90780cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:19.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f907659c700 1 --2- 172.21.15.81:0/629590150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90780d1da0 0x7f90780cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f907659c700 1 -- 172.21.15.81:0/629590150 >> v1:172.21.15.81:6789/0 conn(0x7f90780cec30 legacy=0x7f9078161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f9075d9b700 1 -- 172.21.15.81:0/629590150 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499956386 0 0) 0x7f90780d3d40 con 0x7f90780cec30 2024-05-08T23:31:19.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f907659c700 1 -- 172.21.15.81:0/629590150 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90780d12c0 con 0x7f9078160920 2024-05-08T23:31:19.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f906dd9b700 1 --2- 172.21.15.81:0/629590150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90780d1da0 0x7f90780cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:19.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f907659c700 1 --2- 172.21.15.81:0/629590150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9078160920 0x7f90780ce6f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9068009850 tx=0x7f9068009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ab0e62cbd136af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:19.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f9075d9b700 1 -- 172.21.15.81:0/629590150 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068011830 con 0x7f9078160920 2024-05-08T23:31:19.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f9075d9b700 1 -- 172.21.15.81:0/629590150 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9068011990 con 0x7f9078160920 2024-05-08T23:31:19.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.712+0000 7f9075d9b700 1 -- 172.21.15.81:0/629590150 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068011d20 con 0x7f9078160920 2024-05-08T23:31:19.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.716+0000 7f907759e700 1 -- 172.21.15.81:0/629590150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9078160920 msgr2=0x7f90780ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.716+0000 7f907759e700 1 --2- 172.21.15.81:0/629590150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9078160920 0x7f90780ce6f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9068009850 tx=0x7f9068009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:19.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.716+0000 7f907759e700 1 -- 172.21.15.81:0/629590150 shutdown_connections 2024-05-08T23:31:19.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.716+0000 7f907759e700 1 --2- 172.21.15.81:0/629590150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90780d1da0 0x7f90780cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.716+0000 7f907759e700 1 --2- 172.21.15.81:0/629590150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9078160920 0x7f90780ce6f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.716+0000 7f907759e700 1 -- 172.21.15.81:0/629590150 >> 172.21.15.81:0/629590150 conn(0x7f907809c340 msgr2=0x7f90780d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:19.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.716+0000 7f907759e700 1 -- 172.21.15.81:0/629590150 shutdown_connections 2024-05-08T23:31:19.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.716+0000 7f907759e700 1 -- 172.21.15.81:0/629590150 wait complete. 2024-05-08T23:31:19.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.716+0000 7f907759e700 1 Processor -- start 2024-05-08T23:31:19.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.716+0000 7f907759e700 1 -- start start 2024-05-08T23:31:19.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f907759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90780cec30 0x7f90780a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f907759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90780d1da0 0x7f90781f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f907759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9078160920 0x7f90781f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f907759e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90781f2ba0 con 0x7f90780d1da0 2024-05-08T23:31:19.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f907759e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90781f7190 con 0x7f90780cec30 2024-05-08T23:31:19.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f907759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907809f6e0 con 0x7f9078160920 2024-05-08T23:31:19.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f906dd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90780d1da0 0x7f90781f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f906dd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90780d1da0 0x7f90781f2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57350/0 (socket says 172.21.15.81:57350) 2024-05-08T23:31:19.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f906dd9b700 1 -- 172.21.15.81:0/2928185632 learned_addr learned my addr 172.21.15.81:0/2928185632 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:19.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f9076d9d700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9078160920 0x7f90781f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f907659c700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90780cec30 0x7f90780a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f906dd9b700 1 -- 172.21.15.81:0/2928185632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9078160920 msgr2=0x7f90781f6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f906dd9b700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9078160920 0x7f90781f6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f906dd9b700 1 -- 172.21.15.81:0/2928185632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90780cec30 msgr2=0x7f90780a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f906dd9b700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90780cec30 0x7f90780a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f906dd9b700 1 -- 172.21.15.81:0/2928185632 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90680094b0 con 0x7f90780d1da0 2024-05-08T23:31:19.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f9076d9d700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9078160920 0x7f90781f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:19.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f907659c700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90780cec30 0x7f90780a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:19.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f906dd9b700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90780d1da0 0x7f90781f2660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f906000e810 tx=0x7f906000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:19.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f906f7fe700 1 -- 172.21.15.81:0/2928185632 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9060013610 con 0x7f90780d1da0 2024-05-08T23:31:19.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f906f7fe700 1 -- 172.21.15.81:0/2928185632 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9060013770 con 0x7f90780d1da0 2024-05-08T23:31:19.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f907759e700 1 -- 172.21.15.81:0/2928185632 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9078160e80 con 0x7f90780d1da0 2024-05-08T23:31:19.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f906f7fe700 1 -- 172.21.15.81:0/2928185632 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9060013b00 con 0x7f90780d1da0 2024-05-08T23:31:19.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f907759e700 1 -- 172.21.15.81:0/2928185632 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907809fca0 con 0x7f90780d1da0 2024-05-08T23:31:19.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f907759e700 1 -- 172.21.15.81:0/2928185632 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90780684b0 con 0x7f90780d1da0 2024-05-08T23:31:19.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.720+0000 7f906f7fe700 1 -- 172.21.15.81:0/2928185632 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9060013c60 con 0x7f90780d1da0 2024-05-08T23:31:19.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.724+0000 7f906f7fe700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9058066730 0x7f9058068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:19.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.724+0000 7f906f7fe700 1 -- 172.21.15.81:0/2928185632 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9060083d40 con 0x7f90780d1da0 2024-05-08T23:31:19.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.724+0000 7f907659c700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9058066730 0x7f9058068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:19.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.724+0000 7f907659c700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9058066730 0x7f9058068bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f90680090b0 tx=0x7f9068008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:19.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.728+0000 7f906f7fe700 1 -- 172.21.15.81:0/2928185632 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f90600510c0 con 0x7f90780d1da0 2024-05-08T23:31:19.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.900+0000 7f907759e700 1 -- 172.21.15.81:0/2928185632 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f90780cf510 con 0x7f90780d1da0 2024-05-08T23:31:19.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.900+0000 7f906f7fe700 1 -- 172.21.15.81:0/2928185632 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f90600546e0 con 0x7f90780d1da0 2024-05-08T23:31:19.907 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:19.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.904+0000 7f907759e700 1 -- 172.21.15.81:0/2928185632 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9058066730 msgr2=0x7f9058068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.904+0000 7f907759e700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9058066730 0x7f9058068bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f90680090b0 tx=0x7f9068008030 comp rx=0 tx=0).stop 2024-05-08T23:31:19.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.904+0000 7f907759e700 1 -- 172.21.15.81:0/2928185632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90780d1da0 msgr2=0x7f90781f2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:19.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.904+0000 7f907759e700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90780d1da0 0x7f90781f2660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f906000e810 tx=0x7f906000ebd0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.904+0000 7f907759e700 1 -- 172.21.15.81:0/2928185632 shutdown_connections 2024-05-08T23:31:19.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.904+0000 7f907759e700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9058066730 0x7f9058068bf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.904+0000 7f907759e700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9078160920 0x7f90781f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.904+0000 7f907759e700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90780d1da0 0x7f90781f2660 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.904+0000 7f907759e700 1 --2- 172.21.15.81:0/2928185632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90780cec30 0x7f90780a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:19.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.904+0000 7f907759e700 1 -- 172.21.15.81:0/2928185632 >> 172.21.15.81:0/2928185632 conn(0x7f907809c340 msgr2=0x7f907806afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:19.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.904+0000 7f907759e700 1 -- 172.21.15.81:0/2928185632 shutdown_connections 2024-05-08T23:31:19.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:19.904+0000 7f907759e700 1 -- 172.21.15.81:0/2928185632 wait complete. 2024-05-08T23:31:19.922 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:31:20.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.028+0000 7f523fe5c700 1 Processor -- start 2024-05-08T23:31:20.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.028+0000 7f523fe5c700 1 -- start start 2024-05-08T23:31:20.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523fe5c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52380cb3b0 0x7f52381478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523fe5c700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52381656a0 con 0x7f5238144d40 2024-05-08T23:31:20.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523fe5c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5238165970 con 0x7f5238165240 2024-05-08T23:31:20.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523fe5c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5238161c50 con 0x7f52380cb3b0 2024-05-08T23:31:20.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523e3f9700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5238144d40 0x7f523806c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34496/0 (socket says 172.21.15.81:34496) 2024-05-08T23:31:20.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523e3f9700 1 -- 172.21.15.81:0/2132709658 learned_addr learned my addr 172.21.15.81:0/2132709658 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:20.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523dbf8700 1 --2- 172.21.15.81:0/2132709658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52380cb3b0 0x7f52381478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 -- 172.21.15.81:0/2132709658 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55536291 0 0) 0x7f52381656a0 con 0x7f5238144d40 2024-05-08T23:31:20.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 -- 172.21.15.81:0/2132709658 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52240035d0 con 0x7f5238144d40 2024-05-08T23:31:20.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 -- 172.21.15.81:0/2132709658 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1339862726 0 0) 0x7f52240035d0 con 0x7f5238144d40 2024-05-08T23:31:20.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 -- 172.21.15.81:0/2132709658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52380cb3b0 msgr2=0x7f52381478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:20.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 --2- 172.21.15.81:0/2132709658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52380cb3b0 0x7f52381478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 -- 172.21.15.81:0/2132709658 >> v1:172.21.15.117:6789/0 conn(0x7f5238165240 legacy=0x7f5238162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 -- 172.21.15.81:0/2132709658 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5238165c50 con 0x7f5238144d40 2024-05-08T23:31:20.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 -- 172.21.15.81:0/2132709658 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f5234001670 con 0x7f5238144d40 2024-05-08T23:31:20.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 -- 172.21.15.81:0/2132709658 >> v1:172.21.15.81:6789/0 conn(0x7f5238144d40 legacy=0x7f523806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 --2- 172.21.15.81:0/2132709658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5224003a50 0x7f5224003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 --2- 172.21.15.81:0/2132709658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5224004400 0x7f52240048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 --2- 172.21.15.81:0/2132709658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52380cb3b0 0x7f5224005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 -- 172.21.15.81:0/2132709658 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522400ba30 con 0x7f5224004400 2024-05-08T23:31:20.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 -- 172.21.15.81:0/2132709658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522400bb70 con 0x7f5224003a50 2024-05-08T23:31:20.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 -- 172.21.15.81:0/2132709658 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522400bd30 con 0x7f52380cb3b0 2024-05-08T23:31:20.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523d3f7700 1 --2- 172.21.15.81:0/2132709658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5224004400 0x7f52240048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523dbf8700 1 --2- 172.21.15.81:0/2132709658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5224003a50 0x7f5224003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523e3f9700 1 --2- 172.21.15.81:0/2132709658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52380cb3b0 0x7f5224005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523d3f7700 1 -- 172.21.15.81:0/2132709658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52380cb3b0 msgr2=0x7f5224005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:20.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523d3f7700 1 --2- 172.21.15.81:0/2132709658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52380cb3b0 0x7f5224005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523d3f7700 1 -- 172.21.15.81:0/2132709658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5224003a50 msgr2=0x7f5224003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:20.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523d3f7700 1 --2- 172.21.15.81:0/2132709658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5224003a50 0x7f5224003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523d3f7700 1 -- 172.21.15.81:0/2132709658 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240035d0 con 0x7f5224004400 2024-05-08T23:31:20.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523d3f7700 1 --2- 172.21.15.81:0/2132709658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5224004400 0x7f52240048d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f52280019a0 tx=0x7f52240058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5d87eb63989670 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:20.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 -- 172.21.15.81:0/2132709658 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228001440 con 0x7f5224004400 2024-05-08T23:31:20.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 -- 172.21.15.81:0/2132709658 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5228006aa0 con 0x7f5224004400 2024-05-08T23:31:20.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523cbf6700 1 -- 172.21.15.81:0/2132709658 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228006e50 con 0x7f5224004400 2024-05-08T23:31:20.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523fe5c700 1 -- 172.21.15.81:0/2132709658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5224004400 msgr2=0x7f52240048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523fe5c700 1 --2- 172.21.15.81:0/2132709658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5224004400 0x7f52240048d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f52280019a0 tx=0x7f52240058a0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523e3f9700 1 -- 172.21.15.81:0/2132709658 reap_dead start 2024-05-08T23:31:20.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523fe5c700 1 -- 172.21.15.81:0/2132709658 shutdown_connections 2024-05-08T23:31:20.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523fe5c700 1 -- 172.21.15.81:0/2132709658 >> 172.21.15.81:0/2132709658 conn(0x7f523806a360 msgr2=0x7f523806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:20.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523fe5c700 1 -- 172.21.15.81:0/2132709658 shutdown_connections 2024-05-08T23:31:20.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.032+0000 7f523fe5c700 1 -- 172.21.15.81:0/2132709658 wait complete. 2024-05-08T23:31:20.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523fe5c700 1 Processor -- start 2024-05-08T23:31:20.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523fe5c700 1 -- start start 2024-05-08T23:31:20.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523fe5c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5238149f30 0x7f5238159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523fe5c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f523815e0f0 0x7f523815e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523fe5c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f523815eab0 0x7f5238135ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523fe5c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523815a480 con 0x7f523815eab0 2024-05-08T23:31:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523fe5c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523815a760 con 0x7f5238149f30 2024-05-08T23:31:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523fe5c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523815aa40 con 0x7f523815e0f0 2024-05-08T23:31:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523e3f9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f523815eab0 0x7f5238135ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523e3f9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f523815eab0 0x7f5238135ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57364/0 (socket says 172.21.15.81:57364) 2024-05-08T23:31:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523e3f9700 1 -- 172.21.15.81:0/3657324883 learned_addr learned my addr 172.21.15.81:0/3657324883 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523d3f7700 1 --2- 172.21.15.81:0/3657324883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f523815e0f0 0x7f523815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523dbf8700 1 --2- 172.21.15.81:0/3657324883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5238149f30 0x7f5238159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523d3f7700 1 -- 172.21.15.81:0/3657324883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5238149f30 msgr2=0x7f5238159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523d3f7700 1 --2- 172.21.15.81:0/3657324883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5238149f30 0x7f5238159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523d3f7700 1 -- 172.21.15.81:0/3657324883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f523815eab0 msgr2=0x7f5238135ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523d3f7700 1 --2- 172.21.15.81:0/3657324883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f523815eab0 0x7f5238135ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523d3f7700 1 -- 172.21.15.81:0/3657324883 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523815ad20 con 0x7f523815e0f0 2024-05-08T23:31:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523e3f9700 1 --2- 172.21.15.81:0/3657324883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f523815eab0 0x7f5238135ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523d3f7700 1 --2- 172.21.15.81:0/3657324883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f523815e0f0 0x7f523815e570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f52280019a0 tx=0x7f52280012b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:20.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f522e7fc700 1 -- 172.21.15.81:0/3657324883 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52280054d0 con 0x7f523815e0f0 2024-05-08T23:31:20.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f522e7fc700 1 -- 172.21.15.81:0/3657324883 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5228007030 con 0x7f523815e0f0 2024-05-08T23:31:20.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f522e7fc700 1 -- 172.21.15.81:0/3657324883 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52280073e0 con 0x7f523815e0f0 2024-05-08T23:31:20.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523fe5c700 1 -- 172.21.15.81:0/3657324883 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523815b000 con 0x7f523815e0f0 2024-05-08T23:31:20.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.036+0000 7f523fe5c700 1 -- 172.21.15.81:0/3657324883 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523815b520 con 0x7f523815e0f0 2024-05-08T23:31:20.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.040+0000 7f523fe5c700 1 -- 172.21.15.81:0/3657324883 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523815b710 con 0x7f523815e0f0 2024-05-08T23:31:20.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.040+0000 7f522e7fc700 1 -- 172.21.15.81:0/3657324883 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5228007540 con 0x7f523815e0f0 2024-05-08T23:31:20.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.040+0000 7f522e7fc700 1 --2- 172.21.15.81:0/3657324883 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f521c066760 0x7f521c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.040+0000 7f522e7fc700 1 -- 172.21.15.81:0/3657324883 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5228084e50 con 0x7f523815e0f0 2024-05-08T23:31:20.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.040+0000 7f523dbf8700 1 --2- 172.21.15.81:0/3657324883 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f521c066760 0x7f521c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.040+0000 7f523dbf8700 1 --2- 172.21.15.81:0/3657324883 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f521c066760 0x7f521c068c20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5230014450 tx=0x7f523000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:20.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.048+0000 7f522e7fc700 1 -- 172.21.15.81:0/3657324883 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5228052150 con 0x7f523815e0f0 2024-05-08T23:31:20.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.236+0000 7f523fe5c700 1 -- 172.21.15.81:0/3657324883 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f52380684b0 con 0x7f523815e0f0 2024-05-08T23:31:20.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.236+0000 7f522e7fc700 1 -- 172.21.15.81:0/3657324883 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5228055770 con 0x7f523815e0f0 2024-05-08T23:31:20.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.240+0000 7f523fe5c700 1 -- 172.21.15.81:0/3657324883 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f521c066760 msgr2=0x7f521c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.240+0000 7f523fe5c700 1 --2- 172.21.15.81:0/3657324883 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f521c066760 0x7f521c068c20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5230014450 tx=0x7f523000b040 comp rx=0 tx=0).stop 2024-05-08T23:31:20.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.240+0000 7f523fe5c700 1 -- 172.21.15.81:0/3657324883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f523815e0f0 msgr2=0x7f523815e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.240+0000 7f523fe5c700 1 --2- 172.21.15.81:0/3657324883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f523815e0f0 0x7f523815e570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f52280019a0 tx=0x7f52280012b0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.240+0000 7f523fe5c700 1 -- 172.21.15.81:0/3657324883 shutdown_connections 2024-05-08T23:31:20.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.240+0000 7f523fe5c700 1 --2- 172.21.15.81:0/3657324883 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f521c066760 0x7f521c068c20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.240+0000 7f523fe5c700 1 --2- 172.21.15.81:0/3657324883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f523815eab0 0x7f5238135ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.240+0000 7f523fe5c700 1 --2- 172.21.15.81:0/3657324883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f523815e0f0 0x7f523815e570 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.240+0000 7f523fe5c700 1 --2- 172.21.15.81:0/3657324883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5238149f30 0x7f5238159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.247 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.240+0000 7f523fe5c700 1 -- 172.21.15.81:0/3657324883 >> 172.21.15.81:0/3657324883 conn(0x7f523806a360 msgr2=0x7f523806aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:20.247 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.240+0000 7f523fe5c700 1 -- 172.21.15.81:0/3657324883 shutdown_connections 2024-05-08T23:31:20.247 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.240+0000 7f523fe5c700 1 -- 172.21.15.81:0/3657324883 wait complete. 2024-05-08T23:31:20.247 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:31:20.258 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2024-05-08T23:31:20.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:20.259 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-05-08T23:31:20.269 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:31:20.270 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:31:20.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:20.270 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-08T23:31:20.318 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:31:20.318 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:31:20.326 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:31:20.327 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:31:20.377 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:20.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.492+0000 7f56ba2e9700 1 Processor -- start 2024-05-08T23:31:20.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.492+0000 7f56ba2e9700 1 -- start start 2024-05-08T23:31:20.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56ba2e9700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f56b4153e40 0x7f56b40b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56ba2e9700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b415d1b0 con 0x7f56b406ba80 2024-05-08T23:31:20.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56ba2e9700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b40b4880 con 0x7f56b40b3e40 2024-05-08T23:31:20.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56ba2e9700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b40ccf50 con 0x7f56b4153e40 2024-05-08T23:31:20.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56b9ae8700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f56b406ba80 0x7f56b40cd750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34512/0 (socket says 172.21.15.81:34512) 2024-05-08T23:31:20.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56b9ae8700 1 -- 172.21.15.81:0/2749552227 learned_addr learned my addr 172.21.15.81:0/2749552227 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:20.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56b92e7700 1 --2- 172.21.15.81:0/2749552227 >> v2:172.21.15.117:3300/0 conn(0x7f56b4153e40 0x7f56b40b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334037714 0 0) 0x7f56b415d1b0 con 0x7f56b406ba80 2024-05-08T23:31:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a00034f0 con 0x7f56b406ba80 2024-05-08T23:31:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137336119 0 0) 0x7f56b40b4880 con 0x7f56b40b3e40 2024-05-08T23:31:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56b415d1b0 con 0x7f56b40b3e40 2024-05-08T23:31:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 996272009 0 0) 0x7f56a00034f0 con 0x7f56b406ba80 2024-05-08T23:31:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 >> v2:172.21.15.117:3300/0 conn(0x7f56b4153e40 msgr2=0x7f56b40b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 --2- 172.21.15.81:0/2749552227 >> v2:172.21.15.117:3300/0 conn(0x7f56b4153e40 0x7f56b40b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 >> v1:172.21.15.117:6789/0 conn(0x7f56b40b3e40 legacy=0x7f56b406b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b415d450 con 0x7f56b406ba80 2024-05-08T23:31:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f56ac0019f0 con 0x7f56b406ba80 2024-05-08T23:31:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 >> v1:172.21.15.81:6789/0 conn(0x7f56b406ba80 legacy=0x7f56b40cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 --2- 172.21.15.81:0/2749552227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a0003ca0 0x7f56a00040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 --2- 172.21.15.81:0/2749552227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a0008a10 0x7f56a000aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 --2- 172.21.15.81:0/2749552227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56a000b510 0x7f56a000d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a000e280 con 0x7f56a000b510 2024-05-08T23:31:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a000e3f0 con 0x7f56a0003ca0 2024-05-08T23:31:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a000e5b0 con 0x7f56a0008a10 2024-05-08T23:31:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56b92e7700 1 --2- 172.21.15.81:0/2749552227 >> v2:172.21.15.117:3300/0 conn(0x7f56b4153e40 0x7f56b40b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56b9ae8700 1 --2- 172.21.15.81:0/2749552227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56a000b510 0x7f56a000d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56b8ae6700 1 --2- 172.21.15.81:0/2749552227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a0008a10 0x7f56a000aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.496+0000 7f56b92e7700 1 --2- 172.21.15.81:0/2749552227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a0003ca0 0x7f56a00040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56b9ae8700 1 -- 172.21.15.81:0/2749552227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a0008a10 msgr2=0x7f56a000aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56b9ae8700 1 --2- 172.21.15.81:0/2749552227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a0008a10 0x7f56a000aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56b9ae8700 1 -- 172.21.15.81:0/2749552227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a0003ca0 msgr2=0x7f56a00040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56b9ae8700 1 --2- 172.21.15.81:0/2749552227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a0003ca0 0x7f56a00040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56b9ae8700 1 -- 172.21.15.81:0/2749552227 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a00034f0 con 0x7f56a000b510 2024-05-08T23:31:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56b92e7700 1 --2- 172.21.15.81:0/2749552227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a0003ca0 0x7f56a00040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56b8ae6700 1 --2- 172.21.15.81:0/2749552227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a0008a10 0x7f56a000aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56b9ae8700 1 -- 172.21.15.81:0/2749552227 reap_dead start 2024-05-08T23:31:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56b9ae8700 1 --2- 172.21.15.81:0/2749552227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56a000b510 0x7f56a000d9d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f56b41577b0 tx=0x7f56a00037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b69aa75fec81311 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac001e70 con 0x7f56a000b510 2024-05-08T23:31:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac0070d0 con 0x7f56a000b510 2024-05-08T23:31:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56abfff700 1 -- 172.21.15.81:0/2749552227 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac007480 con 0x7f56a000b510 2024-05-08T23:31:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56ba2e9700 1 -- 172.21.15.81:0/2749552227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56a000b510 msgr2=0x7f56a000d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56ba2e9700 1 --2- 172.21.15.81:0/2749552227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56a000b510 0x7f56a000d9d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f56b41577b0 tx=0x7f56a00037a0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56ba2e9700 1 -- 172.21.15.81:0/2749552227 shutdown_connections 2024-05-08T23:31:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56ba2e9700 1 --2- 172.21.15.81:0/2749552227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56a000b510 0x7f56a000d9d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56ba2e9700 1 -- 172.21.15.81:0/2749552227 >> 172.21.15.81:0/2749552227 conn(0x7f56b4070960 msgr2=0x7f56b4070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56ba2e9700 1 -- 172.21.15.81:0/2749552227 shutdown_connections 2024-05-08T23:31:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.500+0000 7f56ba2e9700 1 -- 172.21.15.81:0/2749552227 wait complete. 2024-05-08T23:31:20.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56ba2e9700 1 Processor -- start 2024-05-08T23:31:20.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56ba2e9700 1 -- start start 2024-05-08T23:31:20.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56ba2e9700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a000b510 0x7f56b414faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56ba2e9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56b414cb10 0x7f56b4138760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56ba2e9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b413b050 0x7f56b413b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56ba2e9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4151690 con 0x7f56b414cb10 2024-05-08T23:31:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56ba2e9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4150aa0 con 0x7f56b413b050 2024-05-08T23:31:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56ba2e9700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b413ba10 con 0x7f56a000b510 2024-05-08T23:31:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56b8ae6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56b414cb10 0x7f56b4138760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56b8ae6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56b414cb10 0x7f56b4138760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57394/0 (socket says 172.21.15.81:57394) 2024-05-08T23:31:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56b8ae6700 1 -- 172.21.15.81:0/1375519708 learned_addr learned my addr 172.21.15.81:0/1375519708 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56b9ae8700 1 --2- 172.21.15.81:0/1375519708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b413b050 0x7f56b413b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56b92e7700 1 --2- 172.21.15.81:0/1375519708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a000b510 0x7f56b414faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56b8ae6700 1 -- 172.21.15.81:0/1375519708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a000b510 msgr2=0x7f56b414faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56b8ae6700 1 --2- 172.21.15.81:0/1375519708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a000b510 0x7f56b414faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56b8ae6700 1 -- 172.21.15.81:0/1375519708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b413b050 msgr2=0x7f56b413b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56b8ae6700 1 --2- 172.21.15.81:0/1375519708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b413b050 0x7f56b413b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56b8ae6700 1 -- 172.21.15.81:0/1375519708 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b4150cf0 con 0x7f56b414cb10 2024-05-08T23:31:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56b8ae6700 1 --2- 172.21.15.81:0/1375519708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56b414cb10 0x7f56b4138760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f56a400e570 tx=0x7f56a400e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56a9ffb700 1 -- 172.21.15.81:0/1375519708 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a40052e0 con 0x7f56b414cb10 2024-05-08T23:31:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56a9ffb700 1 -- 172.21.15.81:0/1375519708 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a4005440 con 0x7f56b414cb10 2024-05-08T23:31:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56a9ffb700 1 -- 172.21.15.81:0/1375519708 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a40038e0 con 0x7f56b414cb10 2024-05-08T23:31:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56ba2e9700 1 -- 172.21.15.81:0/1375519708 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b4150fd0 con 0x7f56b414cb10 2024-05-08T23:31:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.504+0000 7f56ba2e9700 1 -- 172.21.15.81:0/1375519708 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b4138ca0 con 0x7f56b414cb10 2024-05-08T23:31:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.508+0000 7f56ba2e9700 1 -- 172.21.15.81:0/1375519708 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b40672a0 con 0x7f56b414cb10 2024-05-08T23:31:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.508+0000 7f56a9ffb700 1 -- 172.21.15.81:0/1375519708 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f56a4003a40 con 0x7f56b414cb10 2024-05-08T23:31:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.508+0000 7f56a9ffb700 1 --2- 172.21.15.81:0/1375519708 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5698066760 0x7f5698068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.508+0000 7f56a9ffb700 1 -- 172.21.15.81:0/1375519708 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f56a4083920 con 0x7f56b414cb10 2024-05-08T23:31:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.508+0000 7f56b92e7700 1 --2- 172.21.15.81:0/1375519708 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5698066760 0x7f5698068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.508+0000 7f56b92e7700 1 --2- 172.21.15.81:0/1375519708 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5698066760 0x7f5698068c20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f56b00045f0 tx=0x7f56b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:20.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.516+0000 7f56a9ffb700 1 -- 172.21.15.81:0/1375519708 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f56a4050cb0 con 0x7f56b414cb10 2024-05-08T23:31:20.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.688+0000 7f56ba2e9700 1 -- 172.21.15.81:0/1375519708 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56b4071550 con 0x7f56b414cb10 2024-05-08T23:31:20.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.688+0000 7f56a9ffb700 1 -- 172.21.15.81:0/1375519708 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f56a4019020 con 0x7f56b414cb10 2024-05-08T23:31:20.697 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:20.697 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:19.844177+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:20.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.692+0000 7f56ba2e9700 1 -- 172.21.15.81:0/1375519708 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5698066760 msgr2=0x7f5698068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.692+0000 7f56ba2e9700 1 --2- 172.21.15.81:0/1375519708 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5698066760 0x7f5698068c20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f56b00045f0 tx=0x7f56b0008040 comp rx=0 tx=0).stop 2024-05-08T23:31:20.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.692+0000 7f56ba2e9700 1 -- 172.21.15.81:0/1375519708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56b414cb10 msgr2=0x7f56b4138760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.692+0000 7f56ba2e9700 1 --2- 172.21.15.81:0/1375519708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56b414cb10 0x7f56b4138760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f56a400e570 tx=0x7f56a400e930 comp rx=0 tx=0).stop 2024-05-08T23:31:20.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.692+0000 7f56ba2e9700 1 -- 172.21.15.81:0/1375519708 shutdown_connections 2024-05-08T23:31:20.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.692+0000 7f56ba2e9700 1 --2- 172.21.15.81:0/1375519708 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5698066760 0x7f5698068c20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.692+0000 7f56ba2e9700 1 --2- 172.21.15.81:0/1375519708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b413b050 0x7f56b413b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.692+0000 7f56ba2e9700 1 --2- 172.21.15.81:0/1375519708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56b414cb10 0x7f56b4138760 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.692+0000 7f56ba2e9700 1 --2- 172.21.15.81:0/1375519708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56a000b510 0x7f56b414faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.692+0000 7f56ba2e9700 1 -- 172.21.15.81:0/1375519708 >> 172.21.15.81:0/1375519708 conn(0x7f56b4070960 msgr2=0x7f56b41524e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:20.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.692+0000 7f56ba2e9700 1 -- 172.21.15.81:0/1375519708 shutdown_connections 2024-05-08T23:31:20.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.692+0000 7f56ba2e9700 1 -- 172.21.15.81:0/1375519708 wait complete. 2024-05-08T23:31:20.712 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:20.713 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:20.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf42856700 1 Processor -- start 2024-05-08T23:31:20.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf42856700 1 -- start start 2024-05-08T23:31:20.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf42856700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faf3c0d3c50 0x7faf3c0d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf42856700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3c0b66d0 con 0x7faf3c160a30 2024-05-08T23:31:20.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf42856700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3c0d11c0 con 0x7faf3c0d1e90 2024-05-08T23:31:20.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf42856700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3c0d14a0 con 0x7faf3c0d3c50 2024-05-08T23:31:20.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3bfff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7faf3c160a30 0x7faf3c0d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34514/0 (socket says 172.21.15.81:34514) 2024-05-08T23:31:20.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3bfff700 1 -- 172.21.15.81:0/875992337 learned_addr learned my addr 172.21.15.81:0/875992337 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:20.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3b7fe700 1 --2- 172.21.15.81:0/875992337 >> v2:172.21.15.117:3300/0 conn(0x7faf3c0d3c50 0x7faf3c0d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3affd700 1 -- 172.21.15.81:0/875992337 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345893088 0 0) 0x7faf3c0b66d0 con 0x7faf3c160a30 2024-05-08T23:31:20.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3affd700 1 -- 172.21.15.81:0/875992337 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf240035d0 con 0x7faf3c160a30 2024-05-08T23:31:20.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3affd700 1 -- 172.21.15.81:0/875992337 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2973872747 0 0) 0x7faf240035d0 con 0x7faf3c160a30 2024-05-08T23:31:20.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3affd700 1 -- 172.21.15.81:0/875992337 >> v2:172.21.15.117:3300/0 conn(0x7faf3c0d3c50 msgr2=0x7faf3c0d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:20.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3affd700 1 --2- 172.21.15.81:0/875992337 >> v2:172.21.15.117:3300/0 conn(0x7faf3c0d3c50 0x7faf3c0d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3affd700 1 -- 172.21.15.81:0/875992337 >> v1:172.21.15.117:6789/0 conn(0x7faf3c0d1e90 legacy=0x7faf3c0b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3affd700 1 -- 172.21.15.81:0/875992337 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0b6080 con 0x7faf3c160a30 2024-05-08T23:31:20.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3affd700 1 -- 172.21.15.81:0/875992337 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7faf2c001670 con 0x7faf3c160a30 2024-05-08T23:31:20.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3affd700 1 -- 172.21.15.81:0/875992337 >> v1:172.21.15.81:6789/0 conn(0x7faf3c160a30 legacy=0x7faf3c0d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3affd700 1 --2- 172.21.15.81:0/875992337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf24003a20 0x7faf24003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3affd700 1 --2- 172.21.15.81:0/875992337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf24004870 0x7faf24004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3affd700 1 --2- 172.21.15.81:0/875992337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf24005230 0x7faf2400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3affd700 1 -- 172.21.15.81:0/875992337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0d14a0 con 0x7faf24005230 2024-05-08T23:31:20.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3affd700 1 -- 172.21.15.81:0/875992337 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2400c0d0 con 0x7faf24003a20 2024-05-08T23:31:20.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3affd700 1 -- 172.21.15.81:0/875992337 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2400c290 con 0x7faf24004870 2024-05-08T23:31:20.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf40df3700 1 --2- 172.21.15.81:0/875992337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf24005230 0x7faf2400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3bfff700 1 --2- 172.21.15.81:0/875992337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf24003a20 0x7faf24003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.832+0000 7faf3b7fe700 1 --2- 172.21.15.81:0/875992337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf24004870 0x7faf24004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf3bfff700 1 -- 172.21.15.81:0/875992337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf24004870 msgr2=0x7faf24004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:20.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf3bfff700 1 --2- 172.21.15.81:0/875992337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf24004870 0x7faf24004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf3bfff700 1 -- 172.21.15.81:0/875992337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf24005230 msgr2=0x7faf2400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:20.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf3bfff700 1 --2- 172.21.15.81:0/875992337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf24005230 0x7faf2400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf3bfff700 1 -- 172.21.15.81:0/875992337 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240035d0 con 0x7faf24003a20 2024-05-08T23:31:20.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf40df3700 1 -- 172.21.15.81:0/875992337 reap_dead start 2024-05-08T23:31:20.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf3bfff700 1 --2- 172.21.15.81:0/875992337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf24003a20 0x7faf24003e40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00c320 tx=0x7faf240058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c44595705b0873 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:20.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf3affd700 1 -- 172.21.15.81:0/875992337 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c007c30 con 0x7faf24003a20 2024-05-08T23:31:20.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf3affd700 1 -- 172.21.15.81:0/875992337 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c00ce60 con 0x7faf24003a20 2024-05-08T23:31:20.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf3affd700 1 -- 172.21.15.81:0/875992337 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c005bc0 con 0x7faf24003a20 2024-05-08T23:31:20.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf42856700 1 -- 172.21.15.81:0/875992337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf24003a20 msgr2=0x7faf24003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf42856700 1 --2- 172.21.15.81:0/875992337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf24003a20 0x7faf24003e40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00c320 tx=0x7faf240058a0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf42856700 1 -- 172.21.15.81:0/875992337 shutdown_connections 2024-05-08T23:31:20.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf42856700 1 --2- 172.21.15.81:0/875992337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf24003a20 0x7faf24003e40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf42856700 1 -- 172.21.15.81:0/875992337 >> 172.21.15.81:0/875992337 conn(0x7faf3c09c500 msgr2=0x7faf3c0d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:20.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf42856700 1 -- 172.21.15.81:0/875992337 shutdown_connections 2024-05-08T23:31:20.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf42856700 1 -- 172.21.15.81:0/875992337 wait complete. 2024-05-08T23:31:20.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf42856700 1 Processor -- start 2024-05-08T23:31:20.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf42856700 1 -- start start 2024-05-08T23:31:20.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf42856700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf24003a20 0x7faf3c06c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf42856700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf3c14f280 0x7faf3c14f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf42856700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf3c09dbf0 0x7faf3c09e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf42856700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c09fb00 con 0x7faf3c09dbf0 2024-05-08T23:31:20.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf42856700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0a0e20 con 0x7faf24003a20 2024-05-08T23:31:20.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf42856700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c14fc20 con 0x7faf3c14f280 2024-05-08T23:31:20.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf40df3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf3c09dbf0 0x7faf3c09e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf3bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf24003a20 0x7faf3c06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf40df3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf3c09dbf0 0x7faf3c09e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57410/0 (socket says 172.21.15.81:57410) 2024-05-08T23:31:20.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf40df3700 1 -- 172.21.15.81:0/512856103 learned_addr learned my addr 172.21.15.81:0/512856103 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:20.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.836+0000 7faf3b7fe700 1 --2- 172.21.15.81:0/512856103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf3c14f280 0x7faf3c14f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.840+0000 7faf3bfff700 1 -- 172.21.15.81:0/512856103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf3c14f280 msgr2=0x7faf3c14f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.840+0000 7faf3bfff700 1 --2- 172.21.15.81:0/512856103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf3c14f280 0x7faf3c14f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.840+0000 7faf3bfff700 1 -- 172.21.15.81:0/512856103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf3c09dbf0 msgr2=0x7faf3c09e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:20.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.840+0000 7faf3bfff700 1 --2- 172.21.15.81:0/512856103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf3c09dbf0 0x7faf3c09e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:20.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.840+0000 7faf3bfff700 1 -- 172.21.15.81:0/512856103 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c09fd80 con 0x7faf24003a20 2024-05-08T23:31:20.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.840+0000 7faf3bfff700 1 --2- 172.21.15.81:0/512856103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf24003a20 0x7faf3c06c390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf2c014000 tx=0x7faf2c001a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:20.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.840+0000 7faf38ff9700 1 -- 172.21.15.81:0/512856103 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c001640 con 0x7faf24003a20 2024-05-08T23:31:20.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.840+0000 7faf38ff9700 1 -- 172.21.15.81:0/512856103 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c0070d0 con 0x7faf24003a20 2024-05-08T23:31:20.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.840+0000 7faf42856700 1 -- 172.21.15.81:0/512856103 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c0a0060 con 0x7faf24003a20 2024-05-08T23:31:20.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.840+0000 7faf38ff9700 1 -- 172.21.15.81:0/512856103 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c007480 con 0x7faf24003a20 2024-05-08T23:31:20.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.840+0000 7faf42856700 1 -- 172.21.15.81:0/512856103 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c0a0620 con 0x7faf24003a20 2024-05-08T23:31:20.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.840+0000 7faf42856700 1 -- 172.21.15.81:0/512856103 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c0a0e20 con 0x7faf24003a20 2024-05-08T23:31:20.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.840+0000 7faf38ff9700 1 -- 172.21.15.81:0/512856103 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faf2c0075e0 con 0x7faf24003a20 2024-05-08T23:31:20.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.844+0000 7faf38ff9700 1 --2- 172.21.15.81:0/512856103 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faf28066760 0x7faf28068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:20.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.844+0000 7faf38ff9700 1 -- 172.21.15.81:0/512856103 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7faf2c085240 con 0x7faf24003a20 2024-05-08T23:31:20.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.844+0000 7faf3b7fe700 1 --2- 172.21.15.81:0/512856103 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faf28066760 0x7faf28068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:20.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.844+0000 7faf3b7fe700 1 --2- 172.21.15.81:0/512856103 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faf28066760 0x7faf28068c20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf3c1f64b0 tx=0x7faf30009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:20.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:20.852+0000 7faf38ff9700 1 -- 172.21.15.81:0/512856103 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faf2c0526a0 con 0x7faf24003a20 2024-05-08T23:31:21.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.024+0000 7faf42856700 1 -- 172.21.15.81:0/512856103 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf3c0684b0 con 0x7faf24003a20 2024-05-08T23:31:21.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.024+0000 7faf38ff9700 1 -- 172.21.15.81:0/512856103 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf2c055cc0 con 0x7faf24003a20 2024-05-08T23:31:21.029 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:21.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.024+0000 7faf42856700 1 -- 172.21.15.81:0/512856103 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faf28066760 msgr2=0x7faf28068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.024+0000 7faf42856700 1 --2- 172.21.15.81:0/512856103 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faf28066760 0x7faf28068c20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf3c1f64b0 tx=0x7faf30009240 comp rx=0 tx=0).stop 2024-05-08T23:31:21.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.024+0000 7faf42856700 1 -- 172.21.15.81:0/512856103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf24003a20 msgr2=0x7faf3c06c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.024+0000 7faf42856700 1 --2- 172.21.15.81:0/512856103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf24003a20 0x7faf3c06c390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf2c014000 tx=0x7faf2c001a00 comp rx=0 tx=0).stop 2024-05-08T23:31:21.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.024+0000 7faf42856700 1 -- 172.21.15.81:0/512856103 shutdown_connections 2024-05-08T23:31:21.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.028+0000 7faf42856700 1 --2- 172.21.15.81:0/512856103 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faf28066760 0x7faf28068c20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.028+0000 7faf42856700 1 --2- 172.21.15.81:0/512856103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf3c09dbf0 0x7faf3c09e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.028+0000 7faf42856700 1 --2- 172.21.15.81:0/512856103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf3c14f280 0x7faf3c14f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.028+0000 7faf42856700 1 --2- 172.21.15.81:0/512856103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf24003a20 0x7faf3c06c390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.028+0000 7faf42856700 1 -- 172.21.15.81:0/512856103 >> 172.21.15.81:0/512856103 conn(0x7faf3c09c500 msgr2=0x7faf3c0b7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:21.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.028+0000 7faf42856700 1 -- 172.21.15.81:0/512856103 shutdown_connections 2024-05-08T23:31:21.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.028+0000 7faf42856700 1 -- 172.21.15.81:0/512856103 wait complete. 2024-05-08T23:31:21.044 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:21.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.156+0000 7f64b29ea700 1 Processor -- start 2024-05-08T23:31:21.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.156+0000 7f64b29ea700 1 -- start start 2024-05-08T23:31:21.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b29ea700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f64ac0d1da0 0x7f64ac0cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b29ea700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64ac0cfe80 0x7f64ac0b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b29ea700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ac0d3d40 con 0x7f64ac160920 2024-05-08T23:31:21.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b29ea700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ac0d0310 con 0x7f64ac0d1da0 2024-05-08T23:31:21.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b29ea700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0d0d00 con 0x7f64ac0cfe80 2024-05-08T23:31:21.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b11e7700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f64ac0d1da0 0x7f64ac0cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b11e7700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f64ac0d1da0 0x7f64ac0cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57412/0 (socket says 172.21.15.81:57412) 2024-05-08T23:31:21.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b11e7700 1 -- 172.21.15.81:0/1203151025 learned_addr learned my addr 172.21.15.81:0/1203151025 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:21.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b19e8700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f64ac160920 0x7f64ac0ce6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34526/0 (socket says 172.21.15.81:34526) 2024-05-08T23:31:21.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b21e9700 1 --2- 172.21.15.81:0/1203151025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64ac0cfe80 0x7f64ac0b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b11e7700 1 -- 172.21.15.81:0/1203151025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64ac0cfe80 msgr2=0x7f64ac0b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:21.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b11e7700 1 --2- 172.21.15.81:0/1203151025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64ac0cfe80 0x7f64ac0b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b11e7700 1 -- 172.21.15.81:0/1203151025 >> v1:172.21.15.81:6789/0 conn(0x7f64ac160920 legacy=0x7f64ac0ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b11e7700 1 -- 172.21.15.81:0/1203151025 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac0d0fe0 con 0x7f64ac0d1da0 2024-05-08T23:31:21.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b09e6700 1 -- 172.21.15.81:0/1203151025 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409125096 0 0) 0x7f64ac0d3d40 con 0x7f64ac160920 2024-05-08T23:31:21.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b11e7700 1 --2- 172.21.15.81:0/1203151025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64ac0d1da0 0x7f64ac0cf940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009910 tx=0x7f64a8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c53a0197b4e449 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:21.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b09e6700 1 -- 172.21.15.81:0/1203151025 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8011830 con 0x7f64ac0d1da0 2024-05-08T23:31:21.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b09e6700 1 -- 172.21.15.81:0/1203151025 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a8011990 con 0x7f64ac0d1da0 2024-05-08T23:31:21.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b09e6700 1 -- 172.21.15.81:0/1203151025 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8011d40 con 0x7f64ac0d1da0 2024-05-08T23:31:21.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b29ea700 1 -- 172.21.15.81:0/1203151025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64ac0d1da0 msgr2=0x7f64ac0cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b29ea700 1 --2- 172.21.15.81:0/1203151025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64ac0d1da0 0x7f64ac0cf940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009910 tx=0x7f64a8009c20 comp rx=0 tx=0).stop 2024-05-08T23:31:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b29ea700 1 -- 172.21.15.81:0/1203151025 shutdown_connections 2024-05-08T23:31:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b29ea700 1 --2- 172.21.15.81:0/1203151025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64ac0cfe80 0x7f64ac0b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b29ea700 1 --2- 172.21.15.81:0/1203151025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64ac0d1da0 0x7f64ac0cf940 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b29ea700 1 -- 172.21.15.81:0/1203151025 >> 172.21.15.81:0/1203151025 conn(0x7f64ac09c340 msgr2=0x7f64ac0d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b29ea700 1 -- 172.21.15.81:0/1203151025 shutdown_connections 2024-05-08T23:31:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.160+0000 7f64b29ea700 1 -- 172.21.15.81:0/1203151025 wait complete. 2024-05-08T23:31:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b29ea700 1 Processor -- start 2024-05-08T23:31:21.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b29ea700 1 -- start start 2024-05-08T23:31:21.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b29ea700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64ac0cfe80 0x7f64ac0b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b29ea700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ac0d1da0 0x7f64ac0c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b29ea700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64ac160920 0x7f64ac155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b29ea700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0c91c0 con 0x7f64ac0cfe80 2024-05-08T23:31:21.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b29ea700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0cb120 con 0x7f64ac0d1da0 2024-05-08T23:31:21.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b29ea700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac155b50 con 0x7f64ac160920 2024-05-08T23:31:21.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b19e8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64ac0cfe80 0x7f64ac0b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b19e8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64ac0cfe80 0x7f64ac0b8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57422/0 (socket says 172.21.15.81:57422) 2024-05-08T23:31:21.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b19e8700 1 -- 172.21.15.81:0/3350782402 learned_addr learned my addr 172.21.15.81:0/3350782402 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:21.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b21e9700 1 --2- 172.21.15.81:0/3350782402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64ac160920 0x7f64ac155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b11e7700 1 --2- 172.21.15.81:0/3350782402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ac0d1da0 0x7f64ac0c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b19e8700 1 -- 172.21.15.81:0/3350782402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64ac160920 msgr2=0x7f64ac155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b19e8700 1 --2- 172.21.15.81:0/3350782402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64ac160920 0x7f64ac155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b19e8700 1 -- 172.21.15.81:0/3350782402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ac0d1da0 msgr2=0x7f64ac0c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b19e8700 1 --2- 172.21.15.81:0/3350782402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ac0d1da0 0x7f64ac0c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b19e8700 1 -- 172.21.15.81:0/3350782402 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a8009570 con 0x7f64ac0cfe80 2024-05-08T23:31:21.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.164+0000 7f64b19e8700 1 --2- 172.21.15.81:0/3350782402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64ac0cfe80 0x7f64ac0b8b40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f649c00c4e0 tx=0x7f649c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:21.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.168+0000 7f64a27fc700 1 -- 172.21.15.81:0/3350782402 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c004ca0 con 0x7f64ac0cfe80 2024-05-08T23:31:21.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.168+0000 7f64a27fc700 1 -- 172.21.15.81:0/3350782402 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649c004e20 con 0x7f64ac0cfe80 2024-05-08T23:31:21.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.168+0000 7f64b29ea700 1 -- 172.21.15.81:0/3350782402 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac155e30 con 0x7f64ac0cfe80 2024-05-08T23:31:21.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.168+0000 7f64a27fc700 1 -- 172.21.15.81:0/3350782402 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c007310 con 0x7f64ac0cfe80 2024-05-08T23:31:21.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.168+0000 7f64b29ea700 1 -- 172.21.15.81:0/3350782402 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac0cb370 con 0x7f64ac0cfe80 2024-05-08T23:31:21.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.168+0000 7f64b29ea700 1 -- 172.21.15.81:0/3350782402 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ac0684b0 con 0x7f64ac0cfe80 2024-05-08T23:31:21.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.168+0000 7f64a27fc700 1 -- 172.21.15.81:0/3350782402 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f649c007470 con 0x7f64ac0cfe80 2024-05-08T23:31:21.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.172+0000 7f64a27fc700 1 --2- 172.21.15.81:0/3350782402 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6490066730 0x7f6490068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.172+0000 7f64a27fc700 1 -- 172.21.15.81:0/3350782402 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f649c083cd0 con 0x7f64ac0cfe80 2024-05-08T23:31:21.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.172+0000 7f64b11e7700 1 --2- 172.21.15.81:0/3350782402 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6490066730 0x7f6490068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.172+0000 7f64b11e7700 1 --2- 172.21.15.81:0/3350782402 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6490066730 0x7f6490068bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f64a8014040 tx=0x7f64a8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:21.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.176+0000 7f64a27fc700 1 -- 172.21.15.81:0/3350782402 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f649c051050 con 0x7f64ac0cfe80 2024-05-08T23:31:21.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.340+0000 7f64b29ea700 1 -- 172.21.15.81:0/3350782402 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64ac0d27d0 con 0x7f64ac0cfe80 2024-05-08T23:31:21.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.340+0000 7f64a27fc700 1 -- 172.21.15.81:0/3350782402 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f649c007720 con 0x7f64ac0cfe80 2024-05-08T23:31:21.347 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:21.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.344+0000 7f64b29ea700 1 -- 172.21.15.81:0/3350782402 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6490066730 msgr2=0x7f6490068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.344+0000 7f64b29ea700 1 --2- 172.21.15.81:0/3350782402 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6490066730 0x7f6490068bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f64a8014040 tx=0x7f64a8008030 comp rx=0 tx=0).stop 2024-05-08T23:31:21.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.344+0000 7f64b29ea700 1 -- 172.21.15.81:0/3350782402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64ac0cfe80 msgr2=0x7f64ac0b8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.344+0000 7f64b29ea700 1 --2- 172.21.15.81:0/3350782402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64ac0cfe80 0x7f64ac0b8b40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f649c00c4e0 tx=0x7f649c00c7f0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.344+0000 7f64b29ea700 1 -- 172.21.15.81:0/3350782402 shutdown_connections 2024-05-08T23:31:21.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.344+0000 7f64b29ea700 1 --2- 172.21.15.81:0/3350782402 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6490066730 0x7f6490068bf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.344+0000 7f64b29ea700 1 --2- 172.21.15.81:0/3350782402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64ac160920 0x7f64ac155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.344+0000 7f64b29ea700 1 --2- 172.21.15.81:0/3350782402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64ac0d1da0 0x7f64ac0c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.344+0000 7f64b29ea700 1 --2- 172.21.15.81:0/3350782402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64ac0cfe80 0x7f64ac0b8b40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.344+0000 7f64b29ea700 1 -- 172.21.15.81:0/3350782402 >> 172.21.15.81:0/3350782402 conn(0x7f64ac09c340 msgr2=0x7f64ac0ce580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:21.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.344+0000 7f64b29ea700 1 -- 172.21.15.81:0/3350782402 shutdown_connections 2024-05-08T23:31:21.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.344+0000 7f64b29ea700 1 -- 172.21.15.81:0/3350782402 wait complete. 2024-05-08T23:31:21.362 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:21.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.476+0000 7fae6cfa2700 1 Processor -- start 2024-05-08T23:31:21.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.476+0000 7fae6cfa2700 1 -- start start 2024-05-08T23:31:21.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae6cfa2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fae6815d180 0x7fae680cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae6cfa2700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fae680d2f60 0x7fae680d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae6cfa2700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae680ce200 con 0x7fae6815d560 2024-05-08T23:31:21.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae6cfa2700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae680d3950 con 0x7fae6815d180 2024-05-08T23:31:21.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae6cfa2700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae680ce480 con 0x7fae680d2f60 2024-05-08T23:31:21.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae677fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fae6815d180 0x7fae680cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae677fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fae6815d180 0x7fae680cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57432/0 (socket says 172.21.15.81:57432) 2024-05-08T23:31:21.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae677fe700 1 -- 172.21.15.81:0/2028122596 learned_addr learned my addr 172.21.15.81:0/2028122596 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:21.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae67fff700 1 --2- 172.21.15.81:0/2028122596 >> v2:172.21.15.117:3300/0 conn(0x7fae680d2f60 0x7fae680d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae677fe700 1 -- 172.21.15.81:0/2028122596 >> v2:172.21.15.117:3300/0 conn(0x7fae680d2f60 msgr2=0x7fae680d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:21.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae677fe700 1 --2- 172.21.15.81:0/2028122596 >> v2:172.21.15.117:3300/0 conn(0x7fae680d2f60 0x7fae680d3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae677fe700 1 -- 172.21.15.81:0/2028122596 >> v1:172.21.15.117:6789/0 conn(0x7fae6815d560 legacy=0x7fae680d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae677fe700 1 -- 172.21.15.81:0/2028122596 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae680ce760 con 0x7fae6815d180 2024-05-08T23:31:21.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae677fe700 1 --2- 172.21.15.81:0/2028122596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae6815d180 0x7fae680cdcc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fae58009910 tx=0x7fae58009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8982b5f1a30e096 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:21.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae66ffd700 1 -- 172.21.15.81:0/2028122596 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58011830 con 0x7fae6815d180 2024-05-08T23:31:21.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae66ffd700 1 -- 172.21.15.81:0/2028122596 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae58011990 con 0x7fae6815d180 2024-05-08T23:31:21.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae66ffd700 1 -- 172.21.15.81:0/2028122596 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58011d40 con 0x7fae6815d180 2024-05-08T23:31:21.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae6cfa2700 1 -- 172.21.15.81:0/2028122596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae6815d180 msgr2=0x7fae680cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae6cfa2700 1 --2- 172.21.15.81:0/2028122596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae6815d180 0x7fae680cdcc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fae58009910 tx=0x7fae58009c20 comp rx=0 tx=0).stop 2024-05-08T23:31:21.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae6cfa2700 1 -- 172.21.15.81:0/2028122596 shutdown_connections 2024-05-08T23:31:21.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae6cfa2700 1 --2- 172.21.15.81:0/2028122596 >> v2:172.21.15.117:3300/0 conn(0x7fae680d2f60 0x7fae680d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae6cfa2700 1 --2- 172.21.15.81:0/2028122596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae6815d180 0x7fae680cdcc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae6cfa2700 1 -- 172.21.15.81:0/2028122596 >> 172.21.15.81:0/2028122596 conn(0x7fae6808c530 msgr2=0x7fae6809c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:21.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae6cfa2700 1 -- 172.21.15.81:0/2028122596 shutdown_connections 2024-05-08T23:31:21.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.480+0000 7fae6cfa2700 1 -- 172.21.15.81:0/2028122596 wait complete. 2024-05-08T23:31:21.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae6cfa2700 1 Processor -- start 2024-05-08T23:31:21.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae6cfa2700 1 -- start start 2024-05-08T23:31:21.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae6cfa2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae680d2f60 0x7fae68155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae6cfa2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae6815d180 0x7fae680ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae6cfa2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae6815d560 0x7fae680c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae6cfa2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae680b72b0 con 0x7fae6815d180 2024-05-08T23:31:21.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae6cfa2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae680b7590 con 0x7fae680d2f60 2024-05-08T23:31:21.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae6cfa2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae680b7870 con 0x7fae6815d560 2024-05-08T23:31:21.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae5ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae6815d180 0x7fae680ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae5ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae6815d180 0x7fae680ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57434/0 (socket says 172.21.15.81:57434) 2024-05-08T23:31:21.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae5ffff700 1 -- 172.21.15.81:0/670809365 learned_addr learned my addr 172.21.15.81:0/670809365 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:21.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae67fff700 1 --2- 172.21.15.81:0/670809365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae6815d560 0x7fae680c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae677fe700 1 --2- 172.21.15.81:0/670809365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae680d2f60 0x7fae68155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae5ffff700 1 -- 172.21.15.81:0/670809365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae6815d560 msgr2=0x7fae680c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae5ffff700 1 --2- 172.21.15.81:0/670809365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae6815d560 0x7fae680c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae5ffff700 1 -- 172.21.15.81:0/670809365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae680d2f60 msgr2=0x7fae68155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae5ffff700 1 --2- 172.21.15.81:0/670809365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae680d2f60 0x7fae68155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae5ffff700 1 -- 172.21.15.81:0/670809365 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae58009570 con 0x7fae6815d180 2024-05-08T23:31:21.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae5ffff700 1 --2- 172.21.15.81:0/670809365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae6815d180 0x7fae680ca0d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fae5000c430 tx=0x7fae5000c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:21.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae64ff9700 1 -- 172.21.15.81:0/670809365 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae500052d0 con 0x7fae6815d180 2024-05-08T23:31:21.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae64ff9700 1 -- 172.21.15.81:0/670809365 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae50005430 con 0x7fae6815d180 2024-05-08T23:31:21.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae6cfa2700 1 -- 172.21.15.81:0/670809365 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae680b7b50 con 0x7fae6815d180 2024-05-08T23:31:21.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae64ff9700 1 -- 172.21.15.81:0/670809365 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae500055a0 con 0x7fae6815d180 2024-05-08T23:31:21.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.484+0000 7fae6cfa2700 1 -- 172.21.15.81:0/670809365 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae68156930 con 0x7fae6815d180 2024-05-08T23:31:21.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.488+0000 7fae6cfa2700 1 -- 172.21.15.81:0/670809365 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae680b8070 con 0x7fae6815d180 2024-05-08T23:31:21.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.488+0000 7fae64ff9700 1 -- 172.21.15.81:0/670809365 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fae500036a0 con 0x7fae6815d180 2024-05-08T23:31:21.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.488+0000 7fae64ff9700 1 --2- 172.21.15.81:0/670809365 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fae48066760 0x7fae48068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.488+0000 7fae64ff9700 1 -- 172.21.15.81:0/670809365 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fae500832e0 con 0x7fae6815d180 2024-05-08T23:31:21.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.488+0000 7fae677fe700 1 --2- 172.21.15.81:0/670809365 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fae48066760 0x7fae48068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.488+0000 7fae677fe700 1 --2- 172.21.15.81:0/670809365 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fae48066760 0x7fae48068c20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fae58009c60 tx=0x7fae58008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:21.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.500+0000 7fae64ff9700 1 -- 172.21.15.81:0/670809365 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fae680b8070 con 0x7fae6815d180 2024-05-08T23:31:21.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.664+0000 7fae6cfa2700 1 -- 172.21.15.81:0/670809365 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae680b8070 con 0x7fae6815d180 2024-05-08T23:31:21.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.664+0000 7fae64ff9700 1 -- 172.21.15.81:0/670809365 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae680b8070 con 0x7fae6815d180 2024-05-08T23:31:21.670 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:21.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.668+0000 7fae6cfa2700 1 -- 172.21.15.81:0/670809365 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fae48066760 msgr2=0x7fae48068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.668+0000 7fae6cfa2700 1 --2- 172.21.15.81:0/670809365 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fae48066760 0x7fae48068c20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fae58009c60 tx=0x7fae58008030 comp rx=0 tx=0).stop 2024-05-08T23:31:21.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.668+0000 7fae6cfa2700 1 -- 172.21.15.81:0/670809365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae6815d180 msgr2=0x7fae680ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.668+0000 7fae6cfa2700 1 --2- 172.21.15.81:0/670809365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae6815d180 0x7fae680ca0d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fae5000c430 tx=0x7fae5000c740 comp rx=0 tx=0).stop 2024-05-08T23:31:21.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.668+0000 7fae6cfa2700 1 -- 172.21.15.81:0/670809365 shutdown_connections 2024-05-08T23:31:21.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.668+0000 7fae6cfa2700 1 --2- 172.21.15.81:0/670809365 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fae48066760 0x7fae48068c20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.668+0000 7fae6cfa2700 1 --2- 172.21.15.81:0/670809365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae6815d560 0x7fae680c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.668+0000 7fae6cfa2700 1 --2- 172.21.15.81:0/670809365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae6815d180 0x7fae680ca0d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.668+0000 7fae6cfa2700 1 --2- 172.21.15.81:0/670809365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae680d2f60 0x7fae68155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.668+0000 7fae6cfa2700 1 -- 172.21.15.81:0/670809365 >> 172.21.15.81:0/670809365 conn(0x7fae6808c530 msgr2=0x7fae6806bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:21.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.668+0000 7fae6cfa2700 1 -- 172.21.15.81:0/670809365 shutdown_connections 2024-05-08T23:31:21.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.668+0000 7fae6cfa2700 1 -- 172.21.15.81:0/670809365 wait complete. 2024-05-08T23:31:21.685 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:31:21.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.796+0000 7f42fded7700 1 Processor -- start 2024-05-08T23:31:21.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.796+0000 7f42fded7700 1 -- start start 2024-05-08T23:31:21.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.796+0000 7f42fded7700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f42f81596a0 0x7f42f80b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.796+0000 7f42fded7700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f42f80b4e50 0x7f42f8159a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.796+0000 7f42fded7700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f80b3590 con 0x7f42f8159fc0 2024-05-08T23:31:21.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.796+0000 7f42fded7700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f80b24f0 con 0x7f42f81596a0 2024-05-08T23:31:21.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.796+0000 7f42fded7700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f80b27d0 con 0x7f42f80b4e50 2024-05-08T23:31:21.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.796+0000 7f42fced5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f42f81596a0 0x7f42f80b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.796+0000 7f42fced5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f42f81596a0 0x7f42f80b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57442/0 (socket says 172.21.15.81:57442) 2024-05-08T23:31:21.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.796+0000 7f42fced5700 1 -- 172.21.15.81:0/1983430890 learned_addr learned my addr 172.21.15.81:0/1983430890 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:21.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.796+0000 7f42f7fff700 1 --2- 172.21.15.81:0/1983430890 >> v2:172.21.15.117:3300/0 conn(0x7f42f80b4e50 0x7f42f8159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.796+0000 7f42fced5700 1 -- 172.21.15.81:0/1983430890 >> v2:172.21.15.117:3300/0 conn(0x7f42f80b4e50 msgr2=0x7f42f8159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:21.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.796+0000 7f42fced5700 1 --2- 172.21.15.81:0/1983430890 >> v2:172.21.15.117:3300/0 conn(0x7f42f80b4e50 0x7f42f8159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.796+0000 7f42fced5700 1 -- 172.21.15.81:0/1983430890 >> v1:172.21.15.117:6789/0 conn(0x7f42f8159fc0 legacy=0x7f42f815d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.796+0000 7f42fced5700 1 -- 172.21.15.81:0/1983430890 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80b2ab0 con 0x7f42f81596a0 2024-05-08T23:31:21.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.800+0000 7f42fced5700 1 --2- 172.21.15.81:0/1983430890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f81596a0 0x7f42f80b4910 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f42f80cee40 tx=0x7f42e8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32feaab00ef61e7b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:21.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.800+0000 7f42f77fe700 1 -- 172.21.15.81:0/1983430890 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8011830 con 0x7f42f81596a0 2024-05-08T23:31:21.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.800+0000 7f42f77fe700 1 -- 172.21.15.81:0/1983430890 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e8011990 con 0x7f42f81596a0 2024-05-08T23:31:21.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.800+0000 7f42f77fe700 1 -- 172.21.15.81:0/1983430890 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8011d20 con 0x7f42f81596a0 2024-05-08T23:31:21.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.800+0000 7f42fded7700 1 -- 172.21.15.81:0/1983430890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f81596a0 msgr2=0x7f42f80b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.800+0000 7f42fded7700 1 --2- 172.21.15.81:0/1983430890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f81596a0 0x7f42f80b4910 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f42f80cee40 tx=0x7f42e8009bf0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.800+0000 7f42fded7700 1 -- 172.21.15.81:0/1983430890 shutdown_connections 2024-05-08T23:31:21.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.800+0000 7f42fded7700 1 --2- 172.21.15.81:0/1983430890 >> v2:172.21.15.117:3300/0 conn(0x7f42f80b4e50 0x7f42f8159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.800+0000 7f42fded7700 1 --2- 172.21.15.81:0/1983430890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f81596a0 0x7f42f80b4910 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.800+0000 7f42fded7700 1 -- 172.21.15.81:0/1983430890 >> 172.21.15.81:0/1983430890 conn(0x7f42f806c330 msgr2=0x7f42f806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:21.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.800+0000 7f42fded7700 1 -- 172.21.15.81:0/1983430890 shutdown_connections 2024-05-08T23:31:21.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.800+0000 7f42fded7700 1 -- 172.21.15.81:0/1983430890 wait complete. 2024-05-08T23:31:21.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.800+0000 7f42fded7700 1 Processor -- start 2024-05-08T23:31:21.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fded7700 1 -- start start 2024-05-08T23:31:21.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fded7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42f80b4e50 0x7f42f80cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fded7700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f81596a0 0x7f42f80cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fded7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f8159fc0 0x7f42f814d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fded7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f815c4c0 con 0x7f42f8159fc0 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fded7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f815c1b0 con 0x7f42f80b4e50 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fded7700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f815ad00 con 0x7f42f81596a0 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fd6d6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f8159fc0 0x7f42f814d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fd6d6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f8159fc0 0x7f42f814d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57458/0 (socket says 172.21.15.81:57458) 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fd6d6700 1 -- 172.21.15.81:0/568993802 learned_addr learned my addr 172.21.15.81:0/568993802 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fced5700 1 --2- 172.21.15.81:0/568993802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42f80b4e50 0x7f42f80cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42f7fff700 1 --2- 172.21.15.81:0/568993802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f81596a0 0x7f42f80cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fd6d6700 1 -- 172.21.15.81:0/568993802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f81596a0 msgr2=0x7f42f80cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fd6d6700 1 --2- 172.21.15.81:0/568993802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f81596a0 0x7f42f80cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fd6d6700 1 -- 172.21.15.81:0/568993802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42f80b4e50 msgr2=0x7f42f80cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fd6d6700 1 --2- 172.21.15.81:0/568993802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42f80b4e50 0x7f42f80cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fd6d6700 1 -- 172.21.15.81:0/568993802 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e8009570 con 0x7f42f8159fc0 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42f7fff700 1 --2- 172.21.15.81:0/568993802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f81596a0 0x7f42f80cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fd6d6700 1 --2- 172.21.15.81:0/568993802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f8159fc0 0x7f42f814d930 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f42f000cc00 tx=0x7f42f000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42f57fa700 1 -- 172.21.15.81:0/568993802 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0003c40 con 0x7f42f8159fc0 2024-05-08T23:31:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42f57fa700 1 -- 172.21.15.81:0/568993802 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f0003da0 con 0x7f42f8159fc0 2024-05-08T23:31:21.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42f57fa700 1 -- 172.21.15.81:0/568993802 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f000ab30 con 0x7f42f8159fc0 2024-05-08T23:31:21.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fded7700 1 -- 172.21.15.81:0/568993802 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f814ee60 con 0x7f42f8159fc0 2024-05-08T23:31:21.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.804+0000 7f42fded7700 1 -- 172.21.15.81:0/568993802 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f815c950 con 0x7f42f8159fc0 2024-05-08T23:31:21.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.808+0000 7f42fded7700 1 -- 172.21.15.81:0/568993802 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f815c670 con 0x7f42f8159fc0 2024-05-08T23:31:21.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.808+0000 7f42f57fa700 1 -- 172.21.15.81:0/568993802 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f42f000ac90 con 0x7f42f8159fc0 2024-05-08T23:31:21.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.808+0000 7f42f57fa700 1 --2- 172.21.15.81:0/568993802 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42d8066730 0x7f42d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:21.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.808+0000 7f42f57fa700 1 -- 172.21.15.81:0/568993802 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f42f00841d0 con 0x7f42f8159fc0 2024-05-08T23:31:21.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.808+0000 7f42fced5700 1 --2- 172.21.15.81:0/568993802 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42d8066730 0x7f42d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:21.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.808+0000 7f42fced5700 1 --2- 172.21.15.81:0/568993802 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42d8066730 0x7f42d8068bf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f42e8000c00 tx=0x7f42e8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:21.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.816+0000 7f42f57fa700 1 -- 172.21.15.81:0/568993802 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f42f00514c0 con 0x7f42f8159fc0 2024-05-08T23:31:22.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:21.996+0000 7f42fded7700 1 -- 172.21.15.81:0/568993802 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f42f80684b0 con 0x7f42f8159fc0 2024-05-08T23:31:22.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.000+0000 7f42f57fa700 1 -- 172.21.15.81:0/568993802 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f42f0054ae0 con 0x7f42f8159fc0 2024-05-08T23:31:22.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.000+0000 7f42fded7700 1 -- 172.21.15.81:0/568993802 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42d8066730 msgr2=0x7f42d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.000+0000 7f42fded7700 1 --2- 172.21.15.81:0/568993802 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42d8066730 0x7f42d8068bf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f42e8000c00 tx=0x7f42e8008030 comp rx=0 tx=0).stop 2024-05-08T23:31:22.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.000+0000 7f42fded7700 1 -- 172.21.15.81:0/568993802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f8159fc0 msgr2=0x7f42f814d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.000+0000 7f42fded7700 1 --2- 172.21.15.81:0/568993802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f8159fc0 0x7f42f814d930 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f42f000cc00 tx=0x7f42f000cf10 comp rx=0 tx=0).stop 2024-05-08T23:31:22.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.000+0000 7f42fded7700 1 -- 172.21.15.81:0/568993802 shutdown_connections 2024-05-08T23:31:22.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.000+0000 7f42fded7700 1 --2- 172.21.15.81:0/568993802 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f42d8066730 0x7f42d8068bf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.000+0000 7f42fded7700 1 --2- 172.21.15.81:0/568993802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f8159fc0 0x7f42f814d930 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.000+0000 7f42fded7700 1 --2- 172.21.15.81:0/568993802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42f81596a0 0x7f42f80cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.000+0000 7f42fded7700 1 --2- 172.21.15.81:0/568993802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42f80b4e50 0x7f42f80cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.000+0000 7f42fded7700 1 -- 172.21.15.81:0/568993802 >> 172.21.15.81:0/568993802 conn(0x7f42f806c330 msgr2=0x7f42f80711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:22.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.000+0000 7f42fded7700 1 -- 172.21.15.81:0/568993802 shutdown_connections 2024-05-08T23:31:22.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.000+0000 7f42fded7700 1 -- 172.21.15.81:0/568993802 wait complete. 2024-05-08T23:31:22.008 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:31:22.023 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:31:22.023 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:31:22.031 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:22.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb4759e700 1 Processor -- start 2024-05-08T23:31:22.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb4759e700 1 -- start start 2024-05-08T23:31:22.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb4759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb4815ac70 0x7ffb4813d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb4759e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb480ce080 con 0x7ffb480ce510 2024-05-08T23:31:22.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb4759e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb480cd7c0 con 0x7ffb48071ab0 2024-05-08T23:31:22.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb4759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb480cda30 con 0x7ffb4815ac70 2024-05-08T23:31:22.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb45d9b700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ffb480ce510 0x7ffb4815a560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34534/0 (socket says 172.21.15.81:34534) 2024-05-08T23:31:22.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb45d9b700 1 -- 172.21.15.81:0/2804754015 learned_addr learned my addr 172.21.15.81:0/2804754015 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:22.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb46d9d700 1 --2- 172.21.15.81:0/2804754015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb4815ac70 0x7ffb4813d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb4559a700 1 -- 172.21.15.81:0/2804754015 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 972294851 0 0) 0x7ffb480ce080 con 0x7ffb480ce510 2024-05-08T23:31:22.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb4559a700 1 -- 172.21.15.81:0/2804754015 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb280057e0 con 0x7ffb480ce510 2024-05-08T23:31:22.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb46d9d700 1 -- 172.21.15.81:0/2804754015 >> v1:172.21.15.117:6789/0 conn(0x7ffb48071ab0 legacy=0x7ffb48071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb46d9d700 1 -- 172.21.15.81:0/2804754015 >> v1:172.21.15.81:6789/0 conn(0x7ffb480ce510 legacy=0x7ffb4815a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb46d9d700 1 -- 172.21.15.81:0/2804754015 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb480cdd10 con 0x7ffb4815ac70 2024-05-08T23:31:22.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb46d9d700 1 --2- 172.21.15.81:0/2804754015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb4815ac70 0x7ffb4813d500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009880 tx=0x7ffb38009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f71fdf23be69bc08 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:22.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb4559a700 1 -- 172.21.15.81:0/2804754015 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38011830 con 0x7ffb4815ac70 2024-05-08T23:31:22.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.140+0000 7ffb4559a700 1 -- 172.21.15.81:0/2804754015 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb38011990 con 0x7ffb4815ac70 2024-05-08T23:31:22.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.144+0000 7ffb4759e700 1 -- 172.21.15.81:0/2804754015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb4815ac70 msgr2=0x7ffb4813d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.144+0000 7ffb4759e700 1 --2- 172.21.15.81:0/2804754015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb4815ac70 0x7ffb4813d500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009880 tx=0x7ffb38009b90 comp rx=0 tx=0).stop 2024-05-08T23:31:22.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.144+0000 7ffb4759e700 1 -- 172.21.15.81:0/2804754015 shutdown_connections 2024-05-08T23:31:22.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.144+0000 7ffb4759e700 1 --2- 172.21.15.81:0/2804754015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb4815ac70 0x7ffb4813d500 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.144+0000 7ffb4759e700 1 -- 172.21.15.81:0/2804754015 >> 172.21.15.81:0/2804754015 conn(0x7ffb4806c330 msgr2=0x7ffb4806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:22.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.144+0000 7ffb4759e700 1 -- 172.21.15.81:0/2804754015 shutdown_connections 2024-05-08T23:31:22.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.144+0000 7ffb4759e700 1 -- 172.21.15.81:0/2804754015 wait complete. 2024-05-08T23:31:22.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.144+0000 7ffb4759e700 1 Processor -- start 2024-05-08T23:31:22.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.144+0000 7ffb4759e700 1 -- start start 2024-05-08T23:31:22.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb4759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb48071ab0 0x7ffb4813c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb4759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb480ce510 0x7ffb48156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb4759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4815ac70 0x7ffb48157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb4759e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4813c5f0 con 0x7ffb4815ac70 2024-05-08T23:31:22.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb4759e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb480b30b0 con 0x7ffb480ce510 2024-05-08T23:31:22.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb4759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb48156240 con 0x7ffb48071ab0 2024-05-08T23:31:22.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb46d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4815ac70 0x7ffb48157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb46d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4815ac70 0x7ffb48157450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57466/0 (socket says 172.21.15.81:57466) 2024-05-08T23:31:22.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb46d9d700 1 -- 172.21.15.81:0/1575176065 learned_addr learned my addr 172.21.15.81:0/1575176065 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:22.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb45d9b700 1 --2- 172.21.15.81:0/1575176065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb480ce510 0x7ffb48156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb4659c700 1 --2- 172.21.15.81:0/1575176065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb48071ab0 0x7ffb4813c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb46d9d700 1 -- 172.21.15.81:0/1575176065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb48071ab0 msgr2=0x7ffb4813c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb46d9d700 1 --2- 172.21.15.81:0/1575176065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb48071ab0 0x7ffb4813c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb46d9d700 1 -- 172.21.15.81:0/1575176065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb480ce510 msgr2=0x7ffb48156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb46d9d700 1 --2- 172.21.15.81:0/1575176065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb480ce510 0x7ffb48156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb46d9d700 1 -- 172.21.15.81:0/1575176065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb380094b0 con 0x7ffb4815ac70 2024-05-08T23:31:22.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb46d9d700 1 --2- 172.21.15.81:0/1575176065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4815ac70 0x7ffb48157450 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffb380097d0 tx=0x7ffb38004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:22.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb3effd700 1 -- 172.21.15.81:0/1575176065 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3801a800 con 0x7ffb4815ac70 2024-05-08T23:31:22.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb3effd700 1 -- 172.21.15.81:0/1575176065 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3801e070 con 0x7ffb4815ac70 2024-05-08T23:31:22.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb4759e700 1 -- 172.21.15.81:0/1575176065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb480b3350 con 0x7ffb4815ac70 2024-05-08T23:31:22.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb3effd700 1 -- 172.21.15.81:0/1575176065 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3801abc0 con 0x7ffb4815ac70 2024-05-08T23:31:22.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.148+0000 7ffb4759e700 1 -- 172.21.15.81:0/1575176065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb480b3910 con 0x7ffb4815ac70 2024-05-08T23:31:22.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.152+0000 7ffb3effd700 1 -- 172.21.15.81:0/1575176065 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffb3801ad20 con 0x7ffb4815ac70 2024-05-08T23:31:22.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.152+0000 7ffb4759e700 1 -- 172.21.15.81:0/1575176065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb480b3b00 con 0x7ffb4815ac70 2024-05-08T23:31:22.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.152+0000 7ffb3effd700 1 --2- 172.21.15.81:0/1575176065 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffb2c066730 0x7ffb2c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.152+0000 7ffb3effd700 1 -- 172.21.15.81:0/1575176065 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ffb38084fc0 con 0x7ffb4815ac70 2024-05-08T23:31:22.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.152+0000 7ffb4659c700 1 --2- 172.21.15.81:0/1575176065 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffb2c066730 0x7ffb2c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.152+0000 7ffb4659c700 1 --2- 172.21.15.81:0/1575176065 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffb2c066730 0x7ffb2c068bf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffb40000f60 tx=0x7ffb40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:22.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.156+0000 7ffb3effd700 1 -- 172.21.15.81:0/1575176065 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffb380b6cf0 con 0x7ffb4815ac70 2024-05-08T23:31:22.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.324+0000 7ffb4759e700 1 -- 172.21.15.81:0/1575176065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb480684b0 con 0x7ffb4815ac70 2024-05-08T23:31:22.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.328+0000 7ffb3effd700 1 -- 172.21.15.81:0/1575176065 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16287 (secure 0 0 0) 0x7ffb38052340 con 0x7ffb4815ac70 2024-05-08T23:31:22.334 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:22.335 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":24,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:20.851517+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:22.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.332+0000 7ffb4759e700 1 -- 172.21.15.81:0/1575176065 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffb2c066730 msgr2=0x7ffb2c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.332+0000 7ffb4759e700 1 --2- 172.21.15.81:0/1575176065 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffb2c066730 0x7ffb2c068bf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffb40000f60 tx=0x7ffb40009040 comp rx=0 tx=0).stop 2024-05-08T23:31:22.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.332+0000 7ffb4759e700 1 -- 172.21.15.81:0/1575176065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4815ac70 msgr2=0x7ffb48157450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.332+0000 7ffb4759e700 1 --2- 172.21.15.81:0/1575176065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4815ac70 0x7ffb48157450 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffb380097d0 tx=0x7ffb38004770 comp rx=0 tx=0).stop 2024-05-08T23:31:22.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.332+0000 7ffb4759e700 1 -- 172.21.15.81:0/1575176065 shutdown_connections 2024-05-08T23:31:22.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.332+0000 7ffb4759e700 1 --2- 172.21.15.81:0/1575176065 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffb2c066730 0x7ffb2c068bf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.332+0000 7ffb4759e700 1 --2- 172.21.15.81:0/1575176065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4815ac70 0x7ffb48157450 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.332+0000 7ffb4759e700 1 --2- 172.21.15.81:0/1575176065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb480ce510 0x7ffb48156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.332+0000 7ffb4759e700 1 --2- 172.21.15.81:0/1575176065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb48071ab0 0x7ffb4813c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.332+0000 7ffb4759e700 1 -- 172.21.15.81:0/1575176065 >> 172.21.15.81:0/1575176065 conn(0x7ffb4806c330 msgr2=0x7ffb4813dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:22.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.332+0000 7ffb4759e700 1 -- 172.21.15.81:0/1575176065 shutdown_connections 2024-05-08T23:31:22.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.332+0000 7ffb4759e700 1 -- 172.21.15.81:0/1575176065 wait complete. 2024-05-08T23:31:22.353 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:22.353 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:22.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694fb8b700 1 Processor -- start 2024-05-08T23:31:22.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694fb8b700 1 -- start start 2024-05-08T23:31:22.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694fb8b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f694813d180 0x7f694815b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694fb8b700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69480b3380 con 0x7f694806ca60 2024-05-08T23:31:22.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694fb8b700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69480b3660 con 0x7f694813cda0 2024-05-08T23:31:22.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694fb8b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694815b6a0 con 0x7f694813d180 2024-05-08T23:31:22.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694d126700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f694806ca60 0x7f694815a950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34540/0 (socket says 172.21.15.81:34540) 2024-05-08T23:31:22.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694d126700 1 -- 172.21.15.81:0/3801524822 learned_addr learned my addr 172.21.15.81:0/3801524822 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:22.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694e128700 1 --2- 172.21.15.81:0/3801524822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f694813d180 0x7f694815b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694c925700 1 -- 172.21.15.81:0/3801524822 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072361708 0 0) 0x7f69480b3380 con 0x7f694806ca60 2024-05-08T23:31:22.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694c925700 1 -- 172.21.15.81:0/3801524822 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69340035d0 con 0x7f694806ca60 2024-05-08T23:31:22.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694e128700 1 -- 172.21.15.81:0/3801524822 >> v1:172.21.15.117:6789/0 conn(0x7f694813cda0 legacy=0x7f694806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694e128700 1 -- 172.21.15.81:0/3801524822 >> v1:172.21.15.81:6789/0 conn(0x7f694806ca60 legacy=0x7f694815a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694e128700 1 -- 172.21.15.81:0/3801524822 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69480b3940 con 0x7f694813d180 2024-05-08T23:31:22.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694e128700 1 --2- 172.21.15.81:0/3801524822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f694813d180 0x7f694815b060 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6940009dc0 tx=0x7f694000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ddf87a86509c6af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:22.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694c925700 1 -- 172.21.15.81:0/3801524822 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694000e070 con 0x7f694813d180 2024-05-08T23:31:22.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694c925700 1 -- 172.21.15.81:0/3801524822 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694000bb20 con 0x7f694813d180 2024-05-08T23:31:22.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.460+0000 7f694c925700 1 -- 172.21.15.81:0/3801524822 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940012430 con 0x7f694813d180 2024-05-08T23:31:22.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.464+0000 7f694fb8b700 1 -- 172.21.15.81:0/3801524822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f694813d180 msgr2=0x7f694815b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.464+0000 7f694fb8b700 1 --2- 172.21.15.81:0/3801524822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f694813d180 0x7f694815b060 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6940009dc0 tx=0x7f694000b330 comp rx=0 tx=0).stop 2024-05-08T23:31:22.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.464+0000 7f694fb8b700 1 -- 172.21.15.81:0/3801524822 shutdown_connections 2024-05-08T23:31:22.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.464+0000 7f694fb8b700 1 --2- 172.21.15.81:0/3801524822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f694813d180 0x7f694815b060 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.464+0000 7f694fb8b700 1 -- 172.21.15.81:0/3801524822 >> 172.21.15.81:0/3801524822 conn(0x7f694806a360 msgr2=0x7f694806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:22.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.464+0000 7f694fb8b700 1 -- 172.21.15.81:0/3801524822 shutdown_connections 2024-05-08T23:31:22.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.464+0000 7f694fb8b700 1 -- 172.21.15.81:0/3801524822 wait complete. 2024-05-08T23:31:22.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.464+0000 7f694fb8b700 1 Processor -- start 2024-05-08T23:31:22.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.464+0000 7f694fb8b700 1 -- start start 2024-05-08T23:31:22.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694fb8b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f694806ca60 0x7f6948071d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694fb8b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f694813cda0 0x7f69481573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694fb8b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f694813d180 0x7f694813b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694fb8b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6948136d20 con 0x7f694806ca60 2024-05-08T23:31:22.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694fb8b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6948154620 con 0x7f694813d180 2024-05-08T23:31:22.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694fb8b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6948153630 con 0x7f694813cda0 2024-05-08T23:31:22.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694d927700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f694806ca60 0x7f6948071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694d927700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f694806ca60 0x7f6948071d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57482/0 (socket says 172.21.15.81:57482) 2024-05-08T23:31:22.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694d927700 1 -- 172.21.15.81:0/1916548578 learned_addr learned my addr 172.21.15.81:0/1916548578 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:22.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694d126700 1 --2- 172.21.15.81:0/1916548578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f694813cda0 0x7f69481573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694e128700 1 --2- 172.21.15.81:0/1916548578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f694813d180 0x7f694813b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694d927700 1 -- 172.21.15.81:0/1916548578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f694813cda0 msgr2=0x7f69481573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694d927700 1 --2- 172.21.15.81:0/1916548578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f694813cda0 0x7f69481573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694d927700 1 -- 172.21.15.81:0/1916548578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f694813d180 msgr2=0x7f694813b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694d927700 1 --2- 172.21.15.81:0/1916548578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f694813d180 0x7f694813b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694d927700 1 -- 172.21.15.81:0/1916548578 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6940009a70 con 0x7f694806ca60 2024-05-08T23:31:22.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694d126700 1 --2- 172.21.15.81:0/1916548578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f694813cda0 0x7f69481573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:22.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694d927700 1 --2- 172.21.15.81:0/1916548578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f694806ca60 0x7f6948071d10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f693c00c500 tx=0x7f693c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:22.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f693a7fc700 1 -- 172.21.15.81:0/1916548578 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c0052d0 con 0x7f694806ca60 2024-05-08T23:31:22.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f693a7fc700 1 -- 172.21.15.81:0/1916548578 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693c005430 con 0x7f694806ca60 2024-05-08T23:31:22.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f693a7fc700 1 -- 172.21.15.81:0/1916548578 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c0055a0 con 0x7f694806ca60 2024-05-08T23:31:22.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694fb8b700 1 -- 172.21.15.81:0/1916548578 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6948153910 con 0x7f694806ca60 2024-05-08T23:31:22.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.468+0000 7f694fb8b700 1 -- 172.21.15.81:0/1916548578 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6948072250 con 0x7f694806ca60 2024-05-08T23:31:22.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.472+0000 7f694fb8b700 1 -- 172.21.15.81:0/1916548578 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6948153b50 con 0x7f694806ca60 2024-05-08T23:31:22.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.472+0000 7f693a7fc700 1 -- 172.21.15.81:0/1916548578 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f693c0038e0 con 0x7f694806ca60 2024-05-08T23:31:22.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.472+0000 7f693a7fc700 1 --2- 172.21.15.81:0/1916548578 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f692c066730 0x7f692c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.472+0000 7f693a7fc700 1 -- 172.21.15.81:0/1916548578 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f693c083740 con 0x7f694806ca60 2024-05-08T23:31:22.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.472+0000 7f694d126700 1 --2- 172.21.15.81:0/1916548578 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f692c066730 0x7f692c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.472+0000 7f694d126700 1 --2- 172.21.15.81:0/1916548578 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f692c066730 0x7f692c068bf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f69481662b0 tx=0x7f6944009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:22.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.480+0000 7f693a7fc700 1 -- 172.21.15.81:0/1916548578 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f693c0509b0 con 0x7f694806ca60 2024-05-08T23:31:22.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.644+0000 7f694fb8b700 1 -- 172.21.15.81:0/1916548578 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f69480684b0 con 0x7f694806ca60 2024-05-08T23:31:22.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.644+0000 7f693a7fc700 1 -- 172.21.15.81:0/1916548578 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f693c053fd0 con 0x7f694806ca60 2024-05-08T23:31:22.650 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:22.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.648+0000 7f694fb8b700 1 -- 172.21.15.81:0/1916548578 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f692c066730 msgr2=0x7f692c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.648+0000 7f694fb8b700 1 --2- 172.21.15.81:0/1916548578 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f692c066730 0x7f692c068bf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f69481662b0 tx=0x7f6944009040 comp rx=0 tx=0).stop 2024-05-08T23:31:22.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.648+0000 7f694fb8b700 1 -- 172.21.15.81:0/1916548578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f694806ca60 msgr2=0x7f6948071d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.648+0000 7f694fb8b700 1 --2- 172.21.15.81:0/1916548578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f694806ca60 0x7f6948071d10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f693c00c500 tx=0x7f693c00c8c0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.648+0000 7f694fb8b700 1 -- 172.21.15.81:0/1916548578 shutdown_connections 2024-05-08T23:31:22.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.648+0000 7f694fb8b700 1 --2- 172.21.15.81:0/1916548578 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f692c066730 0x7f692c068bf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.648+0000 7f694fb8b700 1 --2- 172.21.15.81:0/1916548578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f694813d180 0x7f694813b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.648+0000 7f694fb8b700 1 --2- 172.21.15.81:0/1916548578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f694813cda0 0x7f69481573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.648+0000 7f694fb8b700 1 --2- 172.21.15.81:0/1916548578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f694806ca60 0x7f6948071d10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.648+0000 7f694fb8b700 1 -- 172.21.15.81:0/1916548578 >> 172.21.15.81:0/1916548578 conn(0x7f694806a360 msgr2=0x7f694815ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:22.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.648+0000 7f694fb8b700 1 -- 172.21.15.81:0/1916548578 shutdown_connections 2024-05-08T23:31:22.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.648+0000 7f694fb8b700 1 -- 172.21.15.81:0/1916548578 wait complete. 2024-05-08T23:31:22.665 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:22.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.780+0000 7ff8e72dd700 1 Processor -- start 2024-05-08T23:31:22.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.780+0000 7ff8e72dd700 1 -- start start 2024-05-08T23:31:22.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e72dd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff8e0158760 0x7ff8e00cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e72dd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8e006aa70 0x7ff8e015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e72dd700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e00cf970 con 0x7ff8e0071dd0 2024-05-08T23:31:22.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e72dd700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e00cfc50 con 0x7ff8e0158760 2024-05-08T23:31:22.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e72dd700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e015b400 con 0x7ff8e006aa70 2024-05-08T23:31:22.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e6adc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8e006aa70 0x7ff8e015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e6adc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8e006aa70 0x7ff8e015e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:38386/0 (socket says 172.21.15.81:38386) 2024-05-08T23:31:22.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e62db700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff8e0158760 0x7ff8e00cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e6adc700 1 -- 172.21.15.81:0/2070249439 learned_addr learned my addr 172.21.15.81:0/2070249439 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:22.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e6adc700 1 -- 172.21.15.81:0/2070249439 >> v2:172.21.15.117:3300/0 conn(0x7ff8e0158760 msgr2=0x7ff8e00cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:22.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e6adc700 1 --2- 172.21.15.81:0/2070249439 >> v2:172.21.15.117:3300/0 conn(0x7ff8e0158760 0x7ff8e00cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e6adc700 1 -- 172.21.15.81:0/2070249439 >> v1:172.21.15.117:6789/0 conn(0x7ff8e0071dd0 legacy=0x7ff8e006a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e6adc700 1 -- 172.21.15.81:0/2070249439 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e015b610 con 0x7ff8e006aa70 2024-05-08T23:31:22.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e6adc700 1 --2- 172.21.15.81:0/2070249439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8e006aa70 0x7ff8e015e9f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009e50 tx=0x7ff8d800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2cd19f883986cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:22.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e52d9700 1 -- 172.21.15.81:0/2070249439 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d800e070 con 0x7ff8e006aa70 2024-05-08T23:31:22.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e52d9700 1 -- 172.21.15.81:0/2070249439 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d800b910 con 0x7ff8e006aa70 2024-05-08T23:31:22.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e52d9700 1 -- 172.21.15.81:0/2070249439 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d800bcc0 con 0x7ff8e006aa70 2024-05-08T23:31:22.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e72dd700 1 -- 172.21.15.81:0/2070249439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8e006aa70 msgr2=0x7ff8e015e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.784+0000 7ff8e72dd700 1 --2- 172.21.15.81:0/2070249439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8e006aa70 0x7ff8e015e9f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009e50 tx=0x7ff8d800b330 comp rx=0 tx=0).stop 2024-05-08T23:31:22.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.788+0000 7ff8e72dd700 1 -- 172.21.15.81:0/2070249439 shutdown_connections 2024-05-08T23:31:22.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.788+0000 7ff8e72dd700 1 --2- 172.21.15.81:0/2070249439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8e006aa70 0x7ff8e015e9f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.788+0000 7ff8e72dd700 1 --2- 172.21.15.81:0/2070249439 >> v2:172.21.15.117:3300/0 conn(0x7ff8e0158760 0x7ff8e00cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.788+0000 7ff8e72dd700 1 -- 172.21.15.81:0/2070249439 >> 172.21.15.81:0/2070249439 conn(0x7ff8e006b470 msgr2=0x7ff8e006b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:22.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.788+0000 7ff8e72dd700 1 -- 172.21.15.81:0/2070249439 shutdown_connections 2024-05-08T23:31:22.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.788+0000 7ff8e72dd700 1 -- 172.21.15.81:0/2070249439 wait complete. 2024-05-08T23:31:22.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.788+0000 7ff8e72dd700 1 Processor -- start 2024-05-08T23:31:22.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.788+0000 7ff8e72dd700 1 -- start start 2024-05-08T23:31:22.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e72dd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8e006aa70 0x7ff8e01436a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e72dd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8e0071dd0 0x7ff8e01f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e72dd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e0158760 0x7ff8e00cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e72dd700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e0152430 con 0x7ff8e006aa70 2024-05-08T23:31:22.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e72dd700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e00cea30 con 0x7ff8e0158760 2024-05-08T23:31:22.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e72dd700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e0143be0 con 0x7ff8e0071dd0 2024-05-08T23:31:22.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e62db700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8e006aa70 0x7ff8e01436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e62db700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8e006aa70 0x7ff8e01436a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57490/0 (socket says 172.21.15.81:57490) 2024-05-08T23:31:22.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e62db700 1 -- 172.21.15.81:0/194228154 learned_addr learned my addr 172.21.15.81:0/194228154 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:22.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e6adc700 1 --2- 172.21.15.81:0/194228154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e0158760 0x7ff8e00cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e5ada700 1 --2- 172.21.15.81:0/194228154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8e0071dd0 0x7ff8e01f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e62db700 1 -- 172.21.15.81:0/194228154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8e0071dd0 msgr2=0x7ff8e01f09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e62db700 1 --2- 172.21.15.81:0/194228154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8e0071dd0 0x7ff8e01f09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e62db700 1 -- 172.21.15.81:0/194228154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e0158760 msgr2=0x7ff8e00cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e62db700 1 --2- 172.21.15.81:0/194228154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e0158760 0x7ff8e00cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e62db700 1 -- 172.21.15.81:0/194228154 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d8009b30 con 0x7ff8e006aa70 2024-05-08T23:31:22.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e62db700 1 --2- 172.21.15.81:0/194228154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8e006aa70 0x7ff8e01436a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00e870 tx=0x7ff8dc00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:22.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8d6ffd700 1 -- 172.21.15.81:0/194228154 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0095a0 con 0x7ff8e006aa70 2024-05-08T23:31:22.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8d6ffd700 1 -- 172.21.15.81:0/194228154 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc013680 con 0x7ff8e006aa70 2024-05-08T23:31:22.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e72dd700 1 -- 172.21.15.81:0/194228154 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e0151ad0 con 0x7ff8e006aa70 2024-05-08T23:31:22.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8d6ffd700 1 -- 172.21.15.81:0/194228154 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0139f0 con 0x7ff8e006aa70 2024-05-08T23:31:22.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.792+0000 7ff8e72dd700 1 -- 172.21.15.81:0/194228154 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e0152050 con 0x7ff8e006aa70 2024-05-08T23:31:22.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.796+0000 7ff8e72dd700 1 -- 172.21.15.81:0/194228154 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e01526a0 con 0x7ff8e006aa70 2024-05-08T23:31:22.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.796+0000 7ff8d6ffd700 1 -- 172.21.15.81:0/194228154 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff8dc013bd0 con 0x7ff8e006aa70 2024-05-08T23:31:22.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.796+0000 7ff8d6ffd700 1 --2- 172.21.15.81:0/194228154 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff8c4066730 0x7ff8c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:22.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.796+0000 7ff8d6ffd700 1 -- 172.21.15.81:0/194228154 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff8dc0843e0 con 0x7ff8e006aa70 2024-05-08T23:31:22.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.796+0000 7ff8e5ada700 1 --2- 172.21.15.81:0/194228154 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff8c4066730 0x7ff8c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:22.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.796+0000 7ff8e5ada700 1 --2- 172.21.15.81:0/194228154 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff8c4066730 0x7ff8c4068bf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00cc840 tx=0x7ff8d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:22.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.804+0000 7ff8d6ffd700 1 -- 172.21.15.81:0/194228154 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff8dc051650 con 0x7ff8e006aa70 2024-05-08T23:31:22.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.976+0000 7ff8e72dd700 1 -- 172.21.15.81:0/194228154 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8e00672a0 con 0x7ff8e006aa70 2024-05-08T23:31:22.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.976+0000 7ff8d6ffd700 1 -- 172.21.15.81:0/194228154 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff8dc054c70 con 0x7ff8e006aa70 2024-05-08T23:31:22.982 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:22.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.976+0000 7ff8e72dd700 1 -- 172.21.15.81:0/194228154 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff8c4066730 msgr2=0x7ff8c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.976+0000 7ff8e72dd700 1 --2- 172.21.15.81:0/194228154 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff8c4066730 0x7ff8c4068bf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00cc840 tx=0x7ff8d0009040 comp rx=0 tx=0).stop 2024-05-08T23:31:22.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.976+0000 7ff8e72dd700 1 -- 172.21.15.81:0/194228154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8e006aa70 msgr2=0x7ff8e01436a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:22.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.976+0000 7ff8e72dd700 1 --2- 172.21.15.81:0/194228154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8e006aa70 0x7ff8e01436a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00e870 tx=0x7ff8dc00eb80 comp rx=0 tx=0).stop 2024-05-08T23:31:22.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.976+0000 7ff8e72dd700 1 -- 172.21.15.81:0/194228154 shutdown_connections 2024-05-08T23:31:22.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.976+0000 7ff8e72dd700 1 --2- 172.21.15.81:0/194228154 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff8c4066730 0x7ff8c4068bf0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.976+0000 7ff8e72dd700 1 --2- 172.21.15.81:0/194228154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e0158760 0x7ff8e00cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.976+0000 7ff8e72dd700 1 --2- 172.21.15.81:0/194228154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8e0071dd0 0x7ff8e01f09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.976+0000 7ff8e72dd700 1 --2- 172.21.15.81:0/194228154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8e006aa70 0x7ff8e01436a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:22.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.976+0000 7ff8e72dd700 1 -- 172.21.15.81:0/194228154 >> 172.21.15.81:0/194228154 conn(0x7ff8e006b470 msgr2=0x7ff8e0158b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:22.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.976+0000 7ff8e72dd700 1 -- 172.21.15.81:0/194228154 shutdown_connections 2024-05-08T23:31:22.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:22.976+0000 7ff8e72dd700 1 -- 172.21.15.81:0/194228154 wait complete. 2024-05-08T23:31:22.996 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:23.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.108+0000 7f7069178700 1 Processor -- start 2024-05-08T23:31:23.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.108+0000 7f7069178700 1 -- start start 2024-05-08T23:31:23.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.108+0000 7f7069178700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f706415ae10 0x7f70640ac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.108+0000 7f7069178700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70640ce6c0 0x7f706406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.108+0000 7f7069178700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706406c7d0 con 0x7f706415ead0 2024-05-08T23:31:23.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.108+0000 7f7069178700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706406c4d0 con 0x7f706415ae10 2024-05-08T23:31:23.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.108+0000 7f7069178700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70640acd90 con 0x7f70640ce6c0 2024-05-08T23:31:23.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7063fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f706415ae10 0x7f70640ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f70637fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f706415ead0 0x7f70640cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34546/0 (socket says 172.21.15.81:34546) 2024-05-08T23:31:23.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f70637fe700 1 -- 172.21.15.81:0/1918607547 learned_addr learned my addr 172.21.15.81:0/1918607547 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:23.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7068977700 1 --2- 172.21.15.81:0/1918607547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70640ce6c0 0x7f706406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7063fff700 1 -- 172.21.15.81:0/1918607547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70640ce6c0 msgr2=0x7f706406cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:23.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7063fff700 1 --2- 172.21.15.81:0/1918607547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70640ce6c0 0x7f706406cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7062ffd700 1 -- 172.21.15.81:0/1918607547 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989688569 0 0) 0x7f706406c7d0 con 0x7f706415ead0 2024-05-08T23:31:23.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7063fff700 1 -- 172.21.15.81:0/1918607547 >> v1:172.21.15.81:6789/0 conn(0x7f706415ead0 legacy=0x7f70640cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7063fff700 1 -- 172.21.15.81:0/1918607547 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70640ad070 con 0x7f706415ae10 2024-05-08T23:31:23.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7063fff700 1 --2- 172.21.15.81:0/1918607547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f706415ae10 0x7f70640ac420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f704c009850 tx=0x7f704c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377dd5246e720645 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:23.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7062ffd700 1 -- 172.21.15.81:0/1918607547 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c011830 con 0x7f706415ae10 2024-05-08T23:31:23.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7062ffd700 1 -- 172.21.15.81:0/1918607547 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c011990 con 0x7f706415ae10 2024-05-08T23:31:23.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7062ffd700 1 -- 172.21.15.81:0/1918607547 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c011d40 con 0x7f706415ae10 2024-05-08T23:31:23.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7069178700 1 -- 172.21.15.81:0/1918607547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f706415ae10 msgr2=0x7f70640ac420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7069178700 1 --2- 172.21.15.81:0/1918607547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f706415ae10 0x7f70640ac420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f704c009850 tx=0x7f704c009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:23.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7069178700 1 -- 172.21.15.81:0/1918607547 shutdown_connections 2024-05-08T23:31:23.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7069178700 1 --2- 172.21.15.81:0/1918607547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70640ce6c0 0x7f706406cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7069178700 1 --2- 172.21.15.81:0/1918607547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f706415ae10 0x7f70640ac420 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7069178700 1 -- 172.21.15.81:0/1918607547 >> 172.21.15.81:0/1918607547 conn(0x7f70640b0de0 msgr2=0x7f70640b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:23.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7069178700 1 -- 172.21.15.81:0/1918607547 shutdown_connections 2024-05-08T23:31:23.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.112+0000 7f7069178700 1 -- 172.21.15.81:0/1918607547 wait complete. 2024-05-08T23:31:23.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7069178700 1 Processor -- start 2024-05-08T23:31:23.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7069178700 1 -- start start 2024-05-08T23:31:23.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7069178700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70640ce6c0 0x7f7064158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7069178700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f706415ae10 0x7f7064152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7069178700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f706415ead0 0x7f70640ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7069178700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70641f1d20 con 0x7f70640ce6c0 2024-05-08T23:31:23.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7069178700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7064159010 con 0x7f706415ae10 2024-05-08T23:31:23.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7069178700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7064156f60 con 0x7f706415ead0 2024-05-08T23:31:23.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7063fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70640ce6c0 0x7f7064158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7063fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70640ce6c0 0x7f7064158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57510/0 (socket says 172.21.15.81:57510) 2024-05-08T23:31:23.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7063fff700 1 -- 172.21.15.81:0/355744077 learned_addr learned my addr 172.21.15.81:0/355744077 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:23.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7068977700 1 --2- 172.21.15.81:0/355744077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f706415ead0 0x7f70640ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f70637fe700 1 --2- 172.21.15.81:0/355744077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f706415ae10 0x7f7064152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7063fff700 1 -- 172.21.15.81:0/355744077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f706415ead0 msgr2=0x7f70640ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7063fff700 1 --2- 172.21.15.81:0/355744077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f706415ead0 0x7f70640ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7063fff700 1 -- 172.21.15.81:0/355744077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f706415ae10 msgr2=0x7f7064152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7063fff700 1 --2- 172.21.15.81:0/355744077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f706415ae10 0x7f7064152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7063fff700 1 -- 172.21.15.81:0/355744077 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c0094b0 con 0x7f70640ce6c0 2024-05-08T23:31:23.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7063fff700 1 --2- 172.21.15.81:0/355744077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70640ce6c0 0x7f7064158ad0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f704c000c00 tx=0x7f704c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:23.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7060ff9700 1 -- 172.21.15.81:0/355744077 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c004550 con 0x7f70640ce6c0 2024-05-08T23:31:23.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7060ff9700 1 -- 172.21.15.81:0/355744077 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c01f070 con 0x7f70640ce6c0 2024-05-08T23:31:23.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7069178700 1 -- 172.21.15.81:0/355744077 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7064157240 con 0x7f70640ce6c0 2024-05-08T23:31:23.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7060ff9700 1 -- 172.21.15.81:0/355744077 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c003900 con 0x7f70640ce6c0 2024-05-08T23:31:23.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.116+0000 7f7069178700 1 -- 172.21.15.81:0/355744077 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7064157800 con 0x7f70640ce6c0 2024-05-08T23:31:23.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.120+0000 7f7060ff9700 1 -- 172.21.15.81:0/355744077 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f704c003a60 con 0x7f70640ce6c0 2024-05-08T23:31:23.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.120+0000 7f7069178700 1 -- 172.21.15.81:0/355744077 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70640b2c40 con 0x7f70640ce6c0 2024-05-08T23:31:23.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.120+0000 7f7060ff9700 1 --2- 172.21.15.81:0/355744077 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7044066760 0x7f7044068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.120+0000 7f7060ff9700 1 -- 172.21.15.81:0/355744077 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f704c021070 con 0x7f70640ce6c0 2024-05-08T23:31:23.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.120+0000 7f70637fe700 1 --2- 172.21.15.81:0/355744077 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7044066760 0x7f7044068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.120+0000 7f70637fe700 1 --2- 172.21.15.81:0/355744077 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7044066760 0x7f7044068c20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f70640cc810 tx=0x7f7054001b70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:23.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.128+0000 7f7060ff9700 1 -- 172.21.15.81:0/355744077 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f704c052c80 con 0x7f70640ce6c0 2024-05-08T23:31:23.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.296+0000 7f7069178700 1 -- 172.21.15.81:0/355744077 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7064157990 con 0x7f70640ce6c0 2024-05-08T23:31:23.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.296+0000 7f7060ff9700 1 -- 172.21.15.81:0/355744077 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f704c0150a0 con 0x7f70640ce6c0 2024-05-08T23:31:23.302 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:23.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.300+0000 7f7069178700 1 -- 172.21.15.81:0/355744077 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7044066760 msgr2=0x7f7044068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.300+0000 7f7069178700 1 --2- 172.21.15.81:0/355744077 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7044066760 0x7f7044068c20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f70640cc810 tx=0x7f7054001b70 comp rx=0 tx=0).stop 2024-05-08T23:31:23.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.300+0000 7f7069178700 1 -- 172.21.15.81:0/355744077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70640ce6c0 msgr2=0x7f7064158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.300+0000 7f7069178700 1 --2- 172.21.15.81:0/355744077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70640ce6c0 0x7f7064158ad0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f704c000c00 tx=0x7f704c004770 comp rx=0 tx=0).stop 2024-05-08T23:31:23.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.300+0000 7f7069178700 1 -- 172.21.15.81:0/355744077 shutdown_connections 2024-05-08T23:31:23.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.300+0000 7f7069178700 1 --2- 172.21.15.81:0/355744077 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7044066760 0x7f7044068c20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.300+0000 7f7069178700 1 --2- 172.21.15.81:0/355744077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f706415ead0 0x7f70640ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.300+0000 7f7069178700 1 --2- 172.21.15.81:0/355744077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f706415ae10 0x7f7064152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.300+0000 7f7069178700 1 --2- 172.21.15.81:0/355744077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70640ce6c0 0x7f7064158ad0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.300+0000 7f7069178700 1 -- 172.21.15.81:0/355744077 >> 172.21.15.81:0/355744077 conn(0x7f70640b0de0 msgr2=0x7f706406d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:23.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.300+0000 7f7069178700 1 -- 172.21.15.81:0/355744077 shutdown_connections 2024-05-08T23:31:23.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.300+0000 7f7069178700 1 -- 172.21.15.81:0/355744077 wait complete. 2024-05-08T23:31:23.317 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:31:23.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.424+0000 7fdd53835700 1 Processor -- start 2024-05-08T23:31:23.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.424+0000 7fdd53835700 1 -- start start 2024-05-08T23:31:23.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd53835700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdd4c15aed0 0x7fdd4c0cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd53835700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdd4c06c350 0x7fdd4c06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd53835700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4c0cd470 con 0x7fdd4c0cb970 2024-05-08T23:31:23.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd53835700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4c06cd00 con 0x7fdd4c06c350 2024-05-08T23:31:23.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd53835700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4c0ced50 con 0x7fdd4c15aed0 2024-05-08T23:31:23.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd51dd2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdd4c06c350 0x7fdd4c06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd51dd2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdd4c06c350 0x7fdd4c06c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57524/0 (socket says 172.21.15.81:57524) 2024-05-08T23:31:23.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd51dd2700 1 -- 172.21.15.81:0/776405157 learned_addr learned my addr 172.21.15.81:0/776405157 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:23.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd515d1700 1 --2- 172.21.15.81:0/776405157 >> v2:172.21.15.117:3300/0 conn(0x7fdd4c15aed0 0x7fdd4c0cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd51dd2700 1 -- 172.21.15.81:0/776405157 >> v2:172.21.15.117:3300/0 conn(0x7fdd4c15aed0 msgr2=0x7fdd4c0cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:23.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd51dd2700 1 --2- 172.21.15.81:0/776405157 >> v2:172.21.15.117:3300/0 conn(0x7fdd4c15aed0 0x7fdd4c0cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd51dd2700 1 -- 172.21.15.81:0/776405157 >> v1:172.21.15.117:6789/0 conn(0x7fdd4c0cb970 legacy=0x7fdd4c0b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd51dd2700 1 -- 172.21.15.81:0/776405157 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c0cceb0 con 0x7fdd4c06c350 2024-05-08T23:31:23.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd51dd2700 1 --2- 172.21.15.81:0/776405157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4c06c350 0x7fdd4c06c7c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c06e3e0 tx=0x7fdd4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9277f3ef38827384 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:23.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd43fff700 1 -- 172.21.15.81:0/776405157 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400e070 con 0x7fdd4c06c350 2024-05-08T23:31:23.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd43fff700 1 -- 172.21.15.81:0/776405157 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4400b910 con 0x7fdd4c06c350 2024-05-08T23:31:23.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd43fff700 1 -- 172.21.15.81:0/776405157 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400bcc0 con 0x7fdd4c06c350 2024-05-08T23:31:23.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd53835700 1 -- 172.21.15.81:0/776405157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4c06c350 msgr2=0x7fdd4c06c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd53835700 1 --2- 172.21.15.81:0/776405157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4c06c350 0x7fdd4c06c7c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c06e3e0 tx=0x7fdd4400b330 comp rx=0 tx=0).stop 2024-05-08T23:31:23.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd53835700 1 -- 172.21.15.81:0/776405157 shutdown_connections 2024-05-08T23:31:23.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd53835700 1 --2- 172.21.15.81:0/776405157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4c06c350 0x7fdd4c06c7c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd53835700 1 --2- 172.21.15.81:0/776405157 >> v2:172.21.15.117:3300/0 conn(0x7fdd4c15aed0 0x7fdd4c0cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd53835700 1 -- 172.21.15.81:0/776405157 >> 172.21.15.81:0/776405157 conn(0x7fdd4c06a360 msgr2=0x7fdd4c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:23.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd53835700 1 -- 172.21.15.81:0/776405157 shutdown_connections 2024-05-08T23:31:23.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.428+0000 7fdd53835700 1 -- 172.21.15.81:0/776405157 wait complete. 2024-05-08T23:31:23.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd53835700 1 Processor -- start 2024-05-08T23:31:23.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd53835700 1 -- start start 2024-05-08T23:31:23.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd53835700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd4c06c350 0x7fdd4c145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd53835700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd4c0cb970 0x7fdd4c0af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd53835700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4c15aed0 0x7fdd4c1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd53835700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c0afd50 con 0x7fdd4c15aed0 2024-05-08T23:31:23.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd53835700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c0b0030 con 0x7fdd4c0cb970 2024-05-08T23:31:23.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd53835700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c0b0310 con 0x7fdd4c06c350 2024-05-08T23:31:23.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd51dd2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4c15aed0 0x7fdd4c1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd51dd2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4c15aed0 0x7fdd4c1539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57538/0 (socket says 172.21.15.81:57538) 2024-05-08T23:31:23.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd51dd2700 1 -- 172.21.15.81:0/114385793 learned_addr learned my addr 172.21.15.81:0/114385793 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:23.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd515d1700 1 --2- 172.21.15.81:0/114385793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd4c06c350 0x7fdd4c145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd50dd0700 1 --2- 172.21.15.81:0/114385793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd4c0cb970 0x7fdd4c0af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd51dd2700 1 -- 172.21.15.81:0/114385793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd4c06c350 msgr2=0x7fdd4c145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd51dd2700 1 --2- 172.21.15.81:0/114385793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd4c06c350 0x7fdd4c145d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd51dd2700 1 -- 172.21.15.81:0/114385793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd4c0cb970 msgr2=0x7fdd4c0af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd51dd2700 1 --2- 172.21.15.81:0/114385793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd4c0cb970 0x7fdd4c0af3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd51dd2700 1 -- 172.21.15.81:0/114385793 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd44009b30 con 0x7fdd4c15aed0 2024-05-08T23:31:23.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd51dd2700 1 --2- 172.21.15.81:0/114385793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4c15aed0 0x7fdd4c1539f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c074490 tx=0x7fdd44004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:23.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd41ffb700 1 -- 172.21.15.81:0/114385793 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44004880 con 0x7fdd4c15aed0 2024-05-08T23:31:23.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd41ffb700 1 -- 172.21.15.81:0/114385793 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd440099b0 con 0x7fdd4c15aed0 2024-05-08T23:31:23.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd53835700 1 -- 172.21.15.81:0/114385793 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4c146710 con 0x7fdd4c15aed0 2024-05-08T23:31:23.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd41ffb700 1 -- 172.21.15.81:0/114385793 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4401b670 con 0x7fdd4c15aed0 2024-05-08T23:31:23.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.432+0000 7fdd53835700 1 -- 172.21.15.81:0/114385793 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c146290 con 0x7fdd4c15aed0 2024-05-08T23:31:23.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.436+0000 7fdd53835700 1 -- 172.21.15.81:0/114385793 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4c146c30 con 0x7fdd4c15aed0 2024-05-08T23:31:23.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.436+0000 7fdd41ffb700 1 -- 172.21.15.81:0/114385793 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdd4401b7d0 con 0x7fdd4c15aed0 2024-05-08T23:31:23.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.436+0000 7fdd41ffb700 1 --2- 172.21.15.81:0/114385793 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdd30066730 0x7fdd30068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.436+0000 7fdd41ffb700 1 -- 172.21.15.81:0/114385793 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdd440856b0 con 0x7fdd4c15aed0 2024-05-08T23:31:23.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.436+0000 7fdd515d1700 1 --2- 172.21.15.81:0/114385793 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdd30066730 0x7fdd30068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.436+0000 7fdd515d1700 1 --2- 172.21.15.81:0/114385793 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdd30066730 0x7fdd30068bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009bf0 tx=0x7fdd3c009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:23.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.448+0000 7fdd41ffb700 1 -- 172.21.15.81:0/114385793 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdd4c146c30 con 0x7fdd4c15aed0 2024-05-08T23:31:23.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.628+0000 7fdd53835700 1 -- 172.21.15.81:0/114385793 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdd4c146c30 con 0x7fdd4c15aed0 2024-05-08T23:31:23.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.628+0000 7fdd41ffb700 1 -- 172.21.15.81:0/114385793 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdd4c146c30 con 0x7fdd4c15aed0 2024-05-08T23:31:23.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.632+0000 7fdd53835700 1 -- 172.21.15.81:0/114385793 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdd30066730 msgr2=0x7fdd30068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.632+0000 7fdd53835700 1 --2- 172.21.15.81:0/114385793 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdd30066730 0x7fdd30068bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009bf0 tx=0x7fdd3c009240 comp rx=0 tx=0).stop 2024-05-08T23:31:23.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.632+0000 7fdd53835700 1 -- 172.21.15.81:0/114385793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4c15aed0 msgr2=0x7fdd4c1539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.632+0000 7fdd53835700 1 --2- 172.21.15.81:0/114385793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4c15aed0 0x7fdd4c1539f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c074490 tx=0x7fdd44004af0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.632+0000 7fdd53835700 1 -- 172.21.15.81:0/114385793 shutdown_connections 2024-05-08T23:31:23.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.632+0000 7fdd53835700 1 --2- 172.21.15.81:0/114385793 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdd30066730 0x7fdd30068bf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.632+0000 7fdd53835700 1 --2- 172.21.15.81:0/114385793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4c15aed0 0x7fdd4c1539f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.632+0000 7fdd53835700 1 --2- 172.21.15.81:0/114385793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd4c0cb970 0x7fdd4c0af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.632+0000 7fdd53835700 1 --2- 172.21.15.81:0/114385793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd4c06c350 0x7fdd4c145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.632+0000 7fdd53835700 1 -- 172.21.15.81:0/114385793 >> 172.21.15.81:0/114385793 conn(0x7fdd4c06a360 msgr2=0x7fdd4c0661e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:23.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.632+0000 7fdd53835700 1 -- 172.21.15.81:0/114385793 shutdown_connections 2024-05-08T23:31:23.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.632+0000 7fdd53835700 1 -- 172.21.15.81:0/114385793 wait complete. 2024-05-08T23:31:23.638 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:31:23.648 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi117.front.sepia.ceph.com... 2024-05-08T23:31:23.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:23.648 DEBUG:teuthology.orchestra.run.smithi117:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-05-08T23:31:23.664 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:31:23.664 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:31:23.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:23.664 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-08T23:31:23.671 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T23:31:23.671 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:31:23.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.780+0000 7fe5b759e700 1 Processor -- start 2024-05-08T23:31:23.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.780+0000 7fe5b759e700 1 -- start start 2024-05-08T23:31:23.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b759e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe5b815afa0 0x7fe5b815b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b759e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b815eda0 con 0x7fe5b815b8c0 2024-05-08T23:31:23.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b759e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b815ea70 con 0x7fe5b80ce990 2024-05-08T23:31:23.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b759e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b80ced70 con 0x7fe5b815afa0 2024-05-08T23:31:23.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b5d9b700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe5b815b8c0 0x7fe5b815f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34552/0 (socket says 172.21.15.81:34552) 2024-05-08T23:31:23.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b5d9b700 1 -- 172.21.15.81:0/3919124606 learned_addr learned my addr 172.21.15.81:0/3919124606 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:23.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b659c700 1 --2- 172.21.15.81:0/3919124606 >> v2:172.21.15.81:3300/0 conn(0x7fe5b815afa0 0x7fe5b815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b559a700 1 -- 172.21.15.81:0/3919124606 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1479122921 0 0) 0x7fe5b815eda0 con 0x7fe5b815b8c0 2024-05-08T23:31:23.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b559a700 1 -- 172.21.15.81:0/3919124606 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5a00035d0 con 0x7fe5b815b8c0 2024-05-08T23:31:23.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b659c700 1 -- 172.21.15.81:0/3919124606 >> v1:172.21.15.117:6789/0 conn(0x7fe5b80ce990 legacy=0x7fe5b80b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b559a700 1 -- 172.21.15.81:0/3919124606 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3313177056 0 0) 0x7fe5a00035d0 con 0x7fe5b815b8c0 2024-05-08T23:31:23.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b659c700 1 -- 172.21.15.81:0/3919124606 >> v1:172.21.15.81:6789/0 conn(0x7fe5b815b8c0 legacy=0x7fe5b815f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b659c700 1 -- 172.21.15.81:0/3919124606 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b815bd40 con 0x7fe5b815afa0 2024-05-08T23:31:23.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b659c700 1 --2- 172.21.15.81:0/3919124606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5b815afa0 0x7fe5b815b380 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009850 tx=0x7fe5ac009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e88a40c9fb0724 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:23.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b559a700 1 -- 172.21.15.81:0/3919124606 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8001670 con 0x7fe5b815afa0 2024-05-08T23:31:23.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b559a700 1 -- 172.21.15.81:0/3919124606 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ac004030 con 0x7fe5b815afa0 2024-05-08T23:31:23.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b559a700 1 -- 172.21.15.81:0/3919124606 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0043a0 con 0x7fe5b815afa0 2024-05-08T23:31:23.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b759e700 1 -- 172.21.15.81:0/3919124606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5b815afa0 msgr2=0x7fe5b815b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b759e700 1 --2- 172.21.15.81:0/3919124606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5b815afa0 0x7fe5b815b380 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009850 tx=0x7fe5ac009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:23.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b759e700 1 -- 172.21.15.81:0/3919124606 shutdown_connections 2024-05-08T23:31:23.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b759e700 1 --2- 172.21.15.81:0/3919124606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5b815afa0 0x7fe5b815b380 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b759e700 1 -- 172.21.15.81:0/3919124606 >> 172.21.15.81:0/3919124606 conn(0x7fe5b806c330 msgr2=0x7fe5b806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:23.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b759e700 1 -- 172.21.15.81:0/3919124606 shutdown_connections 2024-05-08T23:31:23.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.784+0000 7fe5b759e700 1 -- 172.21.15.81:0/3919124606 wait complete. 2024-05-08T23:31:23.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b759e700 1 Processor -- start 2024-05-08T23:31:23.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b759e700 1 -- start start 2024-05-08T23:31:23.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5b80ce990 0x7fe5b8143770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5b815afa0 0x7fe5b8158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5b815b8c0 0x7fe5b815d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b759e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b80b22e0 con 0x7fe5b80ce990 2024-05-08T23:31:23.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b759e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b81595c0 con 0x7fe5b815afa0 2024-05-08T23:31:23.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b81598a0 con 0x7fe5b815b8c0 2024-05-08T23:31:23.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5b80ce990 0x7fe5b8143770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5b80ce990 0x7fe5b8143770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57556/0 (socket says 172.21.15.81:57556) 2024-05-08T23:31:23.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b659c700 1 -- 172.21.15.81:0/2946984704 learned_addr learned my addr 172.21.15.81:0/2946984704 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:23.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b5d9b700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5b815afa0 0x7fe5b8158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b6d9d700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5b815b8c0 0x7fe5b815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b659c700 1 -- 172.21.15.81:0/2946984704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5b815b8c0 msgr2=0x7fe5b815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b659c700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5b815b8c0 0x7fe5b815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b659c700 1 -- 172.21.15.81:0/2946984704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5b815afa0 msgr2=0x7fe5b8158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:23.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b659c700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5b815afa0 0x7fe5b8158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:23.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b659c700 1 -- 172.21.15.81:0/2946984704 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac0094b0 con 0x7fe5b80ce990 2024-05-08T23:31:23.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b6d9d700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5b815b8c0 0x7fe5b815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:23.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.788+0000 7fe5b5d9b700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5b815afa0 0x7fe5b8158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:23.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.792+0000 7fe5b659c700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5b80ce990 0x7fe5b8143770 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0090b0 tx=0x7fe5ac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:23.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.792+0000 7fe5a6ffd700 1 -- 172.21.15.81:0/2946984704 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac003ea0 con 0x7fe5b80ce990 2024-05-08T23:31:23.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.792+0000 7fe5a6ffd700 1 -- 172.21.15.81:0/2946984704 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ac01f070 con 0x7fe5b80ce990 2024-05-08T23:31:23.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.792+0000 7fe5b759e700 1 -- 172.21.15.81:0/2946984704 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b8159b80 con 0x7fe5b80ce990 2024-05-08T23:31:23.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.792+0000 7fe5a6ffd700 1 -- 172.21.15.81:0/2946984704 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac011a90 con 0x7fe5b80ce990 2024-05-08T23:31:23.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.792+0000 7fe5b759e700 1 -- 172.21.15.81:0/2946984704 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b81591b0 con 0x7fe5b80ce990 2024-05-08T23:31:23.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.792+0000 7fe5b759e700 1 -- 172.21.15.81:0/2946984704 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b80b2060 con 0x7fe5b80ce990 2024-05-08T23:31:23.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.792+0000 7fe5a6ffd700 1 -- 172.21.15.81:0/2946984704 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe5ac011bf0 con 0x7fe5b80ce990 2024-05-08T23:31:23.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.796+0000 7fe5a6ffd700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe598066730 0x7fe598068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:23.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.796+0000 7fe5a6ffd700 1 -- 172.21.15.81:0/2946984704 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe5ac021070 con 0x7fe5b80ce990 2024-05-08T23:31:23.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.796+0000 7fe5b5d9b700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe598066730 0x7fe598068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:23.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.796+0000 7fe5b5d9b700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe598066730 0x7fe598068bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80b3eb0 tx=0x7fe5a8001c80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:23.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:23.800+0000 7fe5a6ffd700 1 -- 172.21.15.81:0/2946984704 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe5ac052f40 con 0x7fe5b80ce990 2024-05-08T23:31:24.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.008+0000 7fe5b759e700 1 -- 172.21.15.81:0/2946984704 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5b80684b0 con 0x7fe5b80ce990 2024-05-08T23:31:24.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.012+0000 7fe5a6ffd700 1 -- 172.21.15.81:0/2946984704 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7fe5ac056560 con 0x7fe5b80ce990 2024-05-08T23:31:24.016 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:24.016 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1475726755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1475726755},{"type":"v1","addr":"172.21.15.81:6835","nonce":1475726755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3535820540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3535820540},{"type":"v1","addr":"172.21.15.81:6837","nonce":3535820540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1531019955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1531019955},{"type":"v1","addr":"172.21.15.117:6833","nonce":1531019955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:30:38.665368+0000","modified":"2024-05-08T23:30:41.672722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/491911303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":491911303},{"type":"v1","addr":"172.21.15.117:6835","nonce":491911303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:31:24.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.012+0000 7fe5b759e700 1 -- 172.21.15.81:0/2946984704 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe598066730 msgr2=0x7fe598068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.012+0000 7fe5b759e700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe598066730 0x7fe598068bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80b3eb0 tx=0x7fe5a8001c80 comp rx=0 tx=0).stop 2024-05-08T23:31:24.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.012+0000 7fe5b759e700 1 -- 172.21.15.81:0/2946984704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5b80ce990 msgr2=0x7fe5b8143770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.012+0000 7fe5b759e700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5b80ce990 0x7fe5b8143770 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0090b0 tx=0x7fe5ac004770 comp rx=0 tx=0).stop 2024-05-08T23:31:24.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.012+0000 7fe5b759e700 1 -- 172.21.15.81:0/2946984704 shutdown_connections 2024-05-08T23:31:24.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.012+0000 7fe5b759e700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe598066730 0x7fe598068bf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.012+0000 7fe5b759e700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5b815b8c0 0x7fe5b815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.012+0000 7fe5b759e700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5b815afa0 0x7fe5b8158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.012+0000 7fe5b759e700 1 --2- 172.21.15.81:0/2946984704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5b80ce990 0x7fe5b8143770 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.012+0000 7fe5b759e700 1 -- 172.21.15.81:0/2946984704 >> 172.21.15.81:0/2946984704 conn(0x7fe5b806c330 msgr2=0x7fe5b8155640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:24.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.012+0000 7fe5b759e700 1 -- 172.21.15.81:0/2946984704 shutdown_connections 2024-05-08T23:31:24.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.012+0000 7fe5b759e700 1 -- 172.21.15.81:0/2946984704 wait complete. 2024-05-08T23:31:24.020 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 7 2024-05-08T23:31:24.035 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:24.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 Processor -- start 2024-05-08T23:31:24.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 -- start start 2024-05-08T23:31:24.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f04140b3c20 0x7f04141624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0414071e70 0x7f04140b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0414165070 0x7f0414165500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04140b1f60 con 0x7f04140b3c20 2024-05-08T23:31:24.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0414158960 con 0x7f0414165070 2024-05-08T23:31:24.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04140b4540 con 0x7f0414071e70 2024-05-08T23:31:24.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f0419aab700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f04140b3c20 0x7f04141624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f0419aab700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f04140b3c20 0x7f04141624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57564/0 (socket says 172.21.15.81:57564) 2024-05-08T23:31:24.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f0419aab700 1 -- 172.21.15.81:0/2098250401 learned_addr learned my addr 172.21.15.81:0/2098250401 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:24.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f04192aa700 1 --2- 172.21.15.81:0/2098250401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0414071e70 0x7f04140b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041a2ac700 1 --2- 172.21.15.81:0/2098250401 >> v2:172.21.15.117:3300/0 conn(0x7f0414165070 0x7f0414165500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f0419aab700 1 -- 172.21.15.81:0/2098250401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0414071e70 msgr2=0x7f04140b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:24.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f0419aab700 1 --2- 172.21.15.81:0/2098250401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0414071e70 0x7f04140b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f0419aab700 1 -- 172.21.15.81:0/2098250401 >> v2:172.21.15.117:3300/0 conn(0x7f0414165070 msgr2=0x7f0414165500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:24.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f0419aab700 1 --2- 172.21.15.81:0/2098250401 >> v2:172.21.15.117:3300/0 conn(0x7f0414165070 0x7f0414165500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f0419aab700 1 -- 172.21.15.81:0/2098250401 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04140ae990 con 0x7f04140b3c20 2024-05-08T23:31:24.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f0419aab700 1 --2- 172.21.15.81:0/2098250401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04140b3c20 0x7f04141624b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f041404f630 tx=0x7f0410009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7eb2d71c27cef4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:24.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f0418aa9700 1 -- 172.21.15.81:0/2098250401 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041000b8a0 con 0x7f04140b3c20 2024-05-08T23:31:24.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f0418aa9700 1 -- 172.21.15.81:0/2098250401 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041000ba00 con 0x7f04140b3c20 2024-05-08T23:31:24.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f0418aa9700 1 -- 172.21.15.81:0/2098250401 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041000bd90 con 0x7f04140b3c20 2024-05-08T23:31:24.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 -- 172.21.15.81:0/2098250401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04140b3c20 msgr2=0x7f04141624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 --2- 172.21.15.81:0/2098250401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04140b3c20 0x7f04141624b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f041404f630 tx=0x7f0410009bf0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 -- 172.21.15.81:0/2098250401 shutdown_connections 2024-05-08T23:31:24.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 --2- 172.21.15.81:0/2098250401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0414071e70 0x7f04140b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 --2- 172.21.15.81:0/2098250401 >> v2:172.21.15.117:3300/0 conn(0x7f0414165070 0x7f0414165500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 --2- 172.21.15.81:0/2098250401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04140b3c20 0x7f04141624b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 -- 172.21.15.81:0/2098250401 >> 172.21.15.81:0/2098250401 conn(0x7f041406c330 msgr2=0x7f041406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:24.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 -- 172.21.15.81:0/2098250401 shutdown_connections 2024-05-08T23:31:24.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.140+0000 7f041aaad700 1 -- 172.21.15.81:0/2098250401 wait complete. 2024-05-08T23:31:24.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041aaad700 1 Processor -- start 2024-05-08T23:31:24.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041aaad700 1 -- start start 2024-05-08T23:31:24.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041aaad700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0414071e70 0x7f041414df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041aaad700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04140b3c20 0x7f04141faa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041aaad700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0414165070 0x7f04141565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041aaad700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04141fb9d0 con 0x7f0414165070 2024-05-08T23:31:24.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041aaad700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04141fb410 con 0x7f04140b3c20 2024-05-08T23:31:24.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041aaad700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04141fb6f0 con 0x7f0414071e70 2024-05-08T23:31:24.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041a2ac700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0414165070 0x7f04141565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041a2ac700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0414165070 0x7f04141565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57572/0 (socket says 172.21.15.81:57572) 2024-05-08T23:31:24.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041a2ac700 1 -- 172.21.15.81:0/3165845732 learned_addr learned my addr 172.21.15.81:0/3165845732 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:24.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f0419aab700 1 --2- 172.21.15.81:0/3165845732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0414071e70 0x7f041414df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f04192aa700 1 --2- 172.21.15.81:0/3165845732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04140b3c20 0x7f04141faa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041a2ac700 1 -- 172.21.15.81:0/3165845732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0414071e70 msgr2=0x7f041414df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041a2ac700 1 --2- 172.21.15.81:0/3165845732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0414071e70 0x7f041414df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041a2ac700 1 -- 172.21.15.81:0/3165845732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04140b3c20 msgr2=0x7f04141faa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041a2ac700 1 --2- 172.21.15.81:0/3165845732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04140b3c20 0x7f04141faa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041a2ac700 1 -- 172.21.15.81:0/3165845732 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0410009570 con 0x7f0414165070 2024-05-08T23:31:24.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f04192aa700 1 --2- 172.21.15.81:0/3165845732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04140b3c20 0x7f04141faa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:24.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041a2ac700 1 --2- 172.21.15.81:0/3165845732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0414165070 0x7f04141565a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f040800d800 tx=0x7f040800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:24.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f04067fc700 1 -- 172.21.15.81:0/3165845732 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04080125f0 con 0x7f0414165070 2024-05-08T23:31:24.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f04067fc700 1 -- 172.21.15.81:0/3165845732 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0408012750 con 0x7f0414165070 2024-05-08T23:31:24.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041aaad700 1 -- 172.21.15.81:0/3165845732 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041415a360 con 0x7f0414165070 2024-05-08T23:31:24.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f04067fc700 1 -- 172.21.15.81:0/3165845732 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0408012b00 con 0x7f0414165070 2024-05-08T23:31:24.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.144+0000 7f041aaad700 1 -- 172.21.15.81:0/3165845732 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041414e4a0 con 0x7f0414165070 2024-05-08T23:31:24.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.148+0000 7f041aaad700 1 -- 172.21.15.81:0/3165845732 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041415a1c0 con 0x7f0414165070 2024-05-08T23:31:24.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.148+0000 7f04067fc700 1 -- 172.21.15.81:0/3165845732 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0408012c60 con 0x7f0414165070 2024-05-08T23:31:24.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.148+0000 7f04067fc700 1 --2- 172.21.15.81:0/3165845732 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f03f8066730 0x7f03f8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.148+0000 7f0419aab700 1 --2- 172.21.15.81:0/3165845732 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f03f8066730 0x7f03f8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.148+0000 7f04067fc700 1 -- 172.21.15.81:0/3165845732 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f04080843c0 con 0x7f0414165070 2024-05-08T23:31:24.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.148+0000 7f0419aab700 1 --2- 172.21.15.81:0/3165845732 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f03f8066730 0x7f03f8068bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0410000c00 tx=0x7f0410008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:24.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.156+0000 7f04067fc700 1 -- 172.21.15.81:0/3165845732 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f041415a1c0 con 0x7f0414165070 2024-05-08T23:31:24.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.328+0000 7f041aaad700 1 -- 172.21.15.81:0/3165845732 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f041415a880 con 0x7f0414165070 2024-05-08T23:31:24.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.332+0000 7f04067fc700 1 -- 172.21.15.81:0/3165845732 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f041415a880 con 0x7f0414165070 2024-05-08T23:31:24.362 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:24.362 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":24,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:20.851517+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:24.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.332+0000 7f041aaad700 1 -- 172.21.15.81:0/3165845732 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f03f8066730 msgr2=0x7f03f8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.332+0000 7f041aaad700 1 --2- 172.21.15.81:0/3165845732 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f03f8066730 0x7f03f8068bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0410000c00 tx=0x7f0410008030 comp rx=0 tx=0).stop 2024-05-08T23:31:24.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.332+0000 7f041aaad700 1 -- 172.21.15.81:0/3165845732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0414165070 msgr2=0x7f04141565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.332+0000 7f041aaad700 1 --2- 172.21.15.81:0/3165845732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0414165070 0x7f04141565a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f040800d800 tx=0x7f040800db10 comp rx=0 tx=0).stop 2024-05-08T23:31:24.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.332+0000 7f041aaad700 1 -- 172.21.15.81:0/3165845732 shutdown_connections 2024-05-08T23:31:24.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.332+0000 7f041aaad700 1 --2- 172.21.15.81:0/3165845732 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f03f8066730 0x7f03f8068bf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.332+0000 7f041aaad700 1 --2- 172.21.15.81:0/3165845732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0414165070 0x7f04141565a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.332+0000 7f041aaad700 1 --2- 172.21.15.81:0/3165845732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04140b3c20 0x7f04141faa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.332+0000 7f041aaad700 1 --2- 172.21.15.81:0/3165845732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0414071e70 0x7f041414df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.332+0000 7f041aaad700 1 -- 172.21.15.81:0/3165845732 >> 172.21.15.81:0/3165845732 conn(0x7f041406c330 msgr2=0x7f0414162290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:24.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.332+0000 7f041aaad700 1 -- 172.21.15.81:0/3165845732 shutdown_connections 2024-05-08T23:31:24.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.332+0000 7f041aaad700 1 -- 172.21.15.81:0/3165845732 wait complete. 2024-05-08T23:31:24.365 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:30:36.469749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:30:37.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:24.366 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:24.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c1ef3700 1 Processor -- start 2024-05-08T23:31:24.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c1ef3700 1 -- start start 2024-05-08T23:31:24.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c1ef3700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc9bc09a180 0x7fc9bc165e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c1ef3700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9bc15c480 con 0x7fc9bc15c000 2024-05-08T23:31:24.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c1ef3700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9bc158eb0 con 0x7fc9bc158ad0 2024-05-08T23:31:24.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c1ef3700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9bc0b57a0 con 0x7fc9bc09a180 2024-05-08T23:31:24.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9bbfff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc9bc15c000 0x7fc9bc099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34558/0 (socket says 172.21.15.81:34558) 2024-05-08T23:31:24.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9bbfff700 1 -- 172.21.15.81:0/3361093046 learned_addr learned my addr 172.21.15.81:0/3361093046 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:24.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c16f2700 1 --2- 172.21.15.81:0/3361093046 >> v2:172.21.15.81:3300/0 conn(0x7fc9bc09a180 0x7fc9bc165e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9bb7fe700 1 -- 172.21.15.81:0/3361093046 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073561039 0 0) 0x7fc9b0001110 con 0x7fc9bc15c000 2024-05-08T23:31:24.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9bb7fe700 1 -- 172.21.15.81:0/3361093046 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9a40035d0 con 0x7fc9bc15c000 2024-05-08T23:31:24.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c16f2700 1 -- 172.21.15.81:0/3361093046 >> v1:172.21.15.117:6789/0 conn(0x7fc9bc158ad0 legacy=0x7fc9bc0b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c16f2700 1 -- 172.21.15.81:0/3361093046 >> v1:172.21.15.81:6789/0 conn(0x7fc9bc15c000 legacy=0x7fc9bc099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9bb7fe700 1 -- 172.21.15.81:0/3361093046 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2147609923 0 0) 0x7fc9a40035d0 con 0x7fc9bc15c000 2024-05-08T23:31:24.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c16f2700 1 -- 172.21.15.81:0/3361093046 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9bc0b51e0 con 0x7fc9bc09a180 2024-05-08T23:31:24.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c16f2700 1 --2- 172.21.15.81:0/3361093046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9bc09a180 0x7fc9bc165e70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009d90 tx=0x7fc9b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34dfbd07c69e501 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:24.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9bb7fe700 1 -- 172.21.15.81:0/3361093046 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b400e070 con 0x7fc9bc09a180 2024-05-08T23:31:24.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9bb7fe700 1 -- 172.21.15.81:0/3361093046 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b4003a40 con 0x7fc9bc09a180 2024-05-08T23:31:24.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9bb7fe700 1 -- 172.21.15.81:0/3361093046 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0001800 con 0x7fc9bc09a180 2024-05-08T23:31:24.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c1ef3700 1 -- 172.21.15.81:0/3361093046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9bc09a180 msgr2=0x7fc9bc165e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c1ef3700 1 --2- 172.21.15.81:0/3361093046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9bc09a180 0x7fc9bc165e70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009d90 tx=0x7fc9b400b330 comp rx=0 tx=0).stop 2024-05-08T23:31:24.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c1ef3700 1 -- 172.21.15.81:0/3361093046 shutdown_connections 2024-05-08T23:31:24.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c1ef3700 1 --2- 172.21.15.81:0/3361093046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9bc09a180 0x7fc9bc165e70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.488+0000 7fc9c1ef3700 1 -- 172.21.15.81:0/3361093046 >> 172.21.15.81:0/3361093046 conn(0x7fc9bc0b6590 msgr2=0x7fc9bc0b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:24.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.492+0000 7fc9c1ef3700 1 -- 172.21.15.81:0/3361093046 shutdown_connections 2024-05-08T23:31:24.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.492+0000 7fc9c1ef3700 1 -- 172.21.15.81:0/3361093046 wait complete. 2024-05-08T23:31:24.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.492+0000 7fc9c1ef3700 1 Processor -- start 2024-05-08T23:31:24.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.492+0000 7fc9c1ef3700 1 -- start start 2024-05-08T23:31:24.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.492+0000 7fc9c1ef3700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9bc09a180 0x7fc9bc1490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.492+0000 7fc9c1ef3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9bc158ad0 0x7fc9bc09df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.492+0000 7fc9c1ef3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9bc15c000 0x7fc9bc1458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.492+0000 7fc9c1ef3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc09e460 con 0x7fc9bc15c000 2024-05-08T23:31:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.492+0000 7fc9c1ef3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc09ef30 con 0x7fc9bc158ad0 2024-05-08T23:31:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.492+0000 7fc9c1ef3700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc09e690 con 0x7fc9bc09a180 2024-05-08T23:31:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.492+0000 7fc9c16f2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9bc15c000 0x7fc9bc1458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.492+0000 7fc9c16f2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9bc15c000 0x7fc9bc1458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57598/0 (socket says 172.21.15.81:57598) 2024-05-08T23:31:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.492+0000 7fc9c16f2700 1 -- 172.21.15.81:0/3518989318 learned_addr learned my addr 172.21.15.81:0/3518989318 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.492+0000 7fc9c0ef1700 1 --2- 172.21.15.81:0/3518989318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9bc09a180 0x7fc9bc1490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.492+0000 7fc9bbfff700 1 --2- 172.21.15.81:0/3518989318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9bc158ad0 0x7fc9bc09df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.496+0000 7fc9c0ef1700 1 -- 172.21.15.81:0/3518989318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9bc158ad0 msgr2=0x7fc9bc09df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.496+0000 7fc9c0ef1700 1 --2- 172.21.15.81:0/3518989318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9bc158ad0 0x7fc9bc09df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.496+0000 7fc9c0ef1700 1 -- 172.21.15.81:0/3518989318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9bc15c000 msgr2=0x7fc9bc1458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.496+0000 7fc9c0ef1700 1 --2- 172.21.15.81:0/3518989318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9bc15c000 0x7fc9bc1458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.496+0000 7fc9c0ef1700 1 -- 172.21.15.81:0/3518989318 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b4009a70 con 0x7fc9bc09a180 2024-05-08T23:31:24.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.496+0000 7fc9c0ef1700 1 --2- 172.21.15.81:0/3518989318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9bc09a180 0x7fc9bc1490c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac001eb0 tx=0x7fc9ac00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:24.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.496+0000 7fc9b97fa700 1 -- 172.21.15.81:0/3518989318 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0054e0 con 0x7fc9bc09a180 2024-05-08T23:31:24.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.496+0000 7fc9b97fa700 1 -- 172.21.15.81:0/3518989318 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ac0036a0 con 0x7fc9bc09a180 2024-05-08T23:31:24.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.496+0000 7fc9c1ef3700 1 -- 172.21.15.81:0/3518989318 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9bc09e970 con 0x7fc9bc09a180 2024-05-08T23:31:24.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.496+0000 7fc9b97fa700 1 -- 172.21.15.81:0/3518989318 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac003a50 con 0x7fc9bc09a180 2024-05-08T23:31:24.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.496+0000 7fc9c1ef3700 1 -- 172.21.15.81:0/3518989318 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9bc0ae4b0 con 0x7fc9bc09a180 2024-05-08T23:31:24.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.496+0000 7fc9c1ef3700 1 -- 172.21.15.81:0/3518989318 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9bc0672a0 con 0x7fc9bc09a180 2024-05-08T23:31:24.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.496+0000 7fc9b97fa700 1 -- 172.21.15.81:0/3518989318 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc9ac003bb0 con 0x7fc9bc09a180 2024-05-08T23:31:24.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.496+0000 7fc9b97fa700 1 --2- 172.21.15.81:0/3518989318 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc9a8066730 0x7fc9a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.496+0000 7fc9b97fa700 1 -- 172.21.15.81:0/3518989318 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc9ac0839a0 con 0x7fc9bc09a180 2024-05-08T23:31:24.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.500+0000 7fc9bbfff700 1 --2- 172.21.15.81:0/3518989318 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc9a8066730 0x7fc9a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.500+0000 7fc9bbfff700 1 --2- 172.21.15.81:0/3518989318 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc9a8066730 0x7fc9a8068bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc158430 tx=0x7fc9b0001d20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:24.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.508+0000 7fc9b97fa700 1 -- 172.21.15.81:0/3518989318 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc9ac050d20 con 0x7fc9bc09a180 2024-05-08T23:31:24.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.680+0000 7fc9c1ef3700 1 -- 172.21.15.81:0/3518989318 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9bc09a900 con 0x7fc9bc09a180 2024-05-08T23:31:24.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.680+0000 7fc9b97fa700 1 -- 172.21.15.81:0/3518989318 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9ac017020 con 0x7fc9bc09a180 2024-05-08T23:31:24.687 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:24.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.684+0000 7fc9c1ef3700 1 -- 172.21.15.81:0/3518989318 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc9a8066730 msgr2=0x7fc9a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.684+0000 7fc9c1ef3700 1 --2- 172.21.15.81:0/3518989318 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc9a8066730 0x7fc9a8068bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc158430 tx=0x7fc9b0001d20 comp rx=0 tx=0).stop 2024-05-08T23:31:24.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.684+0000 7fc9c1ef3700 1 -- 172.21.15.81:0/3518989318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9bc09a180 msgr2=0x7fc9bc1490c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.684+0000 7fc9c1ef3700 1 --2- 172.21.15.81:0/3518989318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9bc09a180 0x7fc9bc1490c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac001eb0 tx=0x7fc9ac00c800 comp rx=0 tx=0).stop 2024-05-08T23:31:24.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.684+0000 7fc9c1ef3700 1 -- 172.21.15.81:0/3518989318 shutdown_connections 2024-05-08T23:31:24.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.684+0000 7fc9c1ef3700 1 --2- 172.21.15.81:0/3518989318 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc9a8066730 0x7fc9a8068bf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.684+0000 7fc9c1ef3700 1 --2- 172.21.15.81:0/3518989318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9bc15c000 0x7fc9bc1458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.684+0000 7fc9c1ef3700 1 --2- 172.21.15.81:0/3518989318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9bc158ad0 0x7fc9bc09df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.684+0000 7fc9c1ef3700 1 --2- 172.21.15.81:0/3518989318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9bc09a180 0x7fc9bc1490c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.684+0000 7fc9c1ef3700 1 -- 172.21.15.81:0/3518989318 >> 172.21.15.81:0/3518989318 conn(0x7fc9bc0b6590 msgr2=0x7fc9bc15c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:24.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.684+0000 7fc9c1ef3700 1 -- 172.21.15.81:0/3518989318 shutdown_connections 2024-05-08T23:31:24.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.684+0000 7fc9c1ef3700 1 -- 172.21.15.81:0/3518989318 wait complete. 2024-05-08T23:31:24.702 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:24.826 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.820+0000 7f869b4b1700 1 Processor -- start 2024-05-08T23:31:24.826 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.820+0000 7f869b4b1700 1 -- start start 2024-05-08T23:31:24.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.820+0000 7f869b4b1700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8694071a70 0x7f869406cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.820+0000 7f869b4b1700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869406d570 con 0x7f869406d150 2024-05-08T23:31:24.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.820+0000 7f869b4b1700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86940b1df0 con 0x7f86941594d0 2024-05-08T23:31:24.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.820+0000 7f869b4b1700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86940ae330 con 0x7f8694071a70 2024-05-08T23:31:24.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.820+0000 7f8699cae700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8694071a70 0x7f869406cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.820+0000 7f869acb0700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f869406d150 0x7f86940b2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34560/0 (socket says 172.21.15.81:34560) 2024-05-08T23:31:24.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.820+0000 7f869acb0700 1 -- 172.21.15.81:0/2153619295 learned_addr learned my addr 172.21.15.81:0/2153619295 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:24.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.820+0000 7f8699cae700 1 -- 172.21.15.81:0/2153619295 >> v1:172.21.15.117:6789/0 conn(0x7f86941594d0 legacy=0x7f86940ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.820+0000 7f8699cae700 1 -- 172.21.15.81:0/2153619295 >> v1:172.21.15.81:6789/0 conn(0x7f869406d150 legacy=0x7f86940b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.820+0000 7f8699cae700 1 -- 172.21.15.81:0/2153619295 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940b20d0 con 0x7f8694071a70 2024-05-08T23:31:24.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.820+0000 7f8699cae700 1 --2- 172.21.15.81:0/2153619295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8694071a70 0x7f869406cc10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8684009850 tx=0x7f8684009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e791939c6f3d6b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:24.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.820+0000 7f86994ad700 1 -- 172.21.15.81:0/2153619295 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684004230 con 0x7f8694071a70 2024-05-08T23:31:24.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.820+0000 7f86994ad700 1 -- 172.21.15.81:0/2153619295 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8684004390 con 0x7f8694071a70 2024-05-08T23:31:24.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.824+0000 7f86994ad700 1 -- 172.21.15.81:0/2153619295 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86840036c0 con 0x7f8694071a70 2024-05-08T23:31:24.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.824+0000 7f869b4b1700 1 -- 172.21.15.81:0/2153619295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8694071a70 msgr2=0x7f869406cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.824+0000 7f869b4b1700 1 --2- 172.21.15.81:0/2153619295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8694071a70 0x7f869406cc10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8684009850 tx=0x7f8684009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:24.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.824+0000 7f869b4b1700 1 -- 172.21.15.81:0/2153619295 shutdown_connections 2024-05-08T23:31:24.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.824+0000 7f869b4b1700 1 --2- 172.21.15.81:0/2153619295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8694071a70 0x7f869406cc10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.824+0000 7f869b4b1700 1 -- 172.21.15.81:0/2153619295 >> 172.21.15.81:0/2153619295 conn(0x7f86940b0de0 msgr2=0x7f86940b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:24.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.824+0000 7f869b4b1700 1 -- 172.21.15.81:0/2153619295 shutdown_connections 2024-05-08T23:31:24.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.824+0000 7f869b4b1700 1 -- 172.21.15.81:0/2153619295 wait complete. 2024-05-08T23:31:24.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.824+0000 7f869b4b1700 1 Processor -- start 2024-05-08T23:31:24.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.824+0000 7f869b4b1700 1 -- start start 2024-05-08T23:31:24.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.824+0000 7f869b4b1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f869406d150 0x7f8694157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.824+0000 7f869b4b1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8694071a70 0x7f86941ee150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.824+0000 7f869b4b1700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86941594d0 0x7f86941ee690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f869b4b1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86941eebd0 con 0x7f8694071a70 2024-05-08T23:31:24.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f869b4b1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86941567d0 con 0x7f869406d150 2024-05-08T23:31:24.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f869b4b1700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86940cdee0 con 0x7f86941594d0 2024-05-08T23:31:24.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f8699cae700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8694071a70 0x7f86941ee150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f8699cae700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8694071a70 0x7f86941ee150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57628/0 (socket says 172.21.15.81:57628) 2024-05-08T23:31:24.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f8699cae700 1 -- 172.21.15.81:0/1611221286 learned_addr learned my addr 172.21.15.81:0/1611221286 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:24.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f869acb0700 1 --2- 172.21.15.81:0/1611221286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86941594d0 0x7f86941ee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f869a4af700 1 --2- 172.21.15.81:0/1611221286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f869406d150 0x7f8694157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f8699cae700 1 -- 172.21.15.81:0/1611221286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86941594d0 msgr2=0x7f86941ee690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f8699cae700 1 --2- 172.21.15.81:0/1611221286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86941594d0 0x7f86941ee690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f8699cae700 1 -- 172.21.15.81:0/1611221286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f869406d150 msgr2=0x7f8694157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:24.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f8699cae700 1 --2- 172.21.15.81:0/1611221286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f869406d150 0x7f8694157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:24.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f8699cae700 1 -- 172.21.15.81:0/1611221286 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840094b0 con 0x7f8694071a70 2024-05-08T23:31:24.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f869a4af700 1 --2- 172.21.15.81:0/1611221286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f869406d150 0x7f8694157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:24.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f8699cae700 1 --2- 172.21.15.81:0/1611221286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8694071a70 0x7f86941ee150 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8684014040 tx=0x7f8684004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:24.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f8692ffd700 1 -- 172.21.15.81:0/1611221286 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684011630 con 0x7f8694071a70 2024-05-08T23:31:24.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f8692ffd700 1 -- 172.21.15.81:0/1611221286 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86840117d0 con 0x7f8694071a70 2024-05-08T23:31:24.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f8692ffd700 1 -- 172.21.15.81:0/1611221286 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684011b80 con 0x7f8694071a70 2024-05-08T23:31:24.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f869b4b1700 1 -- 172.21.15.81:0/1611221286 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86940ce190 con 0x7f8694071a70 2024-05-08T23:31:24.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f869b4b1700 1 -- 172.21.15.81:0/1611221286 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8694156a40 con 0x7f8694071a70 2024-05-08T23:31:24.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f869b4b1700 1 -- 172.21.15.81:0/1611221286 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86940684b0 con 0x7f8694071a70 2024-05-08T23:31:24.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.828+0000 7f8692ffd700 1 -- 172.21.15.81:0/1611221286 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8684018050 con 0x7f8694071a70 2024-05-08T23:31:24.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.832+0000 7f8692ffd700 1 --2- 172.21.15.81:0/1611221286 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8680066730 0x7f8680068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:24.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.832+0000 7f8692ffd700 1 -- 172.21.15.81:0/1611221286 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8684084da0 con 0x7f8694071a70 2024-05-08T23:31:24.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.832+0000 7f869a4af700 1 --2- 172.21.15.81:0/1611221286 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8680066730 0x7f8680068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:24.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.832+0000 7f869a4af700 1 --2- 172.21.15.81:0/1611221286 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8680066730 0x7f8680068bf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8688001700 tx=0x7f8688009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:24.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:24.836+0000 7f8692ffd700 1 -- 172.21.15.81:0/1611221286 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8684052240 con 0x7f8694071a70 2024-05-08T23:31:25.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.004+0000 7f869b4b1700 1 -- 172.21.15.81:0/1611221286 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f869415ce30 con 0x7f8694071a70 2024-05-08T23:31:25.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.008+0000 7f8692ffd700 1 -- 172.21.15.81:0/1611221286 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8684016020 con 0x7f8694071a70 2024-05-08T23:31:25.012 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:25.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.008+0000 7f869b4b1700 1 -- 172.21.15.81:0/1611221286 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8680066730 msgr2=0x7f8680068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.008+0000 7f869b4b1700 1 --2- 172.21.15.81:0/1611221286 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8680066730 0x7f8680068bf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8688001700 tx=0x7f8688009040 comp rx=0 tx=0).stop 2024-05-08T23:31:25.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.008+0000 7f869b4b1700 1 -- 172.21.15.81:0/1611221286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8694071a70 msgr2=0x7f86941ee150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.008+0000 7f869b4b1700 1 --2- 172.21.15.81:0/1611221286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8694071a70 0x7f86941ee150 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8684014040 tx=0x7f8684004770 comp rx=0 tx=0).stop 2024-05-08T23:31:25.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.008+0000 7f869b4b1700 1 -- 172.21.15.81:0/1611221286 shutdown_connections 2024-05-08T23:31:25.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.008+0000 7f869b4b1700 1 --2- 172.21.15.81:0/1611221286 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8680066730 0x7f8680068bf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.008+0000 7f869b4b1700 1 --2- 172.21.15.81:0/1611221286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86941594d0 0x7f86941ee690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.008+0000 7f869b4b1700 1 --2- 172.21.15.81:0/1611221286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8694071a70 0x7f86941ee150 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.008+0000 7f869b4b1700 1 --2- 172.21.15.81:0/1611221286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f869406d150 0x7f8694157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.008+0000 7f869b4b1700 1 -- 172.21.15.81:0/1611221286 >> 172.21.15.81:0/1611221286 conn(0x7f86940b0de0 msgr2=0x7f8694150580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:25.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.008+0000 7f869b4b1700 1 -- 172.21.15.81:0/1611221286 shutdown_connections 2024-05-08T23:31:25.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.008+0000 7f869b4b1700 1 -- 172.21.15.81:0/1611221286 wait complete. 2024-05-08T23:31:25.029 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:25.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafdeec1700 1 Processor -- start 2024-05-08T23:31:25.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafdeec1700 1 -- start start 2024-05-08T23:31:25.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafdeec1700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd815d2f0 0x7fafd80ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafdeec1700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fafd80cf8b0 0x7fafd80cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafdeec1700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd80ceb40 con 0x7fafd80b3550 2024-05-08T23:31:25.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafdeec1700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd80d3a10 con 0x7fafd80cf8b0 2024-05-08T23:31:25.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafdeec1700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd815d6d0 con 0x7fafd815d2f0 2024-05-08T23:31:25.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafddebf700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd815d2f0 0x7fafd80ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafdd6be700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fafd80cf8b0 0x7fafd80cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafddebf700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd815d2f0 0x7fafd80ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:38502/0 (socket says 172.21.15.81:38502) 2024-05-08T23:31:25.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafddebf700 1 -- 172.21.15.81:0/3998196131 learned_addr learned my addr 172.21.15.81:0/3998196131 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafdd6be700 1 -- 172.21.15.81:0/3998196131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd815d2f0 msgr2=0x7fafd80ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafdd6be700 1 --2- 172.21.15.81:0/3998196131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd815d2f0 0x7fafd80ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafdd6be700 1 -- 172.21.15.81:0/3998196131 >> v1:172.21.15.117:6789/0 conn(0x7fafd80b3550 legacy=0x7fafd80b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafdd6be700 1 -- 172.21.15.81:0/3998196131 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd80d0250 con 0x7fafd80cf8b0 2024-05-08T23:31:25.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.144+0000 7fafddebf700 1 --2- 172.21.15.81:0/3998196131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd815d2f0 0x7fafd80ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:25.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.148+0000 7fafdd6be700 1 --2- 172.21.15.81:0/3998196131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80cf8b0 0x7fafd80cfd10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009850 tx=0x7fafcc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=756fe4ba34654e1e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:25.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.148+0000 7fafdcebd700 1 -- 172.21.15.81:0/3998196131 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc011830 con 0x7fafd80cf8b0 2024-05-08T23:31:25.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.148+0000 7fafdcebd700 1 -- 172.21.15.81:0/3998196131 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc011990 con 0x7fafd80cf8b0 2024-05-08T23:31:25.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.148+0000 7fafdcebd700 1 -- 172.21.15.81:0/3998196131 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc011d40 con 0x7fafd80cf8b0 2024-05-08T23:31:25.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.148+0000 7fafdeec1700 1 -- 172.21.15.81:0/3998196131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80cf8b0 msgr2=0x7fafd80cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.148+0000 7fafdeec1700 1 --2- 172.21.15.81:0/3998196131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80cf8b0 0x7fafd80cfd10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009850 tx=0x7fafcc009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:25.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.148+0000 7fafdeec1700 1 -- 172.21.15.81:0/3998196131 shutdown_connections 2024-05-08T23:31:25.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.148+0000 7fafdeec1700 1 --2- 172.21.15.81:0/3998196131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80cf8b0 0x7fafd80cfd10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.148+0000 7fafdeec1700 1 --2- 172.21.15.81:0/3998196131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd815d2f0 0x7fafd80ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.148+0000 7fafdeec1700 1 -- 172.21.15.81:0/3998196131 >> 172.21.15.81:0/3998196131 conn(0x7fafd809c320 msgr2=0x7fafd80d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:25.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.148+0000 7fafdeec1700 1 -- 172.21.15.81:0/3998196131 shutdown_connections 2024-05-08T23:31:25.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.148+0000 7fafdeec1700 1 -- 172.21.15.81:0/3998196131 wait complete. 2024-05-08T23:31:25.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.148+0000 7fafdeec1700 1 Processor -- start 2024-05-08T23:31:25.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.148+0000 7fafdeec1700 1 -- start start 2024-05-08T23:31:25.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafdeec1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80b3550 0x7fafd80b8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafdeec1700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd80cf8b0 0x7fafd80c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafdeec1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fafd815d2f0 0x7fafd8148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafdeec1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd80cb230 con 0x7fafd815d2f0 2024-05-08T23:31:25.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafdeec1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd80c9510 con 0x7fafd80b3550 2024-05-08T23:31:25.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafdeec1700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd80c97f0 con 0x7fafd80cf8b0 2024-05-08T23:31:25.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafde6c0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fafd815d2f0 0x7fafd8148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafdd6be700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd80cf8b0 0x7fafd80c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafde6c0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fafd815d2f0 0x7fafd8148610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57634/0 (socket says 172.21.15.81:57634) 2024-05-08T23:31:25.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafde6c0700 1 -- 172.21.15.81:0/921507663 learned_addr learned my addr 172.21.15.81:0/921507663 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:25.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafddebf700 1 --2- 172.21.15.81:0/921507663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80b3550 0x7fafd80b8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafde6c0700 1 -- 172.21.15.81:0/921507663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd80cf8b0 msgr2=0x7fafd80c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafde6c0700 1 --2- 172.21.15.81:0/921507663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd80cf8b0 0x7fafd80c6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafde6c0700 1 -- 172.21.15.81:0/921507663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80b3550 msgr2=0x7fafd80b8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafde6c0700 1 --2- 172.21.15.81:0/921507663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80b3550 0x7fafd80b8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafde6c0700 1 -- 172.21.15.81:0/921507663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd0009b30 con 0x7fafd815d2f0 2024-05-08T23:31:25.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafdd6be700 1 --2- 172.21.15.81:0/921507663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd80cf8b0 0x7fafd80c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:25.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafde6c0700 1 --2- 172.21.15.81:0/921507663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fafd815d2f0 0x7fafd8148610 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fafc800c430 tx=0x7fafc800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:25.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafd67fc700 1 -- 172.21.15.81:0/921507663 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc80052d0 con 0x7fafd815d2f0 2024-05-08T23:31:25.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafd67fc700 1 -- 172.21.15.81:0/921507663 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc8005430 con 0x7fafd815d2f0 2024-05-08T23:31:25.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafdeec1700 1 -- 172.21.15.81:0/921507663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc0094b0 con 0x7fafd815d2f0 2024-05-08T23:31:25.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafd67fc700 1 -- 172.21.15.81:0/921507663 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc80055a0 con 0x7fafd815d2f0 2024-05-08T23:31:25.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.152+0000 7fafdeec1700 1 -- 172.21.15.81:0/921507663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd80c9db0 con 0x7fafd815d2f0 2024-05-08T23:31:25.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.156+0000 7fafd67fc700 1 -- 172.21.15.81:0/921507663 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fafc80038e0 con 0x7fafd815d2f0 2024-05-08T23:31:25.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.156+0000 7fafdeec1700 1 -- 172.21.15.81:0/921507663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd80cb3a0 con 0x7fafd815d2f0 2024-05-08T23:31:25.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.156+0000 7fafd67fc700 1 --2- 172.21.15.81:0/921507663 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fafc4066760 0x7fafc4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.156+0000 7fafd67fc700 1 -- 172.21.15.81:0/921507663 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fafc8083460 con 0x7fafd815d2f0 2024-05-08T23:31:25.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.156+0000 7fafddebf700 1 --2- 172.21.15.81:0/921507663 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fafc4066760 0x7fafc4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.156+0000 7fafddebf700 1 --2- 172.21.15.81:0/921507663 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fafc4066760 0x7fafc4068c20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fafd00072c0 tx=0x7fafd000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:25.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.164+0000 7fafd67fc700 1 -- 172.21.15.81:0/921507663 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fafc8050760 con 0x7fafd815d2f0 2024-05-08T23:31:25.339 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.332+0000 7fafdeec1700 1 -- 172.21.15.81:0/921507663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fafd80684b0 con 0x7fafd815d2f0 2024-05-08T23:31:25.340 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.336+0000 7fafd67fc700 1 -- 172.21.15.81:0/921507663 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fafc805a070 con 0x7fafd815d2f0 2024-05-08T23:31:25.340 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:25.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.336+0000 7fafdeec1700 1 -- 172.21.15.81:0/921507663 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fafc4066760 msgr2=0x7fafc4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.336+0000 7fafdeec1700 1 --2- 172.21.15.81:0/921507663 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fafc4066760 0x7fafc4068c20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fafd00072c0 tx=0x7fafd000b040 comp rx=0 tx=0).stop 2024-05-08T23:31:25.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.336+0000 7fafdeec1700 1 -- 172.21.15.81:0/921507663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fafd815d2f0 msgr2=0x7fafd8148610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.336+0000 7fafdeec1700 1 --2- 172.21.15.81:0/921507663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fafd815d2f0 0x7fafd8148610 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fafc800c430 tx=0x7fafc800c740 comp rx=0 tx=0).stop 2024-05-08T23:31:25.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.336+0000 7fafdeec1700 1 -- 172.21.15.81:0/921507663 shutdown_connections 2024-05-08T23:31:25.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.336+0000 7fafdeec1700 1 --2- 172.21.15.81:0/921507663 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fafc4066760 0x7fafc4068c20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.336+0000 7fafdeec1700 1 --2- 172.21.15.81:0/921507663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fafd815d2f0 0x7fafd8148610 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.336+0000 7fafdeec1700 1 --2- 172.21.15.81:0/921507663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd80cf8b0 0x7fafd80c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.336+0000 7fafdeec1700 1 --2- 172.21.15.81:0/921507663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80b3550 0x7fafd80b8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.336+0000 7fafdeec1700 1 -- 172.21.15.81:0/921507663 >> 172.21.15.81:0/921507663 conn(0x7fafd809c320 msgr2=0x7fafd80b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:25.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.336+0000 7fafdeec1700 1 -- 172.21.15.81:0/921507663 shutdown_connections 2024-05-08T23:31:25.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.336+0000 7fafdeec1700 1 -- 172.21.15.81:0/921507663 wait complete. 2024-05-08T23:31:25.355 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:31:25.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.460+0000 7f5230d52700 1 Processor -- start 2024-05-08T23:31:25.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.460+0000 7f5230d52700 1 -- start start 2024-05-08T23:31:25.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.460+0000 7f5230d52700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f522c15ac70 0x7f522c13d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.460+0000 7f5230d52700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522c0ce080 con 0x7f522c071ab0 2024-05-08T23:31:25.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.460+0000 7f5230d52700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522c0cd7c0 con 0x7f522c0ce510 2024-05-08T23:31:25.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.460+0000 7f5230d52700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f522c0cda30 con 0x7f522c15ac70 2024-05-08T23:31:25.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f522b7fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f522c071ab0 0x7f522c071e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34570/0 (socket says 172.21.15.81:34570) 2024-05-08T23:31:25.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f522b7fe700 1 -- 172.21.15.81:0/1083926047 learned_addr learned my addr 172.21.15.81:0/1083926047 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:25.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f522bfff700 1 --2- 172.21.15.81:0/1083926047 >> v2:172.21.15.117:3300/0 conn(0x7f522c15ac70 0x7f522c13d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f522affd700 1 -- 172.21.15.81:0/1083926047 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416360340 0 0) 0x7f522c0ce080 con 0x7f522c071ab0 2024-05-08T23:31:25.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f522affd700 1 -- 172.21.15.81:0/1083926047 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52180035d0 con 0x7f522c071ab0 2024-05-08T23:31:25.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f522bfff700 1 -- 172.21.15.81:0/1083926047 >> v1:172.21.15.117:6789/0 conn(0x7f522c0ce510 legacy=0x7f522c15a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f522bfff700 1 -- 172.21.15.81:0/1083926047 >> v1:172.21.15.81:6789/0 conn(0x7f522c071ab0 legacy=0x7f522c071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f522bfff700 1 -- 172.21.15.81:0/1083926047 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c0cdd10 con 0x7f522c15ac70 2024-05-08T23:31:25.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f522bfff700 1 --2- 172.21.15.81:0/1083926047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f522c15ac70 0x7f522c13d500 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5224009d90 tx=0x7f522400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2793c1100c81aa85 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:25.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f522affd700 1 -- 172.21.15.81:0/1083926047 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522400bd20 con 0x7f522c15ac70 2024-05-08T23:31:25.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f522affd700 1 -- 172.21.15.81:0/1083926047 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522400be80 con 0x7f522c15ac70 2024-05-08T23:31:25.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f522affd700 1 -- 172.21.15.81:0/1083926047 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52240116a0 con 0x7f522c15ac70 2024-05-08T23:31:25.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f5230d52700 1 -- 172.21.15.81:0/1083926047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f522c15ac70 msgr2=0x7f522c13d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f5230d52700 1 --2- 172.21.15.81:0/1083926047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f522c15ac70 0x7f522c13d500 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5224009d90 tx=0x7f522400b330 comp rx=0 tx=0).stop 2024-05-08T23:31:25.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f5230d52700 1 -- 172.21.15.81:0/1083926047 shutdown_connections 2024-05-08T23:31:25.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f5230d52700 1 --2- 172.21.15.81:0/1083926047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f522c15ac70 0x7f522c13d500 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f5230d52700 1 -- 172.21.15.81:0/1083926047 >> 172.21.15.81:0/1083926047 conn(0x7f522c06c330 msgr2=0x7f522c06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:25.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f5230d52700 1 -- 172.21.15.81:0/1083926047 shutdown_connections 2024-05-08T23:31:25.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.464+0000 7f5230d52700 1 -- 172.21.15.81:0/1083926047 wait complete. 2024-05-08T23:31:25.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5230d52700 1 Processor -- start 2024-05-08T23:31:25.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5230d52700 1 -- start start 2024-05-08T23:31:25.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5230d52700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f522c071ab0 0x7f522c0b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5230d52700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f522c0ce510 0x7f522c13c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5230d52700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f522c15ac70 0x7f522c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5230d52700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0b39a0 con 0x7f522c0ce510 2024-05-08T23:31:25.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5230d52700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c13ccb0 con 0x7f522c071ab0 2024-05-08T23:31:25.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5230d52700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0b23c0 con 0x7f522c15ac70 2024-05-08T23:31:25.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5222ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f522c0ce510 0x7f522c13c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5222ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f522c0ce510 0x7f522c13c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57646/0 (socket says 172.21.15.81:57646) 2024-05-08T23:31:25.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5222ffd700 1 -- 172.21.15.81:0/2333275034 learned_addr learned my addr 172.21.15.81:0/2333275034 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:25.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f522bfff700 1 --2- 172.21.15.81:0/2333275034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f522c15ac70 0x7f522c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f522b7fe700 1 --2- 172.21.15.81:0/2333275034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f522c071ab0 0x7f522c0b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5222ffd700 1 -- 172.21.15.81:0/2333275034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f522c15ac70 msgr2=0x7f522c0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5222ffd700 1 --2- 172.21.15.81:0/2333275034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f522c15ac70 0x7f522c0b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5222ffd700 1 -- 172.21.15.81:0/2333275034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f522c071ab0 msgr2=0x7f522c0b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5222ffd700 1 --2- 172.21.15.81:0/2333275034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f522c071ab0 0x7f522c0b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5222ffd700 1 -- 172.21.15.81:0/2333275034 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5224009a70 con 0x7f522c0ce510 2024-05-08T23:31:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5222ffd700 1 --2- 172.21.15.81:0/2333275034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f522c0ce510 0x7f522c13c770 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f521400c500 tx=0x7f521400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5228ff9700 1 -- 172.21.15.81:0/2333275034 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52140052d0 con 0x7f522c0ce510 2024-05-08T23:31:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5228ff9700 1 -- 172.21.15.81:0/2333275034 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5214005430 con 0x7f522c0ce510 2024-05-08T23:31:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5230d52700 1 -- 172.21.15.81:0/2333275034 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522c0b26a0 con 0x7f522c0ce510 2024-05-08T23:31:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5228ff9700 1 -- 172.21.15.81:0/2333275034 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52140055a0 con 0x7f522c0ce510 2024-05-08T23:31:25.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.468+0000 7f5230d52700 1 -- 172.21.15.81:0/2333275034 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522c0b2c60 con 0x7f522c0ce510 2024-05-08T23:31:25.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.472+0000 7f5230d52700 1 -- 172.21.15.81:0/2333275034 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522c0684b0 con 0x7f522c0ce510 2024-05-08T23:31:25.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.472+0000 7f5228ff9700 1 -- 172.21.15.81:0/2333275034 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f52140038e0 con 0x7f522c0ce510 2024-05-08T23:31:25.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.476+0000 7f5228ff9700 1 --2- 172.21.15.81:0/2333275034 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f520c066730 0x7f520c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.476+0000 7f522b7fe700 1 --2- 172.21.15.81:0/2333275034 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f520c066730 0x7f520c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.476+0000 7f5228ff9700 1 -- 172.21.15.81:0/2333275034 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5214083580 con 0x7f522c0ce510 2024-05-08T23:31:25.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.476+0000 7f522b7fe700 1 --2- 172.21.15.81:0/2333275034 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f520c066730 0x7f520c068bf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f521c007fd0 tx=0x7f521c007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:25.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.484+0000 7f5228ff9700 1 -- 172.21.15.81:0/2333275034 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5214050870 con 0x7f522c0ce510 2024-05-08T23:31:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.692+0000 7f5230d52700 1 -- 172.21.15.81:0/2333275034 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f522c070110 con 0x7f522c0ce510 2024-05-08T23:31:25.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.692+0000 7f5228ff9700 1 -- 172.21.15.81:0/2333275034 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7f5214053e90 con 0x7f522c0ce510 2024-05-08T23:31:25.700 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:25.700 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1475726755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1475726755},{"type":"v1","addr":"172.21.15.81:6835","nonce":1475726755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3535820540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3535820540},{"type":"v1","addr":"172.21.15.81:6837","nonce":3535820540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1531019955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1531019955},{"type":"v1","addr":"172.21.15.117:6833","nonce":1531019955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:30:38.665368+0000","modified":"2024-05-08T23:30:41.672722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/491911303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":491911303},{"type":"v1","addr":"172.21.15.117:6835","nonce":491911303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:31:25.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.696+0000 7f5230d52700 1 -- 172.21.15.81:0/2333275034 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f520c066730 msgr2=0x7f520c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.696+0000 7f5230d52700 1 --2- 172.21.15.81:0/2333275034 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f520c066730 0x7f520c068bf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f521c007fd0 tx=0x7f521c007de0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.696+0000 7f5230d52700 1 -- 172.21.15.81:0/2333275034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f522c0ce510 msgr2=0x7f522c13c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.696+0000 7f5230d52700 1 --2- 172.21.15.81:0/2333275034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f522c0ce510 0x7f522c13c770 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f521400c500 tx=0x7f521400c810 comp rx=0 tx=0).stop 2024-05-08T23:31:25.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.696+0000 7f5230d52700 1 -- 172.21.15.81:0/2333275034 shutdown_connections 2024-05-08T23:31:25.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.696+0000 7f5230d52700 1 --2- 172.21.15.81:0/2333275034 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f520c066730 0x7f520c068bf0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.696+0000 7f5230d52700 1 --2- 172.21.15.81:0/2333275034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f522c15ac70 0x7f522c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.696+0000 7f5230d52700 1 --2- 172.21.15.81:0/2333275034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f522c0ce510 0x7f522c13c770 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.696+0000 7f5230d52700 1 --2- 172.21.15.81:0/2333275034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f522c071ab0 0x7f522c0b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.696+0000 7f5230d52700 1 -- 172.21.15.81:0/2333275034 >> 172.21.15.81:0/2333275034 conn(0x7f522c06c330 msgr2=0x7f522c15cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:25.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.696+0000 7f5230d52700 1 -- 172.21.15.81:0/2333275034 shutdown_connections 2024-05-08T23:31:25.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.696+0000 7f5230d52700 1 -- 172.21.15.81:0/2333275034 wait complete. 2024-05-08T23:31:25.703 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 7 2024-05-08T23:31:25.714 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:25.827 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.820+0000 7f54f439c700 1 Processor -- start 2024-05-08T23:31:25.827 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.820+0000 7f54f439c700 1 -- start start 2024-05-08T23:31:25.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.820+0000 7f54f439c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec06ca60 0x7f54ec15a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.820+0000 7f54f439c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f54ec13d180 0x7f54ec15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.820+0000 7f54f439c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ec15b3d0 con 0x7f54ec13cda0 2024-05-08T23:31:25.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.820+0000 7f54f439c700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ec15b6a0 con 0x7f54ec13d180 2024-05-08T23:31:25.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.820+0000 7f54f439c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec0b3380 con 0x7f54ec06ca60 2024-05-08T23:31:25.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f2939700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f54ec13d180 0x7f54ec15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f2939700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f54ec13d180 0x7f54ec15ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57652/0 (socket says 172.21.15.81:57652) 2024-05-08T23:31:25.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f2939700 1 -- 172.21.15.81:0/3569767 learned_addr learned my addr 172.21.15.81:0/3569767 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:25.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f1937700 1 --2- 172.21.15.81:0/3569767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec06ca60 0x7f54ec15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f2939700 1 -- 172.21.15.81:0/3569767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec06ca60 msgr2=0x7f54ec15a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:25.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f2939700 1 --2- 172.21.15.81:0/3569767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec06ca60 0x7f54ec15a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f2939700 1 -- 172.21.15.81:0/3569767 >> v1:172.21.15.117:6789/0 conn(0x7f54ec13cda0 legacy=0x7f54ec06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f2939700 1 -- 172.21.15.81:0/3569767 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec0b3660 con 0x7f54ec13d180 2024-05-08T23:31:25.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f2939700 1 --2- 172.21.15.81:0/3569767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54ec13d180 0x7f54ec15ae90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f54e400bbb0 tx=0x7f54e400bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a5fc50461cd6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:25.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f1136700 1 -- 172.21.15.81:0/3569767 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e400dd30 con 0x7f54ec13d180 2024-05-08T23:31:25.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f1136700 1 -- 172.21.15.81:0/3569767 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e400de90 con 0x7f54ec13d180 2024-05-08T23:31:25.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f1136700 1 -- 172.21.15.81:0/3569767 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4010690 con 0x7f54ec13d180 2024-05-08T23:31:25.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f439c700 1 -- 172.21.15.81:0/3569767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54ec13d180 msgr2=0x7f54ec15ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f439c700 1 --2- 172.21.15.81:0/3569767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54ec13d180 0x7f54ec15ae90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f54e400bbb0 tx=0x7f54e400bec0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f439c700 1 -- 172.21.15.81:0/3569767 shutdown_connections 2024-05-08T23:31:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f439c700 1 --2- 172.21.15.81:0/3569767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54ec13d180 0x7f54ec15ae90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f439c700 1 --2- 172.21.15.81:0/3569767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec06ca60 0x7f54ec15a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f439c700 1 -- 172.21.15.81:0/3569767 >> 172.21.15.81:0/3569767 conn(0x7f54ec06a360 msgr2=0x7f54ec06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f439c700 1 -- 172.21.15.81:0/3569767 shutdown_connections 2024-05-08T23:31:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.824+0000 7f54f439c700 1 -- 172.21.15.81:0/3569767 wait complete. 2024-05-08T23:31:25.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.828+0000 7f54f439c700 1 Processor -- start 2024-05-08T23:31:25.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.828+0000 7f54f439c700 1 -- start start 2024-05-08T23:31:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.828+0000 7f54f439c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54ec06ca60 0x7f54ec0ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.828+0000 7f54f439c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54ec13cda0 0x7f54ec1f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f439c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec13d180 0x7f54ec0cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f439c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec0ad7d0 con 0x7f54ec13cda0 2024-05-08T23:31:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f439c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec0ade90 con 0x7f54ec06ca60 2024-05-08T23:31:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f439c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec1f6570 con 0x7f54ec13d180 2024-05-08T23:31:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f1937700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54ec13cda0 0x7f54ec1f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f1937700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54ec13cda0 0x7f54ec1f6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57658/0 (socket says 172.21.15.81:57658) 2024-05-08T23:31:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f1937700 1 -- 172.21.15.81:0/4245190873 learned_addr learned my addr 172.21.15.81:0/4245190873 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f2138700 1 --2- 172.21.15.81:0/4245190873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54ec06ca60 0x7f54ec0ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f2939700 1 --2- 172.21.15.81:0/4245190873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec13d180 0x7f54ec0cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f1937700 1 -- 172.21.15.81:0/4245190873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec13d180 msgr2=0x7f54ec0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f1937700 1 --2- 172.21.15.81:0/4245190873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec13d180 0x7f54ec0cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f1937700 1 -- 172.21.15.81:0/4245190873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54ec06ca60 msgr2=0x7f54ec0ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f1937700 1 --2- 172.21.15.81:0/4245190873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54ec06ca60 0x7f54ec0ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f1937700 1 -- 172.21.15.81:0/4245190873 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e400b810 con 0x7f54ec13cda0 2024-05-08T23:31:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f1937700 1 --2- 172.21.15.81:0/4245190873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54ec13cda0 0x7f54ec1f6030 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009db0 tx=0x7f54e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54e2ffd700 1 -- 172.21.15.81:0/4245190873 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8009c20 con 0x7f54ec13cda0 2024-05-08T23:31:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54e2ffd700 1 -- 172.21.15.81:0/4245190873 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e800f040 con 0x7f54ec13cda0 2024-05-08T23:31:25.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f439c700 1 -- 172.21.15.81:0/4245190873 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ec14c2a0 con 0x7f54ec13cda0 2024-05-08T23:31:25.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54e2ffd700 1 -- 172.21.15.81:0/4245190873 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e800ced0 con 0x7f54ec13cda0 2024-05-08T23:31:25.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f439c700 1 -- 172.21.15.81:0/4245190873 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ec1f6fd0 con 0x7f54ec13cda0 2024-05-08T23:31:25.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.832+0000 7f54f439c700 1 -- 172.21.15.81:0/4245190873 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ec0b31a0 con 0x7f54ec13cda0 2024-05-08T23:31:25.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.836+0000 7f54e2ffd700 1 -- 172.21.15.81:0/4245190873 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f54e8014480 con 0x7f54ec13cda0 2024-05-08T23:31:25.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.836+0000 7f54e2ffd700 1 --2- 172.21.15.81:0/4245190873 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f54d0066760 0x7f54d0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:25.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.836+0000 7f54e2ffd700 1 -- 172.21.15.81:0/4245190873 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f54e8084c00 con 0x7f54ec13cda0 2024-05-08T23:31:25.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.836+0000 7f54f2138700 1 --2- 172.21.15.81:0/4245190873 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f54d0066760 0x7f54d0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:25.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.836+0000 7f54f2138700 1 --2- 172.21.15.81:0/4245190873 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f54d0066760 0x7f54d0068c20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f54dc000f60 tx=0x7f54dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:25.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:25.840+0000 7f54e2ffd700 1 -- 172.21.15.81:0/4245190873 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f54e808b080 con 0x7f54ec13cda0 2024-05-08T23:31:26.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.008+0000 7f54f439c700 1 -- 172.21.15.81:0/4245190873 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54ec0684b0 con 0x7f54ec13cda0 2024-05-08T23:31:26.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.012+0000 7f54e2ffd700 1 -- 172.21.15.81:0/4245190873 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f54e8051f80 con 0x7f54ec13cda0 2024-05-08T23:31:26.018 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:26.018 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":24,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:20.851517+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:26.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.016+0000 7f54f439c700 1 -- 172.21.15.81:0/4245190873 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f54d0066760 msgr2=0x7f54d0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.016+0000 7f54f439c700 1 --2- 172.21.15.81:0/4245190873 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f54d0066760 0x7f54d0068c20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f54dc000f60 tx=0x7f54dc009040 comp rx=0 tx=0).stop 2024-05-08T23:31:26.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.016+0000 7f54f439c700 1 -- 172.21.15.81:0/4245190873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54ec13cda0 msgr2=0x7f54ec1f6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.016+0000 7f54f439c700 1 --2- 172.21.15.81:0/4245190873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54ec13cda0 0x7f54ec1f6030 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009db0 tx=0x7f54e800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.016+0000 7f54f439c700 1 -- 172.21.15.81:0/4245190873 shutdown_connections 2024-05-08T23:31:26.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.016+0000 7f54f439c700 1 --2- 172.21.15.81:0/4245190873 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f54d0066760 0x7f54d0068c20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.016+0000 7f54f439c700 1 --2- 172.21.15.81:0/4245190873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec13d180 0x7f54ec0cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.016+0000 7f54f439c700 1 --2- 172.21.15.81:0/4245190873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54ec13cda0 0x7f54ec1f6030 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.016+0000 7f54f439c700 1 --2- 172.21.15.81:0/4245190873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54ec06ca60 0x7f54ec0ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.016+0000 7f54f439c700 1 -- 172.21.15.81:0/4245190873 >> 172.21.15.81:0/4245190873 conn(0x7f54ec06a360 msgr2=0x7f54ec14e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:26.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.016+0000 7f54f439c700 1 -- 172.21.15.81:0/4245190873 shutdown_connections 2024-05-08T23:31:26.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.016+0000 7f54f439c700 1 -- 172.21.15.81:0/4245190873 wait complete. 2024-05-08T23:31:26.033 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T23:31:26.033 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:31:26.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.140+0000 7fc25bfff700 1 Processor -- start 2024-05-08T23:31:26.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.140+0000 7fc25bfff700 1 -- start start 2024-05-08T23:31:26.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.140+0000 7fc25bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc25c071e70 0x7fc25c1624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.140+0000 7fc25bfff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc25c158960 con 0x7fc25c0b3c20 2024-05-08T23:31:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.140+0000 7fc25bfff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc25c0ae990 con 0x7fc25c070110 2024-05-08T23:31:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.140+0000 7fc25bfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25c0b1f60 con 0x7fc25c071e70 2024-05-08T23:31:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.140+0000 7fc25affd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc25c0b3c20 0x7fc25c0b4000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:32794/0 (socket says 172.21.15.81:32794) 2024-05-08T23:31:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.140+0000 7fc25affd700 1 -- 172.21.15.81:0/2954859914 learned_addr learned my addr 172.21.15.81:0/2954859914 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25a7fc700 1 --2- 172.21.15.81:0/2954859914 >> v2:172.21.15.117:3300/0 conn(0x7fc25c071e70 0x7fc25c1624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825418147 0 0) 0x7fc25c158960 con 0x7fc25c0b3c20 2024-05-08T23:31:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2480035d0 con 0x7fc25c0b3c20 2024-05-08T23:31:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009219529 0 0) 0x7fc25c0ae990 con 0x7fc25c070110 2024-05-08T23:31:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc25c158960 con 0x7fc25c070110 2024-05-08T23:31:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2819597338 0 0) 0x7fc2480035d0 con 0x7fc25c0b3c20 2024-05-08T23:31:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 >> v2:172.21.15.117:3300/0 conn(0x7fc25c071e70 msgr2=0x7fc25c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 --2- 172.21.15.81:0/2954859914 >> v2:172.21.15.117:3300/0 conn(0x7fc25c071e70 0x7fc25c1624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 >> v1:172.21.15.117:6789/0 conn(0x7fc25c070110 legacy=0x7fc25c0705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c0b2170 con 0x7fc25c0b3c20 2024-05-08T23:31:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7fc2540019f0 con 0x7fc25c0b3c20 2024-05-08T23:31:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 >> v1:172.21.15.81:6789/0 conn(0x7fc25c0b3c20 legacy=0x7fc25c0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 --2- 172.21.15.81:0/2954859914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc248003c00 0x7fc248004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 --2- 172.21.15.81:0/2954859914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc248004870 0x7fc248004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 --2- 172.21.15.81:0/2954859914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc248005280 0x7fc24800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c0b1f60 con 0x7fc248003c00 2024-05-08T23:31:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24800c1b0 con 0x7fc248005280 2024-05-08T23:31:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24800c370 con 0x7fc248004870 2024-05-08T23:31:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25affd700 1 --2- 172.21.15.81:0/2954859914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc248003c00 0x7fc248004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25b7fe700 1 --2- 172.21.15.81:0/2954859914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc248005280 0x7fc24800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25a7fc700 1 --2- 172.21.15.81:0/2954859914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc248004870 0x7fc248004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25affd700 1 -- 172.21.15.81:0/2954859914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc248004870 msgr2=0x7fc248004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25affd700 1 --2- 172.21.15.81:0/2954859914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc248004870 0x7fc248004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25affd700 1 -- 172.21.15.81:0/2954859914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc248005280 msgr2=0x7fc24800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25affd700 1 --2- 172.21.15.81:0/2954859914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc248005280 0x7fc24800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25affd700 1 -- 172.21.15.81:0/2954859914 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2480035d0 con 0x7fc248003c00 2024-05-08T23:31:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25b7fe700 1 --2- 172.21.15.81:0/2954859914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc248005280 0x7fc24800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25b7fe700 1 -- 172.21.15.81:0/2954859914 reap_dead start 2024-05-08T23:31:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25a7fc700 1 --2- 172.21.15.81:0/2954859914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc248004870 0x7fc248004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25affd700 1 --2- 172.21.15.81:0/2954859914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc248003c00 0x7fc248004000 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc25400a700 tx=0x7fc2480058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb71363f9da8856 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2540070d0 con 0x7fc248003c00 2024-05-08T23:31:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc254007230 con 0x7fc248003c00 2024-05-08T23:31:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc259ffb700 1 -- 172.21.15.81:0/2954859914 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2540075e0 con 0x7fc248003c00 2024-05-08T23:31:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25bfff700 1 -- 172.21.15.81:0/2954859914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc248003c00 msgr2=0x7fc248004000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25bfff700 1 --2- 172.21.15.81:0/2954859914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc248003c00 0x7fc248004000 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc25400a700 tx=0x7fc2480058a0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25bfff700 1 -- 172.21.15.81:0/2954859914 shutdown_connections 2024-05-08T23:31:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25bfff700 1 --2- 172.21.15.81:0/2954859914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc248003c00 0x7fc248004000 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25bfff700 1 -- 172.21.15.81:0/2954859914 >> 172.21.15.81:0/2954859914 conn(0x7fc25c06c330 msgr2=0x7fc25c06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25bfff700 1 -- 172.21.15.81:0/2954859914 shutdown_connections 2024-05-08T23:31:26.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.144+0000 7fc25bfff700 1 -- 172.21.15.81:0/2954859914 wait complete. 2024-05-08T23:31:26.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25bfff700 1 Processor -- start 2024-05-08T23:31:26.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25bfff700 1 -- start start 2024-05-08T23:31:26.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc248003c00 0x7fc25c1fa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25bfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc25c1fbd40 0x7fc25c0af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc25c0af8e0 0x7fc25c13f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25bfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c164590 con 0x7fc25c0af8e0 2024-05-08T23:31:26.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25bfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c0afd60 con 0x7fc248003c00 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25bfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c1fb8b0 con 0x7fc25c1fbd40 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc25c0af8e0 0x7fc25c13f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc25c0af8e0 0x7fc25c13f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42550/0 (socket says 172.21.15.81:42550) 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25b7fe700 1 -- 172.21.15.81:0/3968437105 learned_addr learned my addr 172.21.15.81:0/3968437105 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25a7fc700 1 --2- 172.21.15.81:0/3968437105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc25c1fbd40 0x7fc25c0af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25affd700 1 --2- 172.21.15.81:0/3968437105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc248003c00 0x7fc25c1fa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25a7fc700 1 -- 172.21.15.81:0/3968437105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc248003c00 msgr2=0x7fc25c1fa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25a7fc700 1 --2- 172.21.15.81:0/3968437105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc248003c00 0x7fc25c1fa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25a7fc700 1 -- 172.21.15.81:0/3968437105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc25c0af8e0 msgr2=0x7fc25c13f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25a7fc700 1 --2- 172.21.15.81:0/3968437105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc25c0af8e0 0x7fc25c13f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25a7fc700 1 -- 172.21.15.81:0/3968437105 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c1403e0 con 0x7fc25c1fbd40 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25b7fe700 1 --2- 172.21.15.81:0/3968437105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc25c0af8e0 0x7fc25c13f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25a7fc700 1 --2- 172.21.15.81:0/3968437105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc25c1fbd40 0x7fc25c0af3a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc244010790 tx=0x7fc244010aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc253fff700 1 -- 172.21.15.81:0/3968437105 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244004220 con 0x7fc25c1fbd40 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc253fff700 1 -- 172.21.15.81:0/3968437105 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc244004380 con 0x7fc25c1fbd40 2024-05-08T23:31:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25bfff700 1 -- 172.21.15.81:0/3968437105 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25c1fb260 con 0x7fc25c1fbd40 2024-05-08T23:31:26.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc253fff700 1 -- 172.21.15.81:0/3968437105 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2440036c0 con 0x7fc25c1fbd40 2024-05-08T23:31:26.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.148+0000 7fc25bfff700 1 -- 172.21.15.81:0/3968437105 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25c1fae20 con 0x7fc25c1fbd40 2024-05-08T23:31:26.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.152+0000 7fc25bfff700 1 -- 172.21.15.81:0/3968437105 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25c0684b0 con 0x7fc25c1fbd40 2024-05-08T23:31:26.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.152+0000 7fc253fff700 1 -- 172.21.15.81:0/3968437105 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc244003820 con 0x7fc25c1fbd40 2024-05-08T23:31:26.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.152+0000 7fc253fff700 1 --2- 172.21.15.81:0/3968437105 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc23c066730 0x7fc23c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.152+0000 7fc25affd700 1 --2- 172.21.15.81:0/3968437105 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc23c066730 0x7fc23c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.152+0000 7fc253fff700 1 -- 172.21.15.81:0/3968437105 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc244083e10 con 0x7fc25c1fbd40 2024-05-08T23:31:26.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.152+0000 7fc25affd700 1 --2- 172.21.15.81:0/3968437105 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc23c066730 0x7fc23c068bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc254016040 tx=0x7fc254009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:26.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.160+0000 7fc253fff700 1 -- 172.21.15.81:0/3968437105 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc244051190 con 0x7fc25c1fbd40 2024-05-08T23:31:26.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.368+0000 7fc25bfff700 1 -- 172.21.15.81:0/3968437105 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc25c156930 con 0x7fc25c1fbd40 2024-05-08T23:31:26.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.372+0000 7fc253fff700 1 -- 172.21.15.81:0/3968437105 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7fc244021090 con 0x7fc25c1fbd40 2024-05-08T23:31:26.376 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:26.377 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1475726755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1475726755},{"type":"v1","addr":"172.21.15.81:6835","nonce":1475726755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3535820540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3535820540},{"type":"v1","addr":"172.21.15.81:6837","nonce":3535820540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1531019955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1531019955},{"type":"v1","addr":"172.21.15.117:6833","nonce":1531019955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:30:38.665368+0000","modified":"2024-05-08T23:30:41.672722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/491911303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":491911303},{"type":"v1","addr":"172.21.15.117:6835","nonce":491911303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:31:26.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.372+0000 7fc25bfff700 1 -- 172.21.15.81:0/3968437105 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc23c066730 msgr2=0x7fc23c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.372+0000 7fc25bfff700 1 --2- 172.21.15.81:0/3968437105 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc23c066730 0x7fc23c068bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc254016040 tx=0x7fc254009000 comp rx=0 tx=0).stop 2024-05-08T23:31:26.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.372+0000 7fc25bfff700 1 -- 172.21.15.81:0/3968437105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc25c1fbd40 msgr2=0x7fc25c0af3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.372+0000 7fc25bfff700 1 --2- 172.21.15.81:0/3968437105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc25c1fbd40 0x7fc25c0af3a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc244010790 tx=0x7fc244010aa0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.372+0000 7fc25bfff700 1 -- 172.21.15.81:0/3968437105 shutdown_connections 2024-05-08T23:31:26.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.372+0000 7fc25bfff700 1 --2- 172.21.15.81:0/3968437105 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc23c066730 0x7fc23c068bf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.372+0000 7fc25bfff700 1 --2- 172.21.15.81:0/3968437105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc25c0af8e0 0x7fc25c13f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.372+0000 7fc25bfff700 1 --2- 172.21.15.81:0/3968437105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc25c1fbd40 0x7fc25c0af3a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.372+0000 7fc25bfff700 1 --2- 172.21.15.81:0/3968437105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc248003c00 0x7fc25c1fa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.372+0000 7fc25bfff700 1 -- 172.21.15.81:0/3968437105 >> 172.21.15.81:0/3968437105 conn(0x7fc25c06c330 msgr2=0x7fc25c06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:26.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.372+0000 7fc25bfff700 1 -- 172.21.15.81:0/3968437105 shutdown_connections 2024-05-08T23:31:26.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.372+0000 7fc25bfff700 1 -- 172.21.15.81:0/3968437105 wait complete. 2024-05-08T23:31:26.381 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 7 2024-05-08T23:31:26.392 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:31:26.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.500+0000 7fbeca757700 1 Processor -- start 2024-05-08T23:31:26.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.500+0000 7fbeca757700 1 -- start start 2024-05-08T23:31:26.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.500+0000 7fbeca757700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec4071ab0 0x7fbec40ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.500+0000 7fbeca757700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbec4071e90 0x7fbec415a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.500+0000 7fbeca757700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec40cea50 con 0x7fbec415aaa0 2024-05-08T23:31:26.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.500+0000 7fbeca757700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec40ce080 con 0x7fbec4071e90 2024-05-08T23:31:26.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.500+0000 7fbeca757700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec40cd7c0 con 0x7fbec4071ab0 2024-05-08T23:31:26.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.500+0000 7fbec8f54700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbec4071e90 0x7fbec415a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.500+0000 7fbec8f54700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbec4071e90 0x7fbec415a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42554/0 (socket says 172.21.15.81:42554) 2024-05-08T23:31:26.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.500+0000 7fbec8f54700 1 -- 172.21.15.81:0/15798586 learned_addr learned my addr 172.21.15.81:0/15798586 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:26.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.500+0000 7fbec9755700 1 --2- 172.21.15.81:0/15798586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec4071ab0 0x7fbec40ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbec8f54700 1 -- 172.21.15.81:0/15798586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec4071ab0 msgr2=0x7fbec40ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:26.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbec8f54700 1 --2- 172.21.15.81:0/15798586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec4071ab0 0x7fbec40ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbec8f54700 1 -- 172.21.15.81:0/15798586 >> v1:172.21.15.117:6789/0 conn(0x7fbec415aaa0 legacy=0x7fbec413d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:26.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbec8f54700 1 -- 172.21.15.81:0/15798586 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec40cda30 con 0x7fbec4071e90 2024-05-08T23:31:26.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbec9755700 1 --2- 172.21.15.81:0/15798586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec4071ab0 0x7fbec40ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:26.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbec8f54700 1 --2- 172.21.15.81:0/15798586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbec4071e90 0x7fbec415a560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400bbb0 tx=0x7fbeb400bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b71f27cfe06cf68 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:26.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbebbfff700 1 -- 172.21.15.81:0/15798586 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb400dd50 con 0x7fbec4071e90 2024-05-08T23:31:26.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbebbfff700 1 -- 172.21.15.81:0/15798586 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb4004030 con 0x7fbec4071e90 2024-05-08T23:31:26.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbebbfff700 1 -- 172.21.15.81:0/15798586 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb40043e0 con 0x7fbec4071e90 2024-05-08T23:31:26.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbeca757700 1 -- 172.21.15.81:0/15798586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbec4071e90 msgr2=0x7fbec415a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbeca757700 1 --2- 172.21.15.81:0/15798586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbec4071e90 0x7fbec415a560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400bbb0 tx=0x7fbeb400bec0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbeca757700 1 -- 172.21.15.81:0/15798586 shutdown_connections 2024-05-08T23:31:26.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbeca757700 1 --2- 172.21.15.81:0/15798586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbec4071e90 0x7fbec415a560 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbeca757700 1 --2- 172.21.15.81:0/15798586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec4071ab0 0x7fbec40ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbeca757700 1 -- 172.21.15.81:0/15798586 >> 172.21.15.81:0/15798586 conn(0x7fbec406c330 msgr2=0x7fbec406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:26.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbeca757700 1 -- 172.21.15.81:0/15798586 shutdown_connections 2024-05-08T23:31:26.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbeca757700 1 -- 172.21.15.81:0/15798586 wait complete. 2024-05-08T23:31:26.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbeca757700 1 Processor -- start 2024-05-08T23:31:26.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.504+0000 7fbeca757700 1 -- start start 2024-05-08T23:31:26.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbeca757700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec4071ab0 0x7fbec412f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbeca757700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec4071e90 0x7fbec413d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbeca757700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbec415aaa0 0x7fbec413f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbeca757700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec40b3c80 con 0x7fbec415aaa0 2024-05-08T23:31:26.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbeca757700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec40b2530 con 0x7fbec4071ab0 2024-05-08T23:31:26.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbeca757700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec412fb70 con 0x7fbec4071e90 2024-05-08T23:31:26.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbec9f56700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbec415aaa0 0x7fbec413f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbec9f56700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbec415aaa0 0x7fbec413f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42560/0 (socket says 172.21.15.81:42560) 2024-05-08T23:31:26.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbec9f56700 1 -- 172.21.15.81:0/281586665 learned_addr learned my addr 172.21.15.81:0/281586665 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:26.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbec8f54700 1 --2- 172.21.15.81:0/281586665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec4071e90 0x7fbec413d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbec9755700 1 --2- 172.21.15.81:0/281586665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec4071ab0 0x7fbec412f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbec9f56700 1 -- 172.21.15.81:0/281586665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec4071e90 msgr2=0x7fbec413d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbec9f56700 1 --2- 172.21.15.81:0/281586665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec4071e90 0x7fbec413d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbec9f56700 1 -- 172.21.15.81:0/281586665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec4071ab0 msgr2=0x7fbec412f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbec9f56700 1 --2- 172.21.15.81:0/281586665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec4071ab0 0x7fbec412f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbec9f56700 1 -- 172.21.15.81:0/281586665 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb400b810 con 0x7fbec415aaa0 2024-05-08T23:31:26.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbec9755700 1 --2- 172.21.15.81:0/281586665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec4071ab0 0x7fbec412f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:26.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbec9f56700 1 --2- 172.21.15.81:0/281586665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbec415aaa0 0x7fbec413f070 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00b970 tx=0x7fbebc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:26.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbeb9ffb700 1 -- 172.21.15.81:0/281586665 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc00c930 con 0x7fbec415aaa0 2024-05-08T23:31:26.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbeca757700 1 -- 172.21.15.81:0/281586665 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec40b1f70 con 0x7fbec415aaa0 2024-05-08T23:31:26.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbeb9ffb700 1 -- 172.21.15.81:0/281586665 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc00cad0 con 0x7fbec415aaa0 2024-05-08T23:31:26.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbeb9ffb700 1 -- 172.21.15.81:0/281586665 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc00ce80 con 0x7fbec415aaa0 2024-05-08T23:31:26.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.508+0000 7fbeca757700 1 -- 172.21.15.81:0/281586665 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec40b38a0 con 0x7fbec415aaa0 2024-05-08T23:31:26.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.512+0000 7fbeca757700 1 -- 172.21.15.81:0/281586665 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec40b3dc0 con 0x7fbec415aaa0 2024-05-08T23:31:26.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.512+0000 7fbeb9ffb700 1 -- 172.21.15.81:0/281586665 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbebc014440 con 0x7fbec415aaa0 2024-05-08T23:31:26.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.512+0000 7fbeb9ffb700 1 --2- 172.21.15.81:0/281586665 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbea8066730 0x7fbea8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.512+0000 7fbeb9ffb700 1 -- 172.21.15.81:0/281586665 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fbebc083e30 con 0x7fbec415aaa0 2024-05-08T23:31:26.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.512+0000 7fbec9755700 1 --2- 172.21.15.81:0/281586665 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbea8066730 0x7fbea8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.512+0000 7fbec9755700 1 --2- 172.21.15.81:0/281586665 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbea8066730 0x7fbea8068bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbec0006fd0 tx=0x7fbec0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:26.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.520+0000 7fbeb9ffb700 1 -- 172.21.15.81:0/281586665 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbec40b3dc0 con 0x7fbec415aaa0 2024-05-08T23:31:26.740 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.736+0000 7fbeca757700 1 -- 172.21.15.81:0/281586665 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbec40b3dc0 con 0x7fbec415aaa0 2024-05-08T23:31:26.741 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.736+0000 7fbeb9ffb700 1 -- 172.21.15.81:0/281586665 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4731 (secure 0 0 0) 0x7fbec40b3dc0 con 0x7fbec415aaa0 2024-05-08T23:31:26.741 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:26.741 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1475726755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1475726755},{"type":"v1","addr":"172.21.15.81:6835","nonce":1475726755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3535820540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3535820540},{"type":"v1","addr":"172.21.15.81:6837","nonce":3535820540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1531019955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1531019955},{"type":"v1","addr":"172.21.15.117:6833","nonce":1531019955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:30:38.665368+0000","modified":"2024-05-08T23:30:41.672722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/491911303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":491911303},{"type":"v1","addr":"172.21.15.117:6835","nonce":491911303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:31:26.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.736+0000 7fbeca757700 1 -- 172.21.15.81:0/281586665 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbea8066730 msgr2=0x7fbea8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.736+0000 7fbeca757700 1 --2- 172.21.15.81:0/281586665 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbea8066730 0x7fbea8068bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbec0006fd0 tx=0x7fbec0008040 comp rx=0 tx=0).stop 2024-05-08T23:31:26.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.736+0000 7fbeca757700 1 -- 172.21.15.81:0/281586665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbec415aaa0 msgr2=0x7fbec413f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.736+0000 7fbeca757700 1 --2- 172.21.15.81:0/281586665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbec415aaa0 0x7fbec413f070 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00b970 tx=0x7fbebc00bc80 comp rx=0 tx=0).stop 2024-05-08T23:31:26.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.736+0000 7fbeca757700 1 -- 172.21.15.81:0/281586665 shutdown_connections 2024-05-08T23:31:26.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.736+0000 7fbeca757700 1 --2- 172.21.15.81:0/281586665 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbea8066730 0x7fbea8068bf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.736+0000 7fbeca757700 1 --2- 172.21.15.81:0/281586665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbec415aaa0 0x7fbec413f070 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.736+0000 7fbeca757700 1 --2- 172.21.15.81:0/281586665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec4071e90 0x7fbec413d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.736+0000 7fbeca757700 1 --2- 172.21.15.81:0/281586665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec4071ab0 0x7fbec412f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.736+0000 7fbeca757700 1 -- 172.21.15.81:0/281586665 >> 172.21.15.81:0/281586665 conn(0x7fbec406c330 msgr2=0x7fbec415c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:26.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.736+0000 7fbeca757700 1 -- 172.21.15.81:0/281586665 shutdown_connections 2024-05-08T23:31:26.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.736+0000 7fbeca757700 1 -- 172.21.15.81:0/281586665 wait complete. 2024-05-08T23:31:26.745 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 7 2024-05-08T23:31:26.756 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:26.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.864+0000 7f3df5d0f700 1 Processor -- start 2024-05-08T23:31:26.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.864+0000 7f3df5d0f700 1 -- start start 2024-05-08T23:31:26.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.864+0000 7f3df5d0f700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3df015afa0 0x7f3df015b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.864+0000 7f3df5d0f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015b8c0 0x7f3df015f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.864+0000 7f3df5d0f700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df015eda0 con 0x7f3df015f770 2024-05-08T23:31:26.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.864+0000 7f3df5d0f700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df015ea70 con 0x7f3df015afa0 2024-05-08T23:31:26.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.864+0000 7f3df5d0f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df015bd40 con 0x7f3df015b8c0 2024-05-08T23:31:26.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.864+0000 7f3df550e700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f3df015f770 0x7f3df00b4270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:32802/0 (socket says 172.21.15.81:32802) 2024-05-08T23:31:26.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.864+0000 7f3df550e700 1 -- 172.21.15.81:0/1487346329 learned_addr learned my addr 172.21.15.81:0/1487346329 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:26.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.864+0000 7f3deffff700 1 --2- 172.21.15.81:0/1487346329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015b8c0 0x7f3df015f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.864+0000 7f3df4d0d700 1 --2- 172.21.15.81:0/1487346329 >> v2:172.21.15.117:3300/0 conn(0x7f3df015afa0 0x7f3df015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3def7fe700 1 -- 172.21.15.81:0/1487346329 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397801883 0 0) 0x7f3df015eda0 con 0x7f3df015f770 2024-05-08T23:31:26.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3def7fe700 1 -- 172.21.15.81:0/1487346329 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dd80035d0 con 0x7f3df015f770 2024-05-08T23:31:26.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3deffff700 1 -- 172.21.15.81:0/1487346329 >> v2:172.21.15.117:3300/0 conn(0x7f3df015afa0 msgr2=0x7f3df015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:26.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3deffff700 1 --2- 172.21.15.81:0/1487346329 >> v2:172.21.15.117:3300/0 conn(0x7f3df015afa0 0x7f3df015b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3deffff700 1 -- 172.21.15.81:0/1487346329 >> v1:172.21.15.81:6789/0 conn(0x7f3df015f770 legacy=0x7f3df00b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3deffff700 1 -- 172.21.15.81:0/1487346329 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df015c020 con 0x7f3df015b8c0 2024-05-08T23:31:26.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3deffff700 1 --2- 172.21.15.81:0/1487346329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015b8c0 0x7f3df015f230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009850 tx=0x7f3de4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5e6fa44e6d8c0c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:26.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3def7fe700 1 -- 172.21.15.81:0/1487346329 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4011830 con 0x7f3df015b8c0 2024-05-08T23:31:26.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3def7fe700 1 -- 172.21.15.81:0/1487346329 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de4011990 con 0x7f3df015b8c0 2024-05-08T23:31:26.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3def7fe700 1 -- 172.21.15.81:0/1487346329 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4011d40 con 0x7f3df015b8c0 2024-05-08T23:31:26.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3df5d0f700 1 -- 172.21.15.81:0/1487346329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015b8c0 msgr2=0x7f3df015f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3df5d0f700 1 --2- 172.21.15.81:0/1487346329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015b8c0 0x7f3df015f230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009850 tx=0x7f3de4009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:26.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3df5d0f700 1 -- 172.21.15.81:0/1487346329 shutdown_connections 2024-05-08T23:31:26.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3df5d0f700 1 --2- 172.21.15.81:0/1487346329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015b8c0 0x7f3df015f230 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3df5d0f700 1 --2- 172.21.15.81:0/1487346329 >> v2:172.21.15.117:3300/0 conn(0x7f3df015afa0 0x7f3df015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3df5d0f700 1 -- 172.21.15.81:0/1487346329 >> 172.21.15.81:0/1487346329 conn(0x7f3df006c330 msgr2=0x7f3df006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:26.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3df5d0f700 1 -- 172.21.15.81:0/1487346329 shutdown_connections 2024-05-08T23:31:26.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3df5d0f700 1 -- 172.21.15.81:0/1487346329 wait complete. 2024-05-08T23:31:26.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3df5d0f700 1 Processor -- start 2024-05-08T23:31:26.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.868+0000 7f3df5d0f700 1 -- start start 2024-05-08T23:31:26.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df5d0f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df015afa0 0x7f3df0147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df5d0f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015b8c0 0x7f3df00aa200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df5d0f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df015f770 0x7f3df01452d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df5d0f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df01f3fc0 con 0x7f3df015f770 2024-05-08T23:31:26.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df5d0f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df00aa740 con 0x7f3df015afa0 2024-05-08T23:31:26.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df5d0f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df0147940 con 0x7f3df015b8c0 2024-05-08T23:31:26.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df550e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df015f770 0x7f3df01452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df550e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df015f770 0x7f3df01452d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42574/0 (socket says 172.21.15.81:42574) 2024-05-08T23:31:26.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df550e700 1 -- 172.21.15.81:0/3877421450 learned_addr learned my addr 172.21.15.81:0/3877421450 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:26.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3deffff700 1 --2- 172.21.15.81:0/3877421450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015b8c0 0x7f3df00aa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df4d0d700 1 --2- 172.21.15.81:0/3877421450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df015afa0 0x7f3df0147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df550e700 1 -- 172.21.15.81:0/3877421450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015b8c0 msgr2=0x7f3df00aa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df550e700 1 --2- 172.21.15.81:0/3877421450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015b8c0 0x7f3df00aa200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df550e700 1 -- 172.21.15.81:0/3877421450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df015afa0 msgr2=0x7f3df0147330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:26.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df550e700 1 --2- 172.21.15.81:0/3877421450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df015afa0 0x7f3df0147330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:26.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df550e700 1 -- 172.21.15.81:0/3877421450 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de40094b0 con 0x7f3df015f770 2024-05-08T23:31:26.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df4d0d700 1 --2- 172.21.15.81:0/3877421450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df015afa0 0x7f3df0147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:26.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df550e700 1 --2- 172.21.15.81:0/3877421450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df015f770 0x7f3df01452d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3de800ca70 tx=0x7f3de800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:26.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3ded7fa700 1 -- 172.21.15.81:0/3877421450 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8007480 con 0x7f3df015f770 2024-05-08T23:31:26.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3ded7fa700 1 -- 172.21.15.81:0/3877421450 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de80075e0 con 0x7f3df015f770 2024-05-08T23:31:26.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df5d0f700 1 -- 172.21.15.81:0/3877421450 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df0147c20 con 0x7f3df015f770 2024-05-08T23:31:26.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3ded7fa700 1 -- 172.21.15.81:0/3877421450 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8007950 con 0x7f3df015f770 2024-05-08T23:31:26.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df5d0f700 1 -- 172.21.15.81:0/3877421450 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df01481e0 con 0x7f3df015f770 2024-05-08T23:31:26.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.872+0000 7f3df5d0f700 1 -- 172.21.15.81:0/3877421450 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df01f4130 con 0x7f3df015f770 2024-05-08T23:31:26.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.876+0000 7f3ded7fa700 1 -- 172.21.15.81:0/3877421450 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3de801b3b0 con 0x7f3df015f770 2024-05-08T23:31:26.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.876+0000 7f3ded7fa700 1 --2- 172.21.15.81:0/3877421450 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3ddc066780 0x7f3ddc068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:26.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.876+0000 7f3ded7fa700 1 -- 172.21.15.81:0/3877421450 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3de8083930 con 0x7f3df015f770 2024-05-08T23:31:26.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.876+0000 7f3df4d0d700 1 --2- 172.21.15.81:0/3877421450 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3ddc066780 0x7f3ddc068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:26.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.876+0000 7f3df4d0d700 1 --2- 172.21.15.81:0/3877421450 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3ddc066780 0x7f3ddc068c40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009c40 tx=0x7f3de0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:26.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:26.880+0000 7f3ded7fa700 1 -- 172.21.15.81:0/3877421450 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3de8050e40 con 0x7f3df015f770 2024-05-08T23:31:27.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.052+0000 7f3df5d0f700 1 -- 172.21.15.81:0/3877421450 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3df00684b0 con 0x7f3df015f770 2024-05-08T23:31:27.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.056+0000 7f3ded7fa700 1 -- 172.21.15.81:0/3877421450 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f3de8017090 con 0x7f3df015f770 2024-05-08T23:31:27.060 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:27.060 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":24,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:20.851517+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:30:36.469749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:30:37.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:27.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.056+0000 7f3df5d0f700 1 -- 172.21.15.81:0/3877421450 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3ddc066780 msgr2=0x7f3ddc068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:27.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.056+0000 7f3df5d0f700 1 --2- 172.21.15.81:0/3877421450 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3ddc066780 0x7f3ddc068c40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009c40 tx=0x7f3de0008040 comp rx=0 tx=0).stop 2024-05-08T23:31:27.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.056+0000 7f3df5d0f700 1 -- 172.21.15.81:0/3877421450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df015f770 msgr2=0x7f3df01452d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:27.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.056+0000 7f3df5d0f700 1 --2- 172.21.15.81:0/3877421450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df015f770 0x7f3df01452d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3de800ca70 tx=0x7f3de800caa0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.056+0000 7f3df5d0f700 1 -- 172.21.15.81:0/3877421450 shutdown_connections 2024-05-08T23:31:27.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.056+0000 7f3df5d0f700 1 --2- 172.21.15.81:0/3877421450 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3ddc066780 0x7f3ddc068c40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.056+0000 7f3df5d0f700 1 --2- 172.21.15.81:0/3877421450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df015f770 0x7f3df01452d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.056+0000 7f3df5d0f700 1 --2- 172.21.15.81:0/3877421450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015b8c0 0x7f3df00aa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.056+0000 7f3df5d0f700 1 --2- 172.21.15.81:0/3877421450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df015afa0 0x7f3df0147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.056+0000 7f3df5d0f700 1 -- 172.21.15.81:0/3877421450 >> 172.21.15.81:0/3877421450 conn(0x7f3df006c330 msgr2=0x7f3df00cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:27.064 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.056+0000 7f3df5d0f700 1 -- 172.21.15.81:0/3877421450 shutdown_connections 2024-05-08T23:31:27.064 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.056+0000 7f3df5d0f700 1 -- 172.21.15.81:0/3877421450 wait complete. 2024-05-08T23:31:27.076 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T23:31:27.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aacc1b700 1 Processor -- start 2024-05-08T23:31:27.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aacc1b700 1 -- start start 2024-05-08T23:31:27.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aacc1b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5aa80d1690 0x7f5aa80d37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:27.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aacc1b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5aa80d0f20 0x7f5aa80d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:27.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aacc1b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5aa80cf6e0 0x7f5aa80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:27.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aacc1b700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa80d3e50 con 0x7f5aa80cf6e0 2024-05-08T23:31:27.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aacc1b700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa80d1fb0 con 0x7f5aa80d0f20 2024-05-08T23:31:27.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aacc1b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa80ce870 con 0x7f5aa80d1690 2024-05-08T23:31:27.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aa7fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5aa80cf6e0 0x7f5aa80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:27.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aa7fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5aa80cf6e0 0x7f5aa80cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42580/0 (socket says 172.21.15.81:42580) 2024-05-08T23:31:27.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aa7fff700 1 -- 172.21.15.81:0/3792988259 learned_addr learned my addr 172.21.15.81:0/3792988259 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:27.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aa77fe700 1 --2- 172.21.15.81:0/3792988259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5aa80d1690 0x7f5aa80d37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:27.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aa6ffd700 1 --2- 172.21.15.81:0/3792988259 >> v2:172.21.15.117:3300/0 conn(0x7f5aa80d0f20 0x7f5aa80d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:27.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aa7fff700 1 -- 172.21.15.81:0/3792988259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5aa80d1690 msgr2=0x7f5aa80d37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:27.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aa7fff700 1 --2- 172.21.15.81:0/3792988259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5aa80d1690 0x7f5aa80d37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aa7fff700 1 -- 172.21.15.81:0/3792988259 >> v2:172.21.15.117:3300/0 conn(0x7f5aa80d0f20 msgr2=0x7f5aa80d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:27.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aa7fff700 1 --2- 172.21.15.81:0/3792988259 >> v2:172.21.15.117:3300/0 conn(0x7f5aa80d0f20 0x7f5aa80d1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aa7fff700 1 -- 172.21.15.81:0/3792988259 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa80ceb50 con 0x7f5aa80cf6e0 2024-05-08T23:31:27.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aa7fff700 1 --2- 172.21.15.81:0/3792988259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aa80cf6e0 0x7f5aa80cfb70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5aa809caf0 tx=0x7f5a98009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b82f05c74da34b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:27.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aa67fc700 1 -- 172.21.15.81:0/3792988259 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98011830 con 0x7f5aa80cf6e0 2024-05-08T23:31:27.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aa67fc700 1 -- 172.21.15.81:0/3792988259 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a98011990 con 0x7f5aa80cf6e0 2024-05-08T23:31:27.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.188+0000 7f5aa67fc700 1 -- 172.21.15.81:0/3792988259 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98011d20 con 0x7f5aa80cf6e0 2024-05-08T23:31:27.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 -- 172.21.15.81:0/3792988259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aa80cf6e0 msgr2=0x7f5aa80cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:27.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 --2- 172.21.15.81:0/3792988259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aa80cf6e0 0x7f5aa80cfb70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5aa809caf0 tx=0x7f5a98009bf0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 -- 172.21.15.81:0/3792988259 shutdown_connections 2024-05-08T23:31:27.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 --2- 172.21.15.81:0/3792988259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aa80cf6e0 0x7f5aa80cfb70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 --2- 172.21.15.81:0/3792988259 >> v2:172.21.15.117:3300/0 conn(0x7f5aa80d0f20 0x7f5aa80d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 --2- 172.21.15.81:0/3792988259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5aa80d1690 0x7f5aa80d37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 -- 172.21.15.81:0/3792988259 >> 172.21.15.81:0/3792988259 conn(0x7f5aa809c3c0 msgr2=0x7f5aa80d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:27.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 -- 172.21.15.81:0/3792988259 shutdown_connections 2024-05-08T23:31:27.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 -- 172.21.15.81:0/3792988259 wait complete. 2024-05-08T23:31:27.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 Processor -- start 2024-05-08T23:31:27.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 -- start start 2024-05-08T23:31:27.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aa80cf6e0 0x7f5aa815e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:27.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5aa80d0f20 0x7f5aa814e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:27.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5aa80d1690 0x7f5aa81f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:27.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa814ea40 con 0x7f5aa80cf6e0 2024-05-08T23:31:27.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa81f6c20 con 0x7f5aa80d1690 2024-05-08T23:31:27.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aacc1b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa81f6f00 con 0x7f5aa80d0f20 2024-05-08T23:31:27.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aa77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aa80cf6e0 0x7f5aa815e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:27.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aa6ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5aa80d0f20 0x7f5aa814e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:27.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aa77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aa80cf6e0 0x7f5aa815e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42588/0 (socket says 172.21.15.81:42588) 2024-05-08T23:31:27.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.192+0000 7f5aa77fe700 1 -- 172.21.15.81:0/3339020835 learned_addr learned my addr 172.21.15.81:0/3339020835 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:27.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.196+0000 7f5aa7fff700 1 --2- 172.21.15.81:0/3339020835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5aa80d1690 0x7f5aa81f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:27.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.196+0000 7f5aa6ffd700 1 -- 172.21.15.81:0/3339020835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5aa80d1690 msgr2=0x7f5aa81f63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:27.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.196+0000 7f5aa6ffd700 1 --2- 172.21.15.81:0/3339020835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5aa80d1690 0x7f5aa81f63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.196+0000 7f5aa6ffd700 1 -- 172.21.15.81:0/3339020835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aa80cf6e0 msgr2=0x7f5aa815e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:27.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.196+0000 7f5aa6ffd700 1 --2- 172.21.15.81:0/3339020835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aa80cf6e0 0x7f5aa815e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.196+0000 7f5aa6ffd700 1 -- 172.21.15.81:0/3339020835 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a98009570 con 0x7f5aa80d0f20 2024-05-08T23:31:27.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.196+0000 7f5aa77fe700 1 --2- 172.21.15.81:0/3339020835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aa80cf6e0 0x7f5aa815e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:27.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.196+0000 7f5aa6ffd700 1 --2- 172.21.15.81:0/3339020835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5aa80d0f20 0x7f5aa814e500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000ead0 tx=0x7f5aa000ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:27.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.196+0000 7f5a9ffff700 1 -- 172.21.15.81:0/3339020835 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0010040 con 0x7f5aa80d0f20 2024-05-08T23:31:27.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.196+0000 7f5a9ffff700 1 -- 172.21.15.81:0/3339020835 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa000b9b0 con 0x7f5aa80d0f20 2024-05-08T23:31:27.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.196+0000 7f5a9ffff700 1 -- 172.21.15.81:0/3339020835 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0009aa0 con 0x7f5aa80d0f20 2024-05-08T23:31:27.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.196+0000 7f5aacc1b700 1 -- 172.21.15.81:0/3339020835 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa81f71e0 con 0x7f5aa80d0f20 2024-05-08T23:31:27.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.196+0000 7f5aacc1b700 1 -- 172.21.15.81:0/3339020835 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa81f6930 con 0x7f5aa80d0f20 2024-05-08T23:31:27.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.196+0000 7f5aacc1b700 1 -- 172.21.15.81:0/3339020835 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa80684b0 con 0x7f5aa80d0f20 2024-05-08T23:31:27.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.200+0000 7f5a9ffff700 1 -- 172.21.15.81:0/3339020835 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5aa001d400 con 0x7f5aa80d0f20 2024-05-08T23:31:27.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.200+0000 7f5a9ffff700 1 --2- 172.21.15.81:0/3339020835 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5a8c066760 0x7f5a8c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:27.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.200+0000 7f5a9ffff700 1 -- 172.21.15.81:0/3339020835 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5aa0084960 con 0x7f5aa80d0f20 2024-05-08T23:31:27.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.200+0000 7f5aa77fe700 1 --2- 172.21.15.81:0/3339020835 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5a8c066760 0x7f5a8c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:27.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.200+0000 7f5aa77fe700 1 --2- 172.21.15.81:0/3339020835 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5a8c066760 0x7f5a8c068c20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a90011310 tx=0x7f5a90009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:27.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.208+0000 7f5a9ffff700 1 -- 172.21.15.81:0/3339020835 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5aa0051dc0 con 0x7f5aa80d0f20 2024-05-08T23:31:27.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.408+0000 7f5aacc1b700 1 -- 172.21.15.81:0/3339020835 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5aa81f7040 con 0x7f5aa80d0f20 2024-05-08T23:31:27.418 INFO:tasks.ceph.mon.a.smithi081.stderr:2024-05-08T23:31:27.412+0000 7f7c02c75700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:31:27.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.428+0000 7f5a9ffff700 1 -- 172.21.15.81:0/3339020835 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5aa81f7040 con 0x7f5aa80d0f20 2024-05-08T23:31:27.436 INFO:tasks.ceph.mds.b.smithi117.stderr: -2954> 2024-05-08T23:30:42.923+0000 7f4bb16db700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:31:27.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.432+0000 7f5aacc1b700 1 -- 172.21.15.81:0/3339020835 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5a8c066760 msgr2=0x7f5a8c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:27.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.432+0000 7f5aacc1b700 1 --2- 172.21.15.81:0/3339020835 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5a8c066760 0x7f5a8c068c20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a90011310 tx=0x7f5a90009240 comp rx=0 tx=0).stop 2024-05-08T23:31:27.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.432+0000 7f5aacc1b700 1 -- 172.21.15.81:0/3339020835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5aa80d0f20 msgr2=0x7f5aa814e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:27.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.432+0000 7f5aacc1b700 1 --2- 172.21.15.81:0/3339020835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5aa80d0f20 0x7f5aa814e500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000ead0 tx=0x7f5aa000ede0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.432+0000 7f5aacc1b700 1 -- 172.21.15.81:0/3339020835 shutdown_connections 2024-05-08T23:31:27.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.432+0000 7f5aacc1b700 1 --2- 172.21.15.81:0/3339020835 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5a8c066760 0x7f5a8c068c20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.432+0000 7f5aacc1b700 1 --2- 172.21.15.81:0/3339020835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5aa80d1690 0x7f5aa81f63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.432+0000 7f5aacc1b700 1 --2- 172.21.15.81:0/3339020835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5aa80d0f20 0x7f5aa814e500 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.432+0000 7f5aacc1b700 1 --2- 172.21.15.81:0/3339020835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aa80cf6e0 0x7f5aa815e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.432+0000 7f5aacc1b700 1 -- 172.21.15.81:0/3339020835 >> 172.21.15.81:0/3339020835 conn(0x7f5aa809c3c0 msgr2=0x7f5aa806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:27.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.432+0000 7f5aacc1b700 1 -- 172.21.15.81:0/3339020835 shutdown_connections 2024-05-08T23:31:27.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.432+0000 7f5aacc1b700 1 -- 172.21.15.81:0/3339020835 wait complete. 2024-05-08T23:31:27.439 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T23:31:27.452 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T23:31:27.468 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-05-08T23:31:27.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d7da8700 1 Processor -- start 2024-05-08T23:31:27.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d7da8700 1 -- start start 2024-05-08T23:31:27.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d7da8700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83d015af70 0x7f83d006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:27.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d7da8700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f83d00b4560 0x7f83d00b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:27.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d7da8700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d006acf0 con 0x7f83d006ee00 2024-05-08T23:31:27.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d7da8700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d00b4f50 con 0x7f83d00b4560 2024-05-08T23:31:27.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d7da8700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d006a400 con 0x7f83d015af70 2024-05-08T23:31:27.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d6345700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f83d00b4560 0x7f83d00b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:27.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d6345700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f83d00b4560 0x7f83d00b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42602/0 (socket says 172.21.15.81:42602) 2024-05-08T23:31:27.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d6345700 1 -- 172.21.15.81:0/1250013088 learned_addr learned my addr 172.21.15.81:0/1250013088 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:27.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d5b44700 1 --2- 172.21.15.81:0/1250013088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83d015af70 0x7f83d006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:27.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d6345700 1 -- 172.21.15.81:0/1250013088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83d015af70 msgr2=0x7f83d006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:27.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d6345700 1 --2- 172.21.15.81:0/1250013088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83d015af70 0x7f83d006e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d6345700 1 -- 172.21.15.81:0/1250013088 >> v1:172.21.15.117:6789/0 conn(0x7f83d006ee00 legacy=0x7f83d015b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:27.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d6345700 1 -- 172.21.15.81:0/1250013088 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d006a6a0 con 0x7f83d00b4560 2024-05-08T23:31:27.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d5b44700 1 --2- 172.21.15.81:0/1250013088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83d015af70 0x7f83d006e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:27.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d6345700 1 --2- 172.21.15.81:0/1250013088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83d00b4560 0x7f83d00b4a10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009e50 tx=0x7f83c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927408e8361bacd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:27.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.564+0000 7f83d4b42700 1 -- 172.21.15.81:0/1250013088 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c800e070 con 0x7f83d00b4560 2024-05-08T23:31:27.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.568+0000 7f83d4b42700 1 -- 172.21.15.81:0/1250013088 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c800b9c0 con 0x7f83d00b4560 2024-05-08T23:31:27.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.568+0000 7f83d4b42700 1 -- 172.21.15.81:0/1250013088 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c800bd70 con 0x7f83d00b4560 2024-05-08T23:31:27.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.568+0000 7f83d7da8700 1 -- 172.21.15.81:0/1250013088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83d00b4560 msgr2=0x7f83d00b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:27.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.568+0000 7f83d7da8700 1 --2- 172.21.15.81:0/1250013088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83d00b4560 0x7f83d00b4a10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009e50 tx=0x7f83c800b330 comp rx=0 tx=0).stop 2024-05-08T23:31:27.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.568+0000 7f83d7da8700 1 -- 172.21.15.81:0/1250013088 shutdown_connections 2024-05-08T23:31:27.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.568+0000 7f83d7da8700 1 --2- 172.21.15.81:0/1250013088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83d00b4560 0x7f83d00b4a10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.568+0000 7f83d7da8700 1 --2- 172.21.15.81:0/1250013088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83d015af70 0x7f83d006e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.568+0000 7f83d7da8700 1 -- 172.21.15.81:0/1250013088 >> 172.21.15.81:0/1250013088 conn(0x7f83d00cf630 msgr2=0x7f83d00cfa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:27.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.568+0000 7f83d7da8700 1 -- 172.21.15.81:0/1250013088 shutdown_connections 2024-05-08T23:31:27.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.568+0000 7f83d7da8700 1 -- 172.21.15.81:0/1250013088 wait complete. 2024-05-08T23:31:27.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.568+0000 7f83d7da8700 1 Processor -- start 2024-05-08T23:31:27.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.568+0000 7f83d7da8700 1 -- start start 2024-05-08T23:31:27.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d7da8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83d006ee00 0x7f83d0143100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:27.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d7da8700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83d00b4560 0x7f83d00ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:27.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d7da8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83d015af70 0x7f83d00ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:27.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d7da8700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d0143640 con 0x7f83d006ee00 2024-05-08T23:31:27.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d7da8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d01557b0 con 0x7f83d015af70 2024-05-08T23:31:27.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d7da8700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d0155a00 con 0x7f83d00b4560 2024-05-08T23:31:27.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d5b44700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83d006ee00 0x7f83d0143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:27.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d5b44700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83d006ee00 0x7f83d0143100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42618/0 (socket says 172.21.15.81:42618) 2024-05-08T23:31:27.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d5b44700 1 -- 172.21.15.81:0/1658219425 learned_addr learned my addr 172.21.15.81:0/1658219425 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:27.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d6345700 1 --2- 172.21.15.81:0/1658219425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83d015af70 0x7f83d00ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:27.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d5343700 1 --2- 172.21.15.81:0/1658219425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83d00b4560 0x7f83d00ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:27.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d5b44700 1 -- 172.21.15.81:0/1658219425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83d00b4560 msgr2=0x7f83d00ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:27.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d5b44700 1 --2- 172.21.15.81:0/1658219425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83d00b4560 0x7f83d00ac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d5b44700 1 -- 172.21.15.81:0/1658219425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83d015af70 msgr2=0x7f83d00ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:27.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d5b44700 1 --2- 172.21.15.81:0/1658219425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83d015af70 0x7f83d00ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:27.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d5b44700 1 -- 172.21.15.81:0/1658219425 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c8009b30 con 0x7f83d006ee00 2024-05-08T23:31:27.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d5b44700 1 --2- 172.21.15.81:0/1658219425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83d006ee00 0x7f83d0143100 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00e810 tx=0x7f83cc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:27.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83c67fc700 1 -- 172.21.15.81:0/1658219425 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc00c070 con 0x7f83d006ee00 2024-05-08T23:31:27.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83c67fc700 1 -- 172.21.15.81:0/1658219425 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc013610 con 0x7f83d006ee00 2024-05-08T23:31:27.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83c67fc700 1 -- 172.21.15.81:0/1658219425 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0139a0 con 0x7f83d006ee00 2024-05-08T23:31:27.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d7da8700 1 -- 172.21.15.81:0/1658219425 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d0155ce0 con 0x7f83d006ee00 2024-05-08T23:31:27.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d7da8700 1 -- 172.21.15.81:0/1658219425 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d01562a0 con 0x7f83d006ee00 2024-05-08T23:31:27.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.572+0000 7f83d7da8700 1 -- 172.21.15.81:0/1658219425 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d0156460 con 0x7f83d006ee00 2024-05-08T23:31:27.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.576+0000 7f83c67fc700 1 -- 172.21.15.81:0/1658219425 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f83cc013b00 con 0x7f83d006ee00 2024-05-08T23:31:27.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.576+0000 7f83c67fc700 1 --2- 172.21.15.81:0/1658219425 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f83b4066730 0x7f83b4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:27.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.576+0000 7f83c67fc700 1 -- 172.21.15.81:0/1658219425 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f83cc083ef0 con 0x7f83d006ee00 2024-05-08T23:31:27.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.576+0000 7f83d5343700 1 --2- 172.21.15.81:0/1658219425 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f83b4066730 0x7f83b4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:27.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.576+0000 7f83d5343700 1 --2- 172.21.15.81:0/1658219425 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f83b4066730 0x7f83b4068bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f83d0071ab0 tx=0x7f83c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:27.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.584+0000 7f83c67fc700 1 -- 172.21.15.81:0/1658219425 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f83cc051100 con 0x7f83d006ee00 2024-05-08T23:31:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:27.792+0000 7f83d7da8700 1 -- 172.21.15.81:0/1658219425 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f83d00672a0 con 0x7f83d006ee00 2024-05-08T23:31:28.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.432+0000 7f83c67fc700 1 -- 172.21.15.81:0/1658219425 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f83cc013db0 con 0x7f83d006ee00 2024-05-08T23:31:28.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.436+0000 7f83d7da8700 1 -- 172.21.15.81:0/1658219425 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f83b4066730 msgr2=0x7f83b4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:28.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.436+0000 7f83d7da8700 1 --2- 172.21.15.81:0/1658219425 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f83b4066730 0x7f83b4068bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f83d0071ab0 tx=0x7f83c0009040 comp rx=0 tx=0).stop 2024-05-08T23:31:28.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.436+0000 7f83d7da8700 1 -- 172.21.15.81:0/1658219425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83d006ee00 msgr2=0x7f83d0143100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:28.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.436+0000 7f83d7da8700 1 --2- 172.21.15.81:0/1658219425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83d006ee00 0x7f83d0143100 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00e810 tx=0x7f83cc00eb20 comp rx=0 tx=0).stop 2024-05-08T23:31:28.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.436+0000 7f83d7da8700 1 -- 172.21.15.81:0/1658219425 shutdown_connections 2024-05-08T23:31:28.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.436+0000 7f83d7da8700 1 --2- 172.21.15.81:0/1658219425 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f83b4066730 0x7f83b4068bf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:28.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.436+0000 7f83d7da8700 1 --2- 172.21.15.81:0/1658219425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83d015af70 0x7f83d00ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:28.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.436+0000 7f83d7da8700 1 --2- 172.21.15.81:0/1658219425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83d00b4560 0x7f83d00ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:28.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.436+0000 7f83d7da8700 1 --2- 172.21.15.81:0/1658219425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83d006ee00 0x7f83d0143100 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:28.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.436+0000 7f83d7da8700 1 -- 172.21.15.81:0/1658219425 >> 172.21.15.81:0/1658219425 conn(0x7f83d00cf630 msgr2=0x7f83d00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:28.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.436+0000 7f83d7da8700 1 -- 172.21.15.81:0/1658219425 shutdown_connections 2024-05-08T23:31:28.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.436+0000 7f83d7da8700 1 -- 172.21.15.81:0/1658219425 wait complete. 2024-05-08T23:31:28.460 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T23:31:28.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.588+0000 7faa1759e700 1 Processor -- start 2024-05-08T23:31:28.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.588+0000 7faa1759e700 1 -- start start 2024-05-08T23:31:28.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1759e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7faa1815d300 0x7faa180d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:28.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1759e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faa180d02b0 0x7faa180d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:28.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1759e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa180b3030 con 0x7faa180d1a30 2024-05-08T23:31:28.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1759e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa180d1210 con 0x7faa1815d300 2024-05-08T23:31:28.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1759e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa180b29e0 con 0x7faa180d02b0 2024-05-08T23:31:28.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1659c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7faa1815d300 0x7faa180d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:28.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa16d9d700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7faa180d1a30 0x7faa180cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:32808/0 (socket says 172.21.15.81:32808) 2024-05-08T23:31:28.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa16d9d700 1 -- 172.21.15.81:0/866459062 learned_addr learned my addr 172.21.15.81:0/866459062 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:28.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa15d9b700 1 --2- 172.21.15.81:0/866459062 >> v2:172.21.15.117:3300/0 conn(0x7faa180d02b0 0x7faa180d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:28.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1659c700 1 -- 172.21.15.81:0/866459062 >> v2:172.21.15.117:3300/0 conn(0x7faa180d02b0 msgr2=0x7faa180d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1659c700 1 --2- 172.21.15.81:0/866459062 >> v2:172.21.15.117:3300/0 conn(0x7faa180d02b0 0x7faa180d14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1659c700 1 -- 172.21.15.81:0/866459062 >> v1:172.21.15.81:6789/0 conn(0x7faa180d1a30 legacy=0x7faa180cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1659c700 1 -- 172.21.15.81:0/866459062 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa180b2cc0 con 0x7faa1815d300 2024-05-08T23:31:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1559a700 1 -- 172.21.15.81:0/866459062 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602383969 0 0) 0x7faa180b3030 con 0x7faa180d1a30 2024-05-08T23:31:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1659c700 1 --2- 172.21.15.81:0/866459062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa1815d300 0x7faa180d0c30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faa1808cd00 tx=0x7faa0c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5d8f2b7d505d80 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1559a700 1 -- 172.21.15.81:0/866459062 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c011830 con 0x7faa1815d300 2024-05-08T23:31:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1559a700 1 -- 172.21.15.81:0/866459062 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c011990 con 0x7faa1815d300 2024-05-08T23:31:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1559a700 1 -- 172.21.15.81:0/866459062 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c011d20 con 0x7faa1815d300 2024-05-08T23:31:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1759e700 1 -- 172.21.15.81:0/866459062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa1815d300 msgr2=0x7faa180d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1759e700 1 --2- 172.21.15.81:0/866459062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa1815d300 0x7faa180d0c30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faa1808cd00 tx=0x7faa0c009b30 comp rx=0 tx=0).stop 2024-05-08T23:31:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1759e700 1 -- 172.21.15.81:0/866459062 shutdown_connections 2024-05-08T23:31:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1759e700 1 --2- 172.21.15.81:0/866459062 >> v2:172.21.15.117:3300/0 conn(0x7faa180d02b0 0x7faa180d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1759e700 1 --2- 172.21.15.81:0/866459062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa1815d300 0x7faa180d0c30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1759e700 1 -- 172.21.15.81:0/866459062 >> 172.21.15.81:0/866459062 conn(0x7faa1809c3b0 msgr2=0x7faa180d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1759e700 1 -- 172.21.15.81:0/866459062 shutdown_connections 2024-05-08T23:31:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.592+0000 7faa1759e700 1 -- 172.21.15.81:0/866459062 wait complete. 2024-05-08T23:31:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa1759e700 1 Processor -- start 2024-05-08T23:31:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa1759e700 1 -- start start 2024-05-08T23:31:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa1759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa180d02b0 0x7faa180cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa1759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa180d1a30 0x7faa181f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa1759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1815d300 0x7faa181f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa1759e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa181f7190 con 0x7faa180d1a30 2024-05-08T23:31:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa1759e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa181ed210 con 0x7faa180d02b0 2024-05-08T23:31:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa1759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa180ca370 con 0x7faa1815d300 2024-05-08T23:31:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa15d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa180d1a30 0x7faa181f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa15d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa180d1a30 0x7faa181f3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42626/0 (socket says 172.21.15.81:42626) 2024-05-08T23:31:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa15d9b700 1 -- 172.21.15.81:0/2480908516 learned_addr learned my addr 172.21.15.81:0/2480908516 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa1659c700 1 --2- 172.21.15.81:0/2480908516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa180d02b0 0x7faa180cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa16d9d700 1 --2- 172.21.15.81:0/2480908516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1815d300 0x7faa181f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa15d9b700 1 -- 172.21.15.81:0/2480908516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1815d300 msgr2=0x7faa181f43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:28.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa15d9b700 1 --2- 172.21.15.81:0/2480908516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1815d300 0x7faa181f43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:28.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa15d9b700 1 -- 172.21.15.81:0/2480908516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa180d02b0 msgr2=0x7faa180cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:28.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa15d9b700 1 --2- 172.21.15.81:0/2480908516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa180d02b0 0x7faa180cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:28.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa15d9b700 1 -- 172.21.15.81:0/2480908516 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c0094b0 con 0x7faa180d1a30 2024-05-08T23:31:28.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa15d9b700 1 --2- 172.21.15.81:0/2480908516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa180d1a30 0x7faa181f3e70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faa0800e9d0 tx=0x7faa0800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:28.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa06ffd700 1 -- 172.21.15.81:0/2480908516 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa08009720 con 0x7faa180d1a30 2024-05-08T23:31:28.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa1759e700 1 -- 172.21.15.81:0/2480908516 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa180ca650 con 0x7faa180d1a30 2024-05-08T23:31:28.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.596+0000 7faa06ffd700 1 -- 172.21.15.81:0/2480908516 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0800ee70 con 0x7faa180d1a30 2024-05-08T23:31:28.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.600+0000 7faa06ffd700 1 -- 172.21.15.81:0/2480908516 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0800ca50 con 0x7faa180d1a30 2024-05-08T23:31:28.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.600+0000 7faa1759e700 1 -- 172.21.15.81:0/2480908516 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa180cac10 con 0x7faa180d1a30 2024-05-08T23:31:28.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.600+0000 7faa1759e700 1 -- 172.21.15.81:0/2480908516 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa180684b0 con 0x7faa180d1a30 2024-05-08T23:31:28.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.600+0000 7faa06ffd700 1 -- 172.21.15.81:0/2480908516 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faa0800cbb0 con 0x7faa180d1a30 2024-05-08T23:31:28.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.600+0000 7faa06ffd700 1 --2- 172.21.15.81:0/2480908516 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa9f8066730 0x7fa9f8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:28.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.600+0000 7faa06ffd700 1 -- 172.21.15.81:0/2480908516 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5911+0+0 (secure 0 0 0) 0x7faa080842b0 con 0x7faa180d1a30 2024-05-08T23:31:28.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.600+0000 7faa1659c700 1 --2- 172.21.15.81:0/2480908516 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa9f8066730 0x7fa9f8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:28.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.600+0000 7faa1659c700 1 --2- 172.21.15.81:0/2480908516 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa9f8066730 0x7fa9f8068bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009fd0 tx=0x7faa0c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:28.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.608+0000 7faa06ffd700 1 -- 172.21.15.81:0/2480908516 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faa080514d0 con 0x7faa180d1a30 2024-05-08T23:31:28.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:28.776+0000 7faa1759e700 1 -- 172.21.15.81:0/2480908516 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faa1806c660 con 0x7faa180d1a30 2024-05-08T23:31:29.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.436+0000 7faa06ffd700 1 -- 172.21.15.81:0/2480908516 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 181+0+0 (secure 0 0 0) 0x7faa0800ce60 con 0x7faa180d1a30 2024-05-08T23:31:29.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.440+0000 7faa1759e700 1 -- 172.21.15.81:0/2480908516 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa9f8066730 msgr2=0x7fa9f8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:29.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.440+0000 7faa1759e700 1 --2- 172.21.15.81:0/2480908516 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa9f8066730 0x7fa9f8068bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009fd0 tx=0x7faa0c008030 comp rx=0 tx=0).stop 2024-05-08T23:31:29.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.440+0000 7faa1759e700 1 -- 172.21.15.81:0/2480908516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa180d1a30 msgr2=0x7faa181f3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:29.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.440+0000 7faa1759e700 1 --2- 172.21.15.81:0/2480908516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa180d1a30 0x7faa181f3e70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faa0800e9d0 tx=0x7faa0800ece0 comp rx=0 tx=0).stop 2024-05-08T23:31:29.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.440+0000 7faa1759e700 1 -- 172.21.15.81:0/2480908516 shutdown_connections 2024-05-08T23:31:29.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.440+0000 7faa1759e700 1 --2- 172.21.15.81:0/2480908516 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa9f8066730 0x7fa9f8068bf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:29.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.440+0000 7faa1759e700 1 --2- 172.21.15.81:0/2480908516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1815d300 0x7faa181f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:29.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.440+0000 7faa1759e700 1 --2- 172.21.15.81:0/2480908516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa180d1a30 0x7faa181f3e70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:29.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.440+0000 7faa1759e700 1 --2- 172.21.15.81:0/2480908516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa180d02b0 0x7faa180cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:29.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.440+0000 7faa1759e700 1 -- 172.21.15.81:0/2480908516 >> 172.21.15.81:0/2480908516 conn(0x7faa1809c3b0 msgr2=0x7faa1806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:29.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.440+0000 7faa1759e700 1 -- 172.21.15.81:0/2480908516 shutdown_connections 2024-05-08T23:31:29.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.440+0000 7faa1759e700 1 -- 172.21.15.81:0/2480908516 wait complete. 2024-05-08T23:31:29.449 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' removed 2024-05-08T23:31:29.463 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T23:31:29.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.580+0000 7fde4bbe3700 1 Processor -- start 2024-05-08T23:31:29.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.580+0000 7fde4bbe3700 1 -- start start 2024-05-08T23:31:29.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4bbe3700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde4415d6f0 0x7fde440cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:29.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4bbe3700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fde440d3870 0x7fde4406b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:29.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4bbe3700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde440d21e0 con 0x7fde440d0f30 2024-05-08T23:31:29.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4bbe3700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde440ce5b0 con 0x7fde440d3870 2024-05-08T23:31:29.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4bbe3700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde440ce890 con 0x7fde4415d6f0 2024-05-08T23:31:29.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4a180700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fde440d3870 0x7fde4406b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:29.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4a180700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fde440d3870 0x7fde4406b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42634/0 (socket says 172.21.15.81:42634) 2024-05-08T23:31:29.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4917e700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fde440d0f30 0x7fde440d3160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:32822/0 (socket says 172.21.15.81:32822) 2024-05-08T23:31:29.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4a180700 1 -- 172.21.15.81:0/2332401998 learned_addr learned my addr 172.21.15.81:0/2332401998 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:29.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4997f700 1 --2- 172.21.15.81:0/2332401998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde4415d6f0 0x7fde440cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:29.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4897d700 1 -- 172.21.15.81:0/2332401998 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639208685 0 0) 0x7fde440d21e0 con 0x7fde440d0f30 2024-05-08T23:31:29.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4a180700 1 -- 172.21.15.81:0/2332401998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde4415d6f0 msgr2=0x7fde440cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:29.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4a180700 1 --2- 172.21.15.81:0/2332401998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde4415d6f0 0x7fde440cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:29.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4a180700 1 -- 172.21.15.81:0/2332401998 >> v1:172.21.15.81:6789/0 conn(0x7fde440d0f30 legacy=0x7fde440d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:29.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4a180700 1 -- 172.21.15.81:0/2332401998 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde440ceb70 con 0x7fde440d3870 2024-05-08T23:31:29.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4a180700 1 --2- 172.21.15.81:0/2332401998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440d3870 0x7fde4406b810 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009e50 tx=0x7fde3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863b73bab96a38a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:29.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4897d700 1 -- 172.21.15.81:0/2332401998 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c00e070 con 0x7fde440d3870 2024-05-08T23:31:29.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4897d700 1 -- 172.21.15.81:0/2332401998 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c00b910 con 0x7fde440d3870 2024-05-08T23:31:29.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4897d700 1 -- 172.21.15.81:0/2332401998 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c00bcc0 con 0x7fde440d3870 2024-05-08T23:31:29.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4bbe3700 1 -- 172.21.15.81:0/2332401998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440d3870 msgr2=0x7fde4406b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:29.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4bbe3700 1 --2- 172.21.15.81:0/2332401998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440d3870 0x7fde4406b810 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009e50 tx=0x7fde3c00b330 comp rx=0 tx=0).stop 2024-05-08T23:31:29.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4bbe3700 1 -- 172.21.15.81:0/2332401998 shutdown_connections 2024-05-08T23:31:29.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4bbe3700 1 --2- 172.21.15.81:0/2332401998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440d3870 0x7fde4406b810 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:29.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4bbe3700 1 --2- 172.21.15.81:0/2332401998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde4415d6f0 0x7fde440cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:29.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4bbe3700 1 -- 172.21.15.81:0/2332401998 >> 172.21.15.81:0/2332401998 conn(0x7fde4408b000 msgr2=0x7fde4409c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:29.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4bbe3700 1 -- 172.21.15.81:0/2332401998 shutdown_connections 2024-05-08T23:31:29.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.584+0000 7fde4bbe3700 1 -- 172.21.15.81:0/2332401998 wait complete. 2024-05-08T23:31:29.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.588+0000 7fde4bbe3700 1 Processor -- start 2024-05-08T23:31:29.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.588+0000 7fde4bbe3700 1 -- start start 2024-05-08T23:31:29.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4bbe3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440d0f30 0x7fde4415fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:29.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4bbe3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde440d3870 0x7fde440b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:29.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4bbe3700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde4415d6f0 0x7fde441f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:29.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4bbe3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440b82d0 con 0x7fde440d0f30 2024-05-08T23:31:29.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4bbe3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440ca4d0 con 0x7fde440d3870 2024-05-08T23:31:29.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4bbe3700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde440b8500 con 0x7fde4415d6f0 2024-05-08T23:31:29.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4997f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440d0f30 0x7fde4415fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:29.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4997f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440d0f30 0x7fde4415fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42644/0 (socket says 172.21.15.81:42644) 2024-05-08T23:31:29.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4997f700 1 -- 172.21.15.81:0/4123841370 learned_addr learned my addr 172.21.15.81:0/4123841370 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:29.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4a180700 1 --2- 172.21.15.81:0/4123841370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde4415d6f0 0x7fde441f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:29.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4917e700 1 --2- 172.21.15.81:0/4123841370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde440d3870 0x7fde440b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:29.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4997f700 1 -- 172.21.15.81:0/4123841370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde4415d6f0 msgr2=0x7fde441f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:29.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4997f700 1 --2- 172.21.15.81:0/4123841370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde4415d6f0 0x7fde441f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:29.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4997f700 1 -- 172.21.15.81:0/4123841370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde440d3870 msgr2=0x7fde440b7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:29.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4997f700 1 --2- 172.21.15.81:0/4123841370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde440d3870 0x7fde440b7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:29.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4997f700 1 -- 172.21.15.81:0/4123841370 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c009b30 con 0x7fde440d0f30 2024-05-08T23:31:29.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4997f700 1 --2- 172.21.15.81:0/4123841370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440d0f30 0x7fde4415fb30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fde3400e9d0 tx=0x7fde3400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:29.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde3a7fc700 1 -- 172.21.15.81:0/4123841370 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34009720 con 0x7fde440d0f30 2024-05-08T23:31:29.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde3a7fc700 1 -- 172.21.15.81:0/4123841370 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3400ee70 con 0x7fde440d0f30 2024-05-08T23:31:29.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde3a7fc700 1 -- 172.21.15.81:0/4123841370 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3400ca50 con 0x7fde440d0f30 2024-05-08T23:31:29.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4bbe3700 1 -- 172.21.15.81:0/4123841370 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4415dc50 con 0x7fde440d0f30 2024-05-08T23:31:29.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.592+0000 7fde4bbe3700 1 -- 172.21.15.81:0/4123841370 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde440caa50 con 0x7fde440d0f30 2024-05-08T23:31:29.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.596+0000 7fde4bbe3700 1 -- 172.21.15.81:0/4123841370 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde44068630 con 0x7fde440d0f30 2024-05-08T23:31:29.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.596+0000 7fde3a7fc700 1 -- 172.21.15.81:0/4123841370 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fde3400cbb0 con 0x7fde440d0f30 2024-05-08T23:31:29.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.596+0000 7fde3a7fc700 1 --2- 172.21.15.81:0/4123841370 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fde28066730 0x7fde28068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:29.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.596+0000 7fde4917e700 1 --2- 172.21.15.81:0/4123841370 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fde28066730 0x7fde28068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:29.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.596+0000 7fde3a7fc700 1 -- 172.21.15.81:0/4123841370 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5473+0+0 (secure 0 0 0) 0x7fde34083ff0 con 0x7fde440d0f30 2024-05-08T23:31:29.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.596+0000 7fde4917e700 1 --2- 172.21.15.81:0/4123841370 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fde28066730 0x7fde28068bf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fde441f74c0 tx=0x7fde40004a70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:29.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.604+0000 7fde3a7fc700 1 -- 172.21.15.81:0/4123841370 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fde340514d0 con 0x7fde440d0f30 2024-05-08T23:31:29.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:29.776+0000 7fde4bbe3700 1 -- 172.21.15.81:0/4123841370 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fde440b3010 con 0x7fde440d0f30 2024-05-08T23:31:30.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.440+0000 7fde3a7fc700 1 -- 172.21.15.81:0/4123841370 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7fde3400ce60 con 0x7fde440d0f30 2024-05-08T23:31:30.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.448+0000 7fde4bbe3700 1 -- 172.21.15.81:0/4123841370 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fde28066730 msgr2=0x7fde28068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:30.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.448+0000 7fde4bbe3700 1 --2- 172.21.15.81:0/4123841370 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fde28066730 0x7fde28068bf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fde441f74c0 tx=0x7fde40004a70 comp rx=0 tx=0).stop 2024-05-08T23:31:30.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.448+0000 7fde4bbe3700 1 -- 172.21.15.81:0/4123841370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440d0f30 msgr2=0x7fde4415fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:30.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.448+0000 7fde4bbe3700 1 --2- 172.21.15.81:0/4123841370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440d0f30 0x7fde4415fb30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fde3400e9d0 tx=0x7fde3400ece0 comp rx=0 tx=0).stop 2024-05-08T23:31:30.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.448+0000 7fde4bbe3700 1 -- 172.21.15.81:0/4123841370 shutdown_connections 2024-05-08T23:31:30.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.448+0000 7fde4bbe3700 1 --2- 172.21.15.81:0/4123841370 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fde28066730 0x7fde28068bf0 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fde441f74c0 tx=0x7fde40004a70 comp rx=0 tx=0).stop 2024-05-08T23:31:30.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.448+0000 7fde4bbe3700 1 --2- 172.21.15.81:0/4123841370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde4415d6f0 0x7fde441f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:30.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.448+0000 7fde4bbe3700 1 --2- 172.21.15.81:0/4123841370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde440d3870 0x7fde440b7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:30.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.448+0000 7fde4bbe3700 1 --2- 172.21.15.81:0/4123841370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440d0f30 0x7fde4415fb30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:30.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.448+0000 7fde4bbe3700 1 -- 172.21.15.81:0/4123841370 >> 172.21.15.81:0/4123841370 conn(0x7fde4408b000 msgr2=0x7fde440b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:30.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.448+0000 7fde4bbe3700 1 -- 172.21.15.81:0/4123841370 shutdown_connections 2024-05-08T23:31:30.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.448+0000 7fde4bbe3700 1 -- 172.21.15.81:0/4123841370 wait complete. 2024-05-08T23:31:30.454 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' removed 2024-05-08T23:31:30.469 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T23:31:30.469 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T23:31:30.469 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T23:31:30.469 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:31:30.469 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T23:31:30.469 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T23:31:30.470 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T23:31:30.470 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:31:30.470 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T23:31:30.470 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T23:31:30.470 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T23:31:30.470 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:31:30.470 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T23:31:30.471 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T23:31:30.471 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T23:31:30.471 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:31:30.480 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T23:31:30.480 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T23:31:30.482 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T23:31:30.482 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T23:31:30.483 INFO:tasks.ceph.mds.a:Started 2024-05-08T23:31:30.484 DEBUG:teuthology.parallel:result is None 2024-05-08T23:31:30.484 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T23:31:30.485 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T23:31:30.486 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T23:31:30.486 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T23:31:30.487 INFO:tasks.ceph.mds.b:Started 2024-05-08T23:31:30.487 INFO:tasks.ceph.mds.c:Started 2024-05-08T23:31:30.487 DEBUG:teuthology.parallel:result is None 2024-05-08T23:31:30.488 DEBUG:teuthology.parallel:result is None 2024-05-08T23:31:30.488 INFO:tasks.ceph.mds.d:Started 2024-05-08T23:31:30.488 DEBUG:teuthology.parallel:result is None 2024-05-08T23:31:30.488 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T23:31:30.551 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2024-05-08T23:31:30.565 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-05-08T23:31:30.567 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-05-08T23:31:30.586 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2024-05-08T23:31:30.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.604+0000 7fa90dbb4700 1 Processor -- start 2024-05-08T23:31:30.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.604+0000 7fa90dbb4700 1 -- start start 2024-05-08T23:31:30.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90dbb4700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa908143ca0 0x7fa9080ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:30.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90dbb4700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa90815f580 0x7fa9080b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:30.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90dbb4700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9080b47b0 0x7fa9080cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:30.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90dbb4700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9080b4c40 con 0x7fa908143ca0 2024-05-08T23:31:30.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90dbb4700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9080ceca0 con 0x7fa90815f580 2024-05-08T23:31:30.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90dbb4700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90815fb10 con 0x7fa9080b47b0 2024-05-08T23:31:30.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90cbb2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa908143ca0 0x7fa9080ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:30.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90cbb2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa908143ca0 0x7fa9080ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42676/0 (socket says 172.21.15.81:42676) 2024-05-08T23:31:30.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90cbb2700 1 -- 172.21.15.81:0/1496278416 learned_addr learned my addr 172.21.15.81:0/1496278416 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:30.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa907fff700 1 --2- 172.21.15.81:0/1496278416 >> v2:172.21.15.117:3300/0 conn(0x7fa90815f580 0x7fa9080b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:30.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90cbb2700 1 -- 172.21.15.81:0/1496278416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9080b47b0 msgr2=0x7fa9080cc350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:31:30.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90cbb2700 1 --2- 172.21.15.81:0/1496278416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9080b47b0 0x7fa9080cc350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:30.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90cbb2700 1 -- 172.21.15.81:0/1496278416 >> v2:172.21.15.117:3300/0 conn(0x7fa90815f580 msgr2=0x7fa9080b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:30.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90cbb2700 1 --2- 172.21.15.81:0/1496278416 >> v2:172.21.15.117:3300/0 conn(0x7fa90815f580 0x7fa9080b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:30.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90cbb2700 1 -- 172.21.15.81:0/1496278416 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90806c440 con 0x7fa908143ca0 2024-05-08T23:31:30.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa907fff700 1 --2- 172.21.15.81:0/1496278416 >> v2:172.21.15.117:3300/0 conn(0x7fa90815f580 0x7fa9080b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:30.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90cbb2700 1 --2- 172.21.15.81:0/1496278416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa908143ca0 0x7fa9080ce760 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa9080b5220 tx=0x7fa90000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c2ff50e906b4f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:30.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa9077fe700 1 -- 172.21.15.81:0/1496278416 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90000e070 con 0x7fa908143ca0 2024-05-08T23:31:30.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa9077fe700 1 -- 172.21.15.81:0/1496278416 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90000b9c0 con 0x7fa908143ca0 2024-05-08T23:31:30.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa9077fe700 1 -- 172.21.15.81:0/1496278416 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90000bd70 con 0x7fa908143ca0 2024-05-08T23:31:30.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90dbb4700 1 -- 172.21.15.81:0/1496278416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa908143ca0 msgr2=0x7fa9080ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:30.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90dbb4700 1 --2- 172.21.15.81:0/1496278416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa908143ca0 0x7fa9080ce760 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa9080b5220 tx=0x7fa90000b330 comp rx=0 tx=0).stop 2024-05-08T23:31:30.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90dbb4700 1 -- 172.21.15.81:0/1496278416 shutdown_connections 2024-05-08T23:31:30.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90dbb4700 1 --2- 172.21.15.81:0/1496278416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9080b47b0 0x7fa9080cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:30.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90dbb4700 1 --2- 172.21.15.81:0/1496278416 >> v2:172.21.15.117:3300/0 conn(0x7fa90815f580 0x7fa9080b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:30.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90dbb4700 1 --2- 172.21.15.81:0/1496278416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa908143ca0 0x7fa9080ce760 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:30.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90dbb4700 1 -- 172.21.15.81:0/1496278416 >> 172.21.15.81:0/1496278416 conn(0x7fa9080aebd0 msgr2=0x7fa9080aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:30.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90dbb4700 1 -- 172.21.15.81:0/1496278416 shutdown_connections 2024-05-08T23:31:30.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.608+0000 7fa90dbb4700 1 -- 172.21.15.81:0/1496278416 wait complete. 2024-05-08T23:31:30.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90dbb4700 1 Processor -- start 2024-05-08T23:31:30.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90dbb4700 1 -- start start 2024-05-08T23:31:30.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90dbb4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9080b47b0 0x7fa90815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:30.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90dbb4700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa908143ca0 0x7fa90815bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:30.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90dbb4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa90815f580 0x7fa90815c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:30.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90dbb4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908143080 con 0x7fa90815f580 2024-05-08T23:31:30.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90dbb4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9081f2b80 con 0x7fa9080b47b0 2024-05-08T23:31:30.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90dbb4700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9081f2e60 con 0x7fa908143ca0 2024-05-08T23:31:30.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90d3b3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa90815f580 0x7fa90815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:30.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90d3b3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa90815f580 0x7fa90815c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42688/0 (socket says 172.21.15.81:42688) 2024-05-08T23:31:30.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90d3b3700 1 -- 172.21.15.81:0/2322550882 learned_addr learned my addr 172.21.15.81:0/2322550882 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:30.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90cbb2700 1 --2- 172.21.15.81:0/2322550882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9080b47b0 0x7fa90815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:30.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa907fff700 1 --2- 172.21.15.81:0/2322550882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa908143ca0 0x7fa90815bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:30.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90d3b3700 1 -- 172.21.15.81:0/2322550882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa908143ca0 msgr2=0x7fa90815bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:30.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90d3b3700 1 --2- 172.21.15.81:0/2322550882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa908143ca0 0x7fa90815bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:30.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90d3b3700 1 -- 172.21.15.81:0/2322550882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9080b47b0 msgr2=0x7fa90815a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:30.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90d3b3700 1 --2- 172.21.15.81:0/2322550882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9080b47b0 0x7fa90815a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:30.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90d3b3700 1 -- 172.21.15.81:0/2322550882 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa900009b30 con 0x7fa90815f580 2024-05-08T23:31:30.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90d3b3700 1 --2- 172.21.15.81:0/2322550882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa90815f580 0x7fa90815c090 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800b410 tx=0x7fa8f800b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:30.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa9057fa700 1 -- 172.21.15.81:0/2322550882 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8011830 con 0x7fa90815f580 2024-05-08T23:31:30.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa9057fa700 1 -- 172.21.15.81:0/2322550882 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f80119b0 con 0x7fa90815f580 2024-05-08T23:31:30.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90dbb4700 1 -- 172.21.15.81:0/2322550882 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9081f3140 con 0x7fa90815f580 2024-05-08T23:31:30.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa9057fa700 1 -- 172.21.15.81:0/2322550882 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8011d60 con 0x7fa90815f580 2024-05-08T23:31:30.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.612+0000 7fa90dbb4700 1 -- 172.21.15.81:0/2322550882 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9081f3700 con 0x7fa90815f580 2024-05-08T23:31:30.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.616+0000 7fa90dbb4700 1 -- 172.21.15.81:0/2322550882 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9080684b0 con 0x7fa90815f580 2024-05-08T23:31:30.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.616+0000 7fa9057fa700 1 -- 172.21.15.81:0/2322550882 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa8f801a400 con 0x7fa90815f580 2024-05-08T23:31:30.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.616+0000 7fa9057fa700 1 --2- 172.21.15.81:0/2322550882 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa8e8066730 0x7fa8e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:30.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.616+0000 7fa9057fa700 1 -- 172.21.15.81:0/2322550882 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa8f8083e60 con 0x7fa90815f580 2024-05-08T23:31:30.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.616+0000 7fa90cbb2700 1 --2- 172.21.15.81:0/2322550882 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa8e8066730 0x7fa8e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:30.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.616+0000 7fa90cbb2700 1 --2- 172.21.15.81:0/2322550882 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa8e8066730 0x7fa8e8068bf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa90000b370 tx=0x7fa900009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:30.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.620+0000 7fa9057fa700 1 -- 172.21.15.81:0/2322550882 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa8f80514d0 con 0x7fa90815f580 2024-05-08T23:31:30.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:30.796+0000 7fa90dbb4700 1 -- 172.21.15.81:0/2322550882 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa9080cbb70 con 0x7fa90815f580 2024-05-08T23:31:31.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.616+0000 7fa9057fa700 1 -- 172.21.15.81:0/2322550882 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa8f8015080 con 0x7fa90815f580 2024-05-08T23:31:31.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.620+0000 7fa90dbb4700 1 -- 172.21.15.81:0/2322550882 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa8e8066730 msgr2=0x7fa8e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:31.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.620+0000 7fa90dbb4700 1 --2- 172.21.15.81:0/2322550882 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa8e8066730 0x7fa8e8068bf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa90000b370 tx=0x7fa900009000 comp rx=0 tx=0).stop 2024-05-08T23:31:31.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.620+0000 7fa90dbb4700 1 -- 172.21.15.81:0/2322550882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa90815f580 msgr2=0x7fa90815c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:31.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.620+0000 7fa90dbb4700 1 --2- 172.21.15.81:0/2322550882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa90815f580 0x7fa90815c090 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800b410 tx=0x7fa8f800b7d0 comp rx=0 tx=0).stop 2024-05-08T23:31:31.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.620+0000 7fa90dbb4700 1 -- 172.21.15.81:0/2322550882 shutdown_connections 2024-05-08T23:31:31.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.620+0000 7fa90dbb4700 1 --2- 172.21.15.81:0/2322550882 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa8e8066730 0x7fa8e8068bf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:31.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.620+0000 7fa90dbb4700 1 --2- 172.21.15.81:0/2322550882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa90815f580 0x7fa90815c090 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:31.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.620+0000 7fa90dbb4700 1 --2- 172.21.15.81:0/2322550882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa908143ca0 0x7fa90815bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:31.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.620+0000 7fa90dbb4700 1 --2- 172.21.15.81:0/2322550882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9080b47b0 0x7fa90815a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:31.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.620+0000 7fa90dbb4700 1 -- 172.21.15.81:0/2322550882 >> 172.21.15.81:0/2322550882 conn(0x7fa9080aebd0 msgr2=0x7fa9080700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:31.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.620+0000 7fa90dbb4700 1 -- 172.21.15.81:0/2322550882 shutdown_connections 2024-05-08T23:31:31.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.620+0000 7fa90dbb4700 1 -- 172.21.15.81:0/2322550882 wait complete. 2024-05-08T23:31:31.628 INFO:teuthology.orchestra.run.smithi081.stderr: removed all blocklist entries 2024-05-08T23:31:31.642 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T23:31:31.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.752+0000 7fe3a633f700 1 Processor -- start 2024-05-08T23:31:31.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.752+0000 7fe3a633f700 1 -- start start 2024-05-08T23:31:31.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a633f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3a00736c0 0x7fe3a00ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:31.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a633f700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe3a00ae570 0x7fe3a006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:31.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a633f700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a00aca10 con 0x7fe3a01577f0 2024-05-08T23:31:31.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a633f700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a00aea00 con 0x7fe3a00ae570 2024-05-08T23:31:31.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a633f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00accf0 con 0x7fe3a00736c0 2024-05-08T23:31:31.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a5b3e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe3a00ae570 0x7fe3a006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:31.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a5b3e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe3a00ae570 0x7fe3a006d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42696/0 (socket says 172.21.15.81:42696) 2024-05-08T23:31:31.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a5b3e700 1 -- 172.21.15.81:0/1221648436 learned_addr learned my addr 172.21.15.81:0/1221648436 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:31.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a4b3c700 1 --2- 172.21.15.81:0/1221648436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3a00736c0 0x7fe3a00ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:31.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a5b3e700 1 -- 172.21.15.81:0/1221648436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3a00736c0 msgr2=0x7fe3a00ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:31.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a5b3e700 1 --2- 172.21.15.81:0/1221648436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3a00736c0 0x7fe3a00ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:31.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a5b3e700 1 -- 172.21.15.81:0/1221648436 >> v1:172.21.15.117:6789/0 conn(0x7fe3a01577f0 legacy=0x7fe3a015e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:31.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a5b3e700 1 -- 172.21.15.81:0/1221648436 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a00acfd0 con 0x7fe3a00ae570 2024-05-08T23:31:31.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a5b3e700 1 --2- 172.21.15.81:0/1221648436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a00ae570 0x7fe3a006d8d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe398009e50 tx=0x7fe39800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487f89beb85b9d87 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:31.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe393fff700 1 -- 172.21.15.81:0/1221648436 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39800e070 con 0x7fe3a00ae570 2024-05-08T23:31:31.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe393fff700 1 -- 172.21.15.81:0/1221648436 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39800b910 con 0x7fe3a00ae570 2024-05-08T23:31:31.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe393fff700 1 -- 172.21.15.81:0/1221648436 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39800bcc0 con 0x7fe3a00ae570 2024-05-08T23:31:31.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a633f700 1 -- 172.21.15.81:0/1221648436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a00ae570 msgr2=0x7fe3a006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:31.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a633f700 1 --2- 172.21.15.81:0/1221648436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a00ae570 0x7fe3a006d8d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe398009e50 tx=0x7fe39800b330 comp rx=0 tx=0).stop 2024-05-08T23:31:31.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a633f700 1 -- 172.21.15.81:0/1221648436 shutdown_connections 2024-05-08T23:31:31.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a633f700 1 --2- 172.21.15.81:0/1221648436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a00ae570 0x7fe3a006d8d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:31.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a633f700 1 --2- 172.21.15.81:0/1221648436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3a00736c0 0x7fe3a00ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:31.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a633f700 1 -- 172.21.15.81:0/1221648436 >> 172.21.15.81:0/1221648436 conn(0x7fe3a00717e0 msgr2=0x7fe3a006a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:31.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a633f700 1 -- 172.21.15.81:0/1221648436 shutdown_connections 2024-05-08T23:31:31.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.756+0000 7fe3a633f700 1 -- 172.21.15.81:0/1221648436 wait complete. 2024-05-08T23:31:31.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a633f700 1 Processor -- start 2024-05-08T23:31:31.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a633f700 1 -- start start 2024-05-08T23:31:31.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a633f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3a00736c0 0x7fe3a0150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:31.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a633f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3a00ae570 0x7fe3a015b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:31.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a633f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a01577f0 0x7fe3a015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:31.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a633f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0150ab0 con 0x7fe3a01577f0 2024-05-08T23:31:31.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a633f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a01f2fa0 con 0x7fe3a00736c0 2024-05-08T23:31:31.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a633f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a015bff0 con 0x7fe3a00ae570 2024-05-08T23:31:31.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a5b3e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a01577f0 0x7fe3a015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:31.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a4b3c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3a00ae570 0x7fe3a015b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:31.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a5b3e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a01577f0 0x7fe3a015b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42708/0 (socket says 172.21.15.81:42708) 2024-05-08T23:31:31.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a5b3e700 1 -- 172.21.15.81:0/2729629403 learned_addr learned my addr 172.21.15.81:0/2729629403 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:31.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a533d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3a00736c0 0x7fe3a0150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:31.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a5b3e700 1 -- 172.21.15.81:0/2729629403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3a00ae570 msgr2=0x7fe3a015b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:31.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a5b3e700 1 --2- 172.21.15.81:0/2729629403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3a00ae570 0x7fe3a015b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:31.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a5b3e700 1 -- 172.21.15.81:0/2729629403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3a00736c0 msgr2=0x7fe3a0150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:31.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a5b3e700 1 --2- 172.21.15.81:0/2729629403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3a00736c0 0x7fe3a0150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:31.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a5b3e700 1 -- 172.21.15.81:0/2729629403 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe398009b30 con 0x7fe3a01577f0 2024-05-08T23:31:31.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe3a5b3e700 1 --2- 172.21.15.81:0/2729629403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a01577f0 0x7fe3a015b9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe3a006c7f0 tx=0x7fe398004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:31.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.760+0000 7fe391ffb700 1 -- 172.21.15.81:0/2729629403 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980048c0 con 0x7fe3a01577f0 2024-05-08T23:31:31.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.764+0000 7fe391ffb700 1 -- 172.21.15.81:0/2729629403 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3980099b0 con 0x7fe3a01577f0 2024-05-08T23:31:31.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.764+0000 7fe3a633f700 1 -- 172.21.15.81:0/2729629403 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a015c2d0 con 0x7fe3a01577f0 2024-05-08T23:31:31.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.764+0000 7fe391ffb700 1 -- 172.21.15.81:0/2729629403 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39801b670 con 0x7fe3a01577f0 2024-05-08T23:31:31.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.764+0000 7fe3a633f700 1 -- 172.21.15.81:0/2729629403 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a015c890 con 0x7fe3a01577f0 2024-05-08T23:31:31.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.764+0000 7fe3a633f700 1 -- 172.21.15.81:0/2729629403 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a0067e40 con 0x7fe3a01577f0 2024-05-08T23:31:31.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.764+0000 7fe391ffb700 1 -- 172.21.15.81:0/2729629403 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe39801b7d0 con 0x7fe3a01577f0 2024-05-08T23:31:31.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.768+0000 7fe391ffb700 1 --2- 172.21.15.81:0/2729629403 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe384066760 0x7fe384068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:31.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.768+0000 7fe3a533d700 1 --2- 172.21.15.81:0/2729629403 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe384066760 0x7fe384068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:31.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.768+0000 7fe391ffb700 1 -- 172.21.15.81:0/2729629403 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe398085560 con 0x7fe3a01577f0 2024-05-08T23:31:31.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.768+0000 7fe3a533d700 1 --2- 172.21.15.81:0/2729629403 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe384066760 0x7fe384068c20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe3a006d0b0 tx=0x7fe39c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:31.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:31.772+0000 7fe391ffb700 1 -- 172.21.15.81:0/2729629403 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe398052cb0 con 0x7fe3a01577f0 2024-05-08T23:31:32.005 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.000+0000 7fe3a633f700 1 -- 172.21.15.81:0/2729629403 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe3a00672a0 con 0x7fe3a01577f0 2024-05-08T23:31:32.005 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.000+0000 7fe391ffb700 1 -- 172.21.15.81:0/2729629403 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v3) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fe3980562d0 con 0x7fe3a01577f0 2024-05-08T23:31:32.006 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:32.006 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:31:32.006 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_dump": [ 2024-05-08T23:31:32.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.006 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.a", 2024-05-08T23:31:32.006 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmmtpFBBAAO8W7v1ZRGwo4hFj/fYPECw==", 2024-05-08T23:31:32.006 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.006 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:31:32.007 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2024-05-08T23:31:32.007 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2024-05-08T23:31:32.007 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:31:32.007 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.007 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.b", 2024-05-08T23:31:32.007 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxms5vJCxAA7vARVP0vvogEEps6DJFK1g==", 2024-05-08T23:31:32.007 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.007 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:31:32.007 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2024-05-08T23:31:32.007 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2024-05-08T23:31:32.007 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:31:32.007 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.008 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.c", 2024-05-08T23:31:32.008 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmXD37BxAALgQN2Yixer9fPYj2YdLW/w==", 2024-05-08T23:31:32.008 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.008 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:31:32.008 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2024-05-08T23:31:32.008 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2024-05-08T23:31:32.008 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:31:32.008 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.008 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.d", 2024-05-08T23:31:32.008 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmpIKFDxAACxYpOsx1laPsl0QNG+NFdw==", 2024-05-08T23:31:32.008 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.008 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:31:32.009 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2024-05-08T23:31:32.009 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2024-05-08T23:31:32.009 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:31:32.009 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.009 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.0", 2024-05-08T23:31:32.009 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDkCjxmUOfLCBAAc0gYW9BV8BE+h7Qhhc01IQ==", 2024-05-08T23:31:32.009 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.009 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:31:32.009 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:31:32.009 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:31:32.009 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.010 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.1", 2024-05-08T23:31:32.010 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDnCjxmKcSHDBAA5dF6MWsChaZUZQskZYtrKg==", 2024-05-08T23:31:32.010 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.010 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:31:32.010 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:31:32.010 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:31:32.011 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.012 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.2", 2024-05-08T23:31:32.012 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDqCjxmeYjnEhAAAVMSJ8jGR+dThJL51VsQxA==", 2024-05-08T23:31:32.012 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.012 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:31:32.012 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:31:32.012 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:31:32.012 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.012 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.3", 2024-05-08T23:31:32.012 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDtCjxmlPycFhAAx+amZqgwyvwydi2d5dBZmg==", 2024-05-08T23:31:32.012 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:31:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:31:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:31:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.4", 2024-05-08T23:31:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDyCjxmcnu+NxAAJ5cn9LG2uTjAZzW2Qv2y5A==", 2024-05-08T23:31:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:31:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:31:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:31:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.014 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.5", 2024-05-08T23:31:32.014 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQD2CjxmQNJoARAAfINQxQDKESMqyuxmEFwwJA==", 2024-05-08T23:31:32.014 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.014 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:31:32.014 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:31:32.014 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:31:32.014 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.014 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.6", 2024-05-08T23:31:32.014 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQD5CjxmGcKCCRAApkAe4V2JvttFUqcwVa7PbA==", 2024-05-08T23:31:32.014 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.014 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:31:32.014 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.7", 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQD8CjxmIbmICxAAqF8ZE7Gxj1pLKuaCvqOIRw==", 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.0", 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmQWD3EhAAkS5P2FRmiLBI9qjk3JjiyA==", 2024-05-08T23:31:32.015 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.016 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:31:32.016 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2024-05-08T23:31:32.016 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2024-05-08T23:31:32.016 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2024-05-08T23:31:32.016 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.016 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.1", 2024-05-08T23:31:32.016 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmuQBJGRAAA7KQ8CtfMfkVgM89n1QBMA==", 2024-05-08T23:31:32.016 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.016 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:31:32.016 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2024-05-08T23:31:32.016 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2024-05-08T23:31:32.016 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2024-05-08T23:31:32.016 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.2", 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmhpPVFRAA2gIDHl4KehKcgXKX0Rs9tg==", 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.3", 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmf6gIHxAARsUHPVNk3HzYs3lQyqt2Cg==", 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.017 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:31:32.018 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2024-05-08T23:31:32.018 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2024-05-08T23:31:32.018 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2024-05-08T23:31:32.018 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.018 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.admin", 2024-05-08T23:31:32.018 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDgCjxmh+ZhMBAA8ifuuRdCV2SQaSOLTWhPsw==", 2024-05-08T23:31:32.018 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.018 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2024-05-08T23:31:32.018 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2024-05-08T23:31:32.018 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2024-05-08T23:31:32.018 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:31:32.018 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.019 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.bootstrap-mds", 2024-05-08T23:31:32.019 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGCzxm7tovEBAAzUa/MX4qk6DUjL4GmrvJWA==", 2024-05-08T23:31:32.019 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.019 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile bootstrap-mds" 2024-05-08T23:31:32.019 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.021 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.bootstrap-mgr", 2024-05-08T23:31:32.021 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGCzxmQPMvEBAAEw4BGo4WGST4rwQo755LIA==", 2024-05-08T23:31:32.021 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.021 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-08T23:31:32.021 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.021 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.bootstrap-osd", 2024-05-08T23:31:32.022 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGCzxmNAkwEBAAFrI6AkihAIDiMOCxIqvehw==", 2024-05-08T23:31:32.022 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.022 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile bootstrap-osd" 2024-05-08T23:31:32.022 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.022 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.bootstrap-rbd", 2024-05-08T23:31:32.022 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGCzxmNSQwEBAAMxDhRpeEzWOoHLLZA0eQEw==", 2024-05-08T23:31:32.022 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.022 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-08T23:31:32.022 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.022 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-08T23:31:32.023 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGCzxmUTowEBAA4BlVma4K2yYkL+tcGPGWIg==", 2024-05-08T23:31:32.023 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.023 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-08T23:31:32.023 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.023 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.bootstrap-rgw", 2024-05-08T23:31:32.023 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGCzxmyFEwEBAANbz16ONadt2eN4eBvnbhgQ==", 2024-05-08T23:31:32.023 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.023 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-08T23:31:32.023 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.023 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mgr.x", 2024-05-08T23:31:32.024 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmUtdWABAARBlm/xpvB6mlamPG1+thYg==", 2024-05-08T23:31:32.024 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.024 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2024-05-08T23:31:32.024 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile mgr", 2024-05-08T23:31:32.024 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:31:32.024 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:32.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:32.024 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mgr.y", 2024-05-08T23:31:32.024 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDgCjxmgG3AOBAAK6T63BGID2I5zE0Z3VR4TQ==", 2024-05-08T23:31:32.024 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:31:32.024 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2024-05-08T23:31:32.024 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile mgr", 2024-05-08T23:31:32.024 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:31:32.024 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.025 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:32.025 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-05-08T23:31:32.025 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:31:32.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.004+0000 7fe3a633f700 1 -- 172.21.15.81:0/2729629403 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe384066760 msgr2=0x7fe384068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.004+0000 7fe3a633f700 1 --2- 172.21.15.81:0/2729629403 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe384066760 0x7fe384068c20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe3a006d0b0 tx=0x7fe39c009040 comp rx=0 tx=0).stop 2024-05-08T23:31:32.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.004+0000 7fe3a633f700 1 -- 172.21.15.81:0/2729629403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a01577f0 msgr2=0x7fe3a015b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.004+0000 7fe3a633f700 1 --2- 172.21.15.81:0/2729629403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a01577f0 0x7fe3a015b9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe3a006c7f0 tx=0x7fe398004af0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.004+0000 7fe3a633f700 1 -- 172.21.15.81:0/2729629403 shutdown_connections 2024-05-08T23:31:32.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.004+0000 7fe3a633f700 1 --2- 172.21.15.81:0/2729629403 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe384066760 0x7fe384068c20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.004+0000 7fe3a633f700 1 --2- 172.21.15.81:0/2729629403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3a01577f0 0x7fe3a015b9c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.004+0000 7fe3a633f700 1 --2- 172.21.15.81:0/2729629403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3a00ae570 0x7fe3a015b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.004+0000 7fe3a633f700 1 --2- 172.21.15.81:0/2729629403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3a00736c0 0x7fe3a0150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.004+0000 7fe3a633f700 1 -- 172.21.15.81:0/2729629403 >> 172.21.15.81:0/2729629403 conn(0x7fe3a00717e0 msgr2=0x7fe3a00ce940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:32.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.004+0000 7fe3a633f700 1 -- 172.21.15.81:0/2729629403 shutdown_connections 2024-05-08T23:31:32.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.004+0000 7fe3a633f700 1 -- 172.21.15.81:0/2729629403 wait complete. 2024-05-08T23:31:32.028 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-08T23:31:32.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.136+0000 7f52da445700 1 Processor -- start 2024-05-08T23:31:32.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.136+0000 7f52da445700 1 -- start start 2024-05-08T23:31:32.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.136+0000 7f52da445700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f52d415afa0 0x7f52d415b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.136+0000 7f52da445700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d415eda0 con 0x7f52d415b8c0 2024-05-08T23:31:32.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.136+0000 7f52da445700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d415ea70 con 0x7f52d40ce990 2024-05-08T23:31:32.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.136+0000 7f52da445700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d40ced70 con 0x7f52d415afa0 2024-05-08T23:31:32.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.136+0000 7f52d8c42700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f52d415b8c0 0x7f52d415f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:32834/0 (socket says 172.21.15.81:32834) 2024-05-08T23:31:32.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.136+0000 7f52d8c42700 1 -- 172.21.15.81:0/1687989390 learned_addr learned my addr 172.21.15.81:0/1687989390 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:32.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52d9443700 1 --2- 172.21.15.81:0/1687989390 >> v2:172.21.15.117:3300/0 conn(0x7f52d415afa0 0x7f52d415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 -- 172.21.15.81:0/1687989390 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392322809 0 0) 0x7f52d415eda0 con 0x7f52d415b8c0 2024-05-08T23:31:32.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 -- 172.21.15.81:0/1687989390 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52bc0035d0 con 0x7f52d415b8c0 2024-05-08T23:31:32.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 -- 172.21.15.81:0/1687989390 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 458926306 0 0) 0x7f52bc0035d0 con 0x7f52d415b8c0 2024-05-08T23:31:32.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 -- 172.21.15.81:0/1687989390 >> v2:172.21.15.117:3300/0 conn(0x7f52d415afa0 msgr2=0x7f52d415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:32.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 --2- 172.21.15.81:0/1687989390 >> v2:172.21.15.117:3300/0 conn(0x7f52d415afa0 0x7f52d415b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 -- 172.21.15.81:0/1687989390 >> v1:172.21.15.117:6789/0 conn(0x7f52d40ce990 legacy=0x7f52d40b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 -- 172.21.15.81:0/1687989390 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d415bd40 con 0x7f52d415b8c0 2024-05-08T23:31:32.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 -- 172.21.15.81:0/1687989390 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f52d00019f0 con 0x7f52d415b8c0 2024-05-08T23:31:32.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 -- 172.21.15.81:0/1687989390 >> v1:172.21.15.81:6789/0 conn(0x7f52d415b8c0 legacy=0x7f52d415f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 --2- 172.21.15.81:0/1687989390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52bc003a20 0x7f52bc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 --2- 172.21.15.81:0/1687989390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52bc004870 0x7f52bc004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 --2- 172.21.15.81:0/1687989390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52bc005230 0x7f52bc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 -- 172.21.15.81:0/1687989390 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40ced70 con 0x7f52bc005230 2024-05-08T23:31:32.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 -- 172.21.15.81:0/1687989390 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc00c0d0 con 0x7f52bc003a20 2024-05-08T23:31:32.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 -- 172.21.15.81:0/1687989390 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc00c290 con 0x7f52bc004870 2024-05-08T23:31:32.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52d9c44700 1 --2- 172.21.15.81:0/1687989390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52bc005230 0x7f52bc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52d9443700 1 --2- 172.21.15.81:0/1687989390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52bc003a20 0x7f52bc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52d8c42700 1 --2- 172.21.15.81:0/1687989390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52bc004870 0x7f52bc004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52d9c44700 1 -- 172.21.15.81:0/1687989390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52bc004870 msgr2=0x7f52bc004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:32.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52d9c44700 1 --2- 172.21.15.81:0/1687989390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52bc004870 0x7f52bc004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52d9c44700 1 -- 172.21.15.81:0/1687989390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52bc003a20 msgr2=0x7f52bc003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:32.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52d9c44700 1 --2- 172.21.15.81:0/1687989390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52bc003a20 0x7f52bc003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52d9c44700 1 -- 172.21.15.81:0/1687989390 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc0035d0 con 0x7f52bc005230 2024-05-08T23:31:32.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52d9443700 1 --2- 172.21.15.81:0/1687989390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52bc003a20 0x7f52bc003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:32.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52d9c44700 1 -- 172.21.15.81:0/1687989390 reap_dead start 2024-05-08T23:31:32.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52d9c44700 1 --2- 172.21.15.81:0/1687989390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52bc005230 0x7f52bc00ba30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0017e0 tx=0x7f52bc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba0afd6166fb0a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:32.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 -- 172.21.15.81:0/1687989390 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0052d0 con 0x7f52bc005230 2024-05-08T23:31:32.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 -- 172.21.15.81:0/1687989390 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52cc005430 con 0x7f52bc005230 2024-05-08T23:31:32.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52cbfff700 1 -- 172.21.15.81:0/1687989390 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0055a0 con 0x7f52bc005230 2024-05-08T23:31:32.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52da445700 1 -- 172.21.15.81:0/1687989390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52bc005230 msgr2=0x7f52bc00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52da445700 1 --2- 172.21.15.81:0/1687989390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52bc005230 0x7f52bc00ba30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0017e0 tx=0x7f52bc0058a0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52da445700 1 -- 172.21.15.81:0/1687989390 shutdown_connections 2024-05-08T23:31:32.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52da445700 1 --2- 172.21.15.81:0/1687989390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52bc005230 0x7f52bc00ba30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52da445700 1 -- 172.21.15.81:0/1687989390 >> 172.21.15.81:0/1687989390 conn(0x7f52d406c330 msgr2=0x7f52d406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:32.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52da445700 1 -- 172.21.15.81:0/1687989390 shutdown_connections 2024-05-08T23:31:32.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.140+0000 7f52da445700 1 -- 172.21.15.81:0/1687989390 wait complete. 2024-05-08T23:31:32.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52da445700 1 Processor -- start 2024-05-08T23:31:32.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52da445700 1 -- start start 2024-05-08T23:31:32.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52da445700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52bc005230 0x7f52d4142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52da445700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52d414fad0 0x7f52d41e4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52da445700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52d41e4690 0x7f52d41e6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52da445700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41e5740 con 0x7f52d41e4690 2024-05-08T23:31:32.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52da445700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41e4b30 con 0x7f52d414fad0 2024-05-08T23:31:32.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52da445700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41e4e10 con 0x7f52bc005230 2024-05-08T23:31:32.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52d9c44700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52d41e4690 0x7f52d41e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52d9c44700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52d41e4690 0x7f52d41e6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42720/0 (socket says 172.21.15.81:42720) 2024-05-08T23:31:32.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52d9c44700 1 -- 172.21.15.81:0/723045681 learned_addr learned my addr 172.21.15.81:0/723045681 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:32.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52d9443700 1 --2- 172.21.15.81:0/723045681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52bc005230 0x7f52d4142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52d8c42700 1 --2- 172.21.15.81:0/723045681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52d414fad0 0x7f52d41e4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52d9c44700 1 -- 172.21.15.81:0/723045681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52bc005230 msgr2=0x7f52d4142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52d9c44700 1 --2- 172.21.15.81:0/723045681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52bc005230 0x7f52d4142840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52d9c44700 1 -- 172.21.15.81:0/723045681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52d414fad0 msgr2=0x7f52d41e4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52d9c44700 1 --2- 172.21.15.81:0/723045681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52d414fad0 0x7f52d41e4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52d9c44700 1 -- 172.21.15.81:0/723045681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d41e50f0 con 0x7f52d41e4690 2024-05-08T23:31:32.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52d9c44700 1 --2- 172.21.15.81:0/723045681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52d41e4690 0x7f52d41e6440 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f52d40712a0 tx=0x7f52cc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:32.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52c9ffb700 1 -- 172.21.15.81:0/723045681 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0055a0 con 0x7f52d41e4690 2024-05-08T23:31:32.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52c9ffb700 1 -- 172.21.15.81:0/723045681 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52cc003e80 con 0x7f52d41e4690 2024-05-08T23:31:32.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52da445700 1 -- 172.21.15.81:0/723045681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d41e53d0 con 0x7f52d41e4690 2024-05-08T23:31:32.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52c9ffb700 1 -- 172.21.15.81:0/723045681 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc01c660 con 0x7f52d41e4690 2024-05-08T23:31:32.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.144+0000 7f52da445700 1 -- 172.21.15.81:0/723045681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d41e5bd0 con 0x7f52d41e4690 2024-05-08T23:31:32.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.148+0000 7f52da445700 1 -- 172.21.15.81:0/723045681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d41e58b0 con 0x7f52d41e4690 2024-05-08T23:31:32.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.148+0000 7f52c9ffb700 1 -- 172.21.15.81:0/723045681 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f52cc01c7c0 con 0x7f52d41e4690 2024-05-08T23:31:32.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.152+0000 7f52c9ffb700 1 --2- 172.21.15.81:0/723045681 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f52c0066730 0x7f52c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.152+0000 7f52c9ffb700 1 -- 172.21.15.81:0/723045681 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f52cc084c80 con 0x7f52d41e4690 2024-05-08T23:31:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.152+0000 7f52d9443700 1 --2- 172.21.15.81:0/723045681 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f52c0066730 0x7f52c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.152+0000 7f52d9443700 1 --2- 172.21.15.81:0/723045681 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f52c0066730 0x7f52c0068bf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f52d41f4010 tx=0x7f52c4009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:32.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.156+0000 7f52c9ffb700 1 -- 172.21.15.81:0/723045681 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f52cc052470 con 0x7f52d41e4690 2024-05-08T23:31:32.392 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.388+0000 7f52da445700 1 -- 172.21.15.81:0/723045681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f52d40684b0 con 0x7f52d41e4690 2024-05-08T23:31:32.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.392+0000 7f52c9ffb700 1 -- 172.21.15.81:0/723045681 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f52cc0160a0 con 0x7f52d41e4690 2024-05-08T23:31:32.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.396+0000 7f52da445700 1 -- 172.21.15.81:0/723045681 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f52c0066730 msgr2=0x7f52c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.396+0000 7f52da445700 1 --2- 172.21.15.81:0/723045681 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f52c0066730 0x7f52c0068bf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f52d41f4010 tx=0x7f52c4009240 comp rx=0 tx=0).stop 2024-05-08T23:31:32.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.396+0000 7f52da445700 1 -- 172.21.15.81:0/723045681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52d41e4690 msgr2=0x7f52d41e6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.396+0000 7f52da445700 1 --2- 172.21.15.81:0/723045681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52d41e4690 0x7f52d41e6440 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f52d40712a0 tx=0x7f52cc001120 comp rx=0 tx=0).stop 2024-05-08T23:31:32.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.396+0000 7f52da445700 1 -- 172.21.15.81:0/723045681 shutdown_connections 2024-05-08T23:31:32.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.396+0000 7f52da445700 1 --2- 172.21.15.81:0/723045681 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f52c0066730 0x7f52c0068bf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.396+0000 7f52da445700 1 --2- 172.21.15.81:0/723045681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52d41e4690 0x7f52d41e6440 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.396+0000 7f52da445700 1 --2- 172.21.15.81:0/723045681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52d414fad0 0x7f52d41e4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.396+0000 7f52da445700 1 --2- 172.21.15.81:0/723045681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52bc005230 0x7f52d4142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.396+0000 7f52da445700 1 -- 172.21.15.81:0/723045681 >> 172.21.15.81:0/723045681 conn(0x7f52d406c330 msgr2=0x7f52d40aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:32.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.396+0000 7f52da445700 1 -- 172.21.15.81:0/723045681 shutdown_connections 2024-05-08T23:31:32.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.396+0000 7f52da445700 1 -- 172.21.15.81:0/723045681 wait complete. 2024-05-08T23:31:32.416 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-08T23:31:32.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41e40f700 1 Processor -- start 2024-05-08T23:31:32.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41e40f700 1 -- start start 2024-05-08T23:31:32.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41e40f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff418140130 0x7ff4180b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41e40f700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff41815e430 0x7ff4180b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41e40f700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4180b4800 con 0x7ff418142880 2024-05-08T23:31:32.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41e40f700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff418140510 con 0x7ff41815e430 2024-05-08T23:31:32.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41e40f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4180b24a0 con 0x7ff418140130 2024-05-08T23:31:32.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41cc0c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff41815e430 0x7ff4180b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41cc0c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff41815e430 0x7ff4180b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42734/0 (socket says 172.21.15.81:42734) 2024-05-08T23:31:32.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41cc0c700 1 -- 172.21.15.81:0/3606598609 learned_addr learned my addr 172.21.15.81:0/3606598609 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:32.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41d40d700 1 --2- 172.21.15.81:0/3606598609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff418140130 0x7ff4180b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41cc0c700 1 -- 172.21.15.81:0/3606598609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff418140130 msgr2=0x7ff4180b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:32.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41cc0c700 1 --2- 172.21.15.81:0/3606598609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff418140130 0x7ff4180b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41cc0c700 1 -- 172.21.15.81:0/3606598609 >> v1:172.21.15.117:6789/0 conn(0x7ff418142880 legacy=0x7ff418160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41cc0c700 1 -- 172.21.15.81:0/3606598609 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4180b3ed0 con 0x7ff41815e430 2024-05-08T23:31:32.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41d40d700 1 --2- 172.21.15.81:0/3606598609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff418140130 0x7ff4180b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:32.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41cc0c700 1 --2- 172.21.15.81:0/3606598609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff41815e430 0x7ff4180b1f60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff408009910 tx=0x7ff408009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5389e54fb638a94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:32.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff40ffff700 1 -- 172.21.15.81:0/3606598609 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408011830 con 0x7ff41815e430 2024-05-08T23:31:32.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff40ffff700 1 -- 172.21.15.81:0/3606598609 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff408011990 con 0x7ff41815e430 2024-05-08T23:31:32.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff40ffff700 1 -- 172.21.15.81:0/3606598609 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408011d40 con 0x7ff41815e430 2024-05-08T23:31:32.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41e40f700 1 -- 172.21.15.81:0/3606598609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff41815e430 msgr2=0x7ff4180b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41e40f700 1 --2- 172.21.15.81:0/3606598609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff41815e430 0x7ff4180b1f60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff408009910 tx=0x7ff408009c20 comp rx=0 tx=0).stop 2024-05-08T23:31:32.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41e40f700 1 -- 172.21.15.81:0/3606598609 shutdown_connections 2024-05-08T23:31:32.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41e40f700 1 --2- 172.21.15.81:0/3606598609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff41815e430 0x7ff4180b1f60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41e40f700 1 --2- 172.21.15.81:0/3606598609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff418140130 0x7ff4180b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.532+0000 7ff41e40f700 1 -- 172.21.15.81:0/3606598609 >> 172.21.15.81:0/3606598609 conn(0x7ff41806c330 msgr2=0x7ff41806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:32.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.536+0000 7ff41e40f700 1 -- 172.21.15.81:0/3606598609 shutdown_connections 2024-05-08T23:31:32.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.536+0000 7ff41e40f700 1 -- 172.21.15.81:0/3606598609 wait complete. 2024-05-08T23:31:32.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.536+0000 7ff41e40f700 1 Processor -- start 2024-05-08T23:31:32.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.536+0000 7ff41e40f700 1 -- start start 2024-05-08T23:31:32.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41e40f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff418140130 0x7ff41813aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41e40f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff418142880 0x7ff4180708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41e40f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff41815e430 0x7ff418070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41e40f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41813afb0 con 0x7ff418142880 2024-05-08T23:31:32.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41e40f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4181576e0 con 0x7ff418140130 2024-05-08T23:31:32.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41e40f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff418071430 con 0x7ff41815e430 2024-05-08T23:31:32.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41cc0c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff418142880 0x7ff4180708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41cc0c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff418142880 0x7ff4180708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42742/0 (socket says 172.21.15.81:42742) 2024-05-08T23:31:32.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41cc0c700 1 -- 172.21.15.81:0/4062864066 learned_addr learned my addr 172.21.15.81:0/4062864066 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:32.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41d40d700 1 --2- 172.21.15.81:0/4062864066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff418140130 0x7ff41813aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41dc0e700 1 --2- 172.21.15.81:0/4062864066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff41815e430 0x7ff418070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41cc0c700 1 -- 172.21.15.81:0/4062864066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff41815e430 msgr2=0x7ff418070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41cc0c700 1 --2- 172.21.15.81:0/4062864066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff41815e430 0x7ff418070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41cc0c700 1 -- 172.21.15.81:0/4062864066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff418140130 msgr2=0x7ff41813aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41cc0c700 1 --2- 172.21.15.81:0/4062864066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff418140130 0x7ff41813aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41cc0c700 1 -- 172.21.15.81:0/4062864066 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff414009a70 con 0x7ff418142880 2024-05-08T23:31:32.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41d40d700 1 --2- 172.21.15.81:0/4062864066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff418140130 0x7ff41813aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:32.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41cc0c700 1 --2- 172.21.15.81:0/4062864066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff418142880 0x7ff4180708e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff408009c60 tx=0x7ff408004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:32.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff40dffb700 1 -- 172.21.15.81:0/4062864066 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408004610 con 0x7ff418142880 2024-05-08T23:31:32.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff40dffb700 1 -- 172.21.15.81:0/4062864066 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40801e070 con 0x7ff418142880 2024-05-08T23:31:32.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff40dffb700 1 -- 172.21.15.81:0/4062864066 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4080038c0 con 0x7ff418142880 2024-05-08T23:31:32.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41e40f700 1 -- 172.21.15.81:0/4062864066 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff408009570 con 0x7ff418142880 2024-05-08T23:31:32.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41e40f700 1 -- 172.21.15.81:0/4062864066 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4180719f0 con 0x7ff418142880 2024-05-08T23:31:32.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff41e40f700 1 -- 172.21.15.81:0/4062864066 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4180684b0 con 0x7ff418142880 2024-05-08T23:31:32.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.540+0000 7ff40dffb700 1 -- 172.21.15.81:0/4062864066 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff408003a20 con 0x7ff418142880 2024-05-08T23:31:32.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.544+0000 7ff40dffb700 1 --2- 172.21.15.81:0/4062864066 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff3fc066730 0x7ff3fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.544+0000 7ff41d40d700 1 --2- 172.21.15.81:0/4062864066 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff3fc066730 0x7ff3fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.544+0000 7ff40dffb700 1 -- 172.21.15.81:0/4062864066 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff408020070 con 0x7ff418142880 2024-05-08T23:31:32.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.544+0000 7ff41d40d700 1 --2- 172.21.15.81:0/4062864066 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff3fc066730 0x7ff3fc068bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff414007640 tx=0x7ff41400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:32.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.548+0000 7ff40dffb700 1 -- 172.21.15.81:0/4062864066 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff408052a60 con 0x7ff418142880 2024-05-08T23:31:32.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.780+0000 7ff41e40f700 1 -- 172.21.15.81:0/4062864066 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7ff41806dfe0 con 0x7ff418142880 2024-05-08T23:31:32.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.784+0000 7ff40dffb700 1 -- 172.21.15.81:0/4062864066 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff408056080 con 0x7ff418142880 2024-05-08T23:31:32.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.788+0000 7ff41e40f700 1 -- 172.21.15.81:0/4062864066 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff3fc066730 msgr2=0x7ff3fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.788+0000 7ff41e40f700 1 --2- 172.21.15.81:0/4062864066 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff3fc066730 0x7ff3fc068bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff414007640 tx=0x7ff41400b040 comp rx=0 tx=0).stop 2024-05-08T23:31:32.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.788+0000 7ff41e40f700 1 -- 172.21.15.81:0/4062864066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff418142880 msgr2=0x7ff4180708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.788+0000 7ff41e40f700 1 --2- 172.21.15.81:0/4062864066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff418142880 0x7ff4180708e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff408009c60 tx=0x7ff408004770 comp rx=0 tx=0).stop 2024-05-08T23:31:32.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.788+0000 7ff41e40f700 1 -- 172.21.15.81:0/4062864066 shutdown_connections 2024-05-08T23:31:32.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.788+0000 7ff41e40f700 1 --2- 172.21.15.81:0/4062864066 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff3fc066730 0x7ff3fc068bf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.788+0000 7ff41e40f700 1 --2- 172.21.15.81:0/4062864066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff41815e430 0x7ff418070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.788+0000 7ff41e40f700 1 --2- 172.21.15.81:0/4062864066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff418142880 0x7ff4180708e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.788+0000 7ff41e40f700 1 --2- 172.21.15.81:0/4062864066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff418140130 0x7ff41813aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.788+0000 7ff41e40f700 1 -- 172.21.15.81:0/4062864066 >> 172.21.15.81:0/4062864066 conn(0x7ff41806c330 msgr2=0x7ff4180b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:32.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.788+0000 7ff41e40f700 1 -- 172.21.15.81:0/4062864066 shutdown_connections 2024-05-08T23:31:32.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.788+0000 7ff41e40f700 1 -- 172.21.15.81:0/4062864066 wait complete. 2024-05-08T23:31:32.805 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-08T23:31:32.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.912+0000 7f4b49cdb700 1 Processor -- start 2024-05-08T23:31:32.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.912+0000 7f4b49cdb700 1 -- start start 2024-05-08T23:31:32.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b49cdb700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b44071e70 0x7f4b441624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b49cdb700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4b44070110 0x7f4b440705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b49cdb700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b44158960 con 0x7f4b440b3c20 2024-05-08T23:31:32.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b49cdb700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b440b1f60 con 0x7f4b44070110 2024-05-08T23:31:32.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b49cdb700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b440ae990 con 0x7f4b44071e70 2024-05-08T23:31:32.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b48cd9700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f4b440b3c20 0x7f4b440b4000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:32840/0 (socket says 172.21.15.81:32840) 2024-05-08T23:31:32.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b48cd9700 1 -- 172.21.15.81:0/1686135063 learned_addr learned my addr 172.21.15.81:0/1686135063 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:32.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b494da700 1 --2- 172.21.15.81:0/1686135063 >> v2:172.21.15.117:3300/0 conn(0x7f4b44070110 0x7f4b440705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b43fff700 1 --2- 172.21.15.81:0/1686135063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b44071e70 0x7f4b441624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b437fe700 1 -- 172.21.15.81:0/1686135063 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656772416 0 0) 0x7f4b44158960 con 0x7f4b440b3c20 2024-05-08T23:31:32.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b437fe700 1 -- 172.21.15.81:0/1686135063 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b300035d0 con 0x7f4b440b3c20 2024-05-08T23:31:32.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b43fff700 1 -- 172.21.15.81:0/1686135063 >> v2:172.21.15.117:3300/0 conn(0x7f4b44070110 msgr2=0x7f4b440705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:32.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b43fff700 1 --2- 172.21.15.81:0/1686135063 >> v2:172.21.15.117:3300/0 conn(0x7f4b44070110 0x7f4b440705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b43fff700 1 -- 172.21.15.81:0/1686135063 >> v1:172.21.15.81:6789/0 conn(0x7f4b440b3c20 legacy=0x7f4b440b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b43fff700 1 -- 172.21.15.81:0/1686135063 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b440b2170 con 0x7f4b44071e70 2024-05-08T23:31:32.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b494da700 1 --2- 172.21.15.81:0/1686135063 >> v2:172.21.15.117:3300/0 conn(0x7f4b44070110 0x7f4b440705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:32.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b43fff700 1 --2- 172.21.15.81:0/1686135063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b44071e70 0x7f4b441624b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009d90 tx=0x7f4b3400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6bb6a93b9ba5a01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:32.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b437fe700 1 -- 172.21.15.81:0/1686135063 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3400e070 con 0x7f4b44071e70 2024-05-08T23:31:32.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b437fe700 1 -- 172.21.15.81:0/1686135063 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3400bb20 con 0x7f4b44071e70 2024-05-08T23:31:32.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b437fe700 1 -- 172.21.15.81:0/1686135063 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3400be90 con 0x7f4b44071e70 2024-05-08T23:31:32.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b49cdb700 1 -- 172.21.15.81:0/1686135063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b44071e70 msgr2=0x7f4b441624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b49cdb700 1 --2- 172.21.15.81:0/1686135063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b44071e70 0x7f4b441624b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009d90 tx=0x7f4b3400b330 comp rx=0 tx=0).stop 2024-05-08T23:31:32.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b49cdb700 1 -- 172.21.15.81:0/1686135063 shutdown_connections 2024-05-08T23:31:32.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b49cdb700 1 --2- 172.21.15.81:0/1686135063 >> v2:172.21.15.117:3300/0 conn(0x7f4b44070110 0x7f4b440705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b49cdb700 1 --2- 172.21.15.81:0/1686135063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b44071e70 0x7f4b441624b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b49cdb700 1 -- 172.21.15.81:0/1686135063 >> 172.21.15.81:0/1686135063 conn(0x7f4b4406c330 msgr2=0x7f4b4406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:32.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b49cdb700 1 -- 172.21.15.81:0/1686135063 shutdown_connections 2024-05-08T23:31:32.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.916+0000 7f4b49cdb700 1 -- 172.21.15.81:0/1686135063 wait complete. 2024-05-08T23:31:32.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b49cdb700 1 Processor -- start 2024-05-08T23:31:32.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b49cdb700 1 -- start start 2024-05-08T23:31:32.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b49cdb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b44070110 0x7f4b441ed020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b49cdb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b44071e70 0x7f4b441ea3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b49cdb700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b440b3c20 0x7f4b44158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b49cdb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b441ea910 con 0x7f4b44071e70 2024-05-08T23:31:32.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b49cdb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b441ebf80 con 0x7f4b44070110 2024-05-08T23:31:32.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b49cdb700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b441ec260 con 0x7f4b440b3c20 2024-05-08T23:31:32.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b43fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b44071e70 0x7f4b441ea3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b43fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b44071e70 0x7f4b441ea3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42748/0 (socket says 172.21.15.81:42748) 2024-05-08T23:31:32.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b43fff700 1 -- 172.21.15.81:0/591936007 learned_addr learned my addr 172.21.15.81:0/591936007 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:32.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b494da700 1 --2- 172.21.15.81:0/591936007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b440b3c20 0x7f4b44158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b48cd9700 1 --2- 172.21.15.81:0/591936007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b44070110 0x7f4b441ed020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b43fff700 1 -- 172.21.15.81:0/591936007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b440b3c20 msgr2=0x7f4b44158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b43fff700 1 --2- 172.21.15.81:0/591936007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b440b3c20 0x7f4b44158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b43fff700 1 -- 172.21.15.81:0/591936007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b44070110 msgr2=0x7f4b441ed020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:32.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b43fff700 1 --2- 172.21.15.81:0/591936007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b44070110 0x7f4b441ed020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:32.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b43fff700 1 -- 172.21.15.81:0/591936007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c0094b0 con 0x7f4b44071e70 2024-05-08T23:31:32.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.920+0000 7f4b494da700 1 --2- 172.21.15.81:0/591936007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b440b3c20 0x7f4b44158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:32.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.924+0000 7f4b43fff700 1 --2- 172.21.15.81:0/591936007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b44071e70 0x7f4b441ea3d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400b370 tx=0x7f4b34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:32.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.924+0000 7f4b417fa700 1 -- 172.21.15.81:0/591936007 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b34007000 con 0x7f4b44071e70 2024-05-08T23:31:32.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.924+0000 7f4b417fa700 1 -- 172.21.15.81:0/591936007 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b34020070 con 0x7f4b44071e70 2024-05-08T23:31:32.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.924+0000 7f4b49cdb700 1 -- 172.21.15.81:0/591936007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b34009a70 con 0x7f4b44071e70 2024-05-08T23:31:32.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.924+0000 7f4b417fa700 1 -- 172.21.15.81:0/591936007 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b340276c0 con 0x7f4b44071e70 2024-05-08T23:31:32.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.924+0000 7f4b49cdb700 1 -- 172.21.15.81:0/591936007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b441ec820 con 0x7f4b44071e70 2024-05-08T23:31:32.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.924+0000 7f4b49cdb700 1 -- 172.21.15.81:0/591936007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b440684b0 con 0x7f4b44071e70 2024-05-08T23:31:32.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.924+0000 7f4b417fa700 1 -- 172.21.15.81:0/591936007 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4b34027820 con 0x7f4b44071e70 2024-05-08T23:31:32.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.924+0000 7f4b417fa700 1 --2- 172.21.15.81:0/591936007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4b24066730 0x7f4b24068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:32.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.924+0000 7f4b48cd9700 1 --2- 172.21.15.81:0/591936007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4b24066730 0x7f4b24068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:32.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.924+0000 7f4b417fa700 1 -- 172.21.15.81:0/591936007 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4b34022070 con 0x7f4b44071e70 2024-05-08T23:31:32.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.928+0000 7f4b48cd9700 1 --2- 172.21.15.81:0/591936007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4b24066730 0x7f4b24068bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c007fd0 tx=0x7f4b2c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:32.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:32.932+0000 7f4b417fa700 1 -- 172.21.15.81:0/591936007 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4b34053850 con 0x7f4b44071e70 2024-05-08T23:31:33.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.164+0000 7f4b49cdb700 1 -- 172.21.15.81:0/591936007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f4b44071c50 con 0x7f4b44071e70 2024-05-08T23:31:33.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.172+0000 7f4b417fa700 1 -- 172.21.15.81:0/591936007 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4b34056e70 con 0x7f4b44071e70 2024-05-08T23:31:33.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.172+0000 7f4b49cdb700 1 -- 172.21.15.81:0/591936007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4b24066730 msgr2=0x7f4b24068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:33.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.172+0000 7f4b49cdb700 1 --2- 172.21.15.81:0/591936007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4b24066730 0x7f4b24068bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c007fd0 tx=0x7f4b2c009040 comp rx=0 tx=0).stop 2024-05-08T23:31:33.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.172+0000 7f4b49cdb700 1 -- 172.21.15.81:0/591936007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b44071e70 msgr2=0x7f4b441ea3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:33.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.172+0000 7f4b49cdb700 1 --2- 172.21.15.81:0/591936007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b44071e70 0x7f4b441ea3d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400b370 tx=0x7f4b34004af0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.172+0000 7f4b49cdb700 1 -- 172.21.15.81:0/591936007 shutdown_connections 2024-05-08T23:31:33.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.176+0000 7f4b49cdb700 1 --2- 172.21.15.81:0/591936007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4b24066730 0x7f4b24068bf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.176+0000 7f4b49cdb700 1 --2- 172.21.15.81:0/591936007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b440b3c20 0x7f4b44158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.176+0000 7f4b49cdb700 1 --2- 172.21.15.81:0/591936007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b44071e70 0x7f4b441ea3d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.176+0000 7f4b49cdb700 1 --2- 172.21.15.81:0/591936007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b44070110 0x7f4b441ed020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.176+0000 7f4b49cdb700 1 -- 172.21.15.81:0/591936007 >> 172.21.15.81:0/591936007 conn(0x7f4b4406c330 msgr2=0x7f4b440cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:33.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.176+0000 7f4b49cdb700 1 -- 172.21.15.81:0/591936007 shutdown_connections 2024-05-08T23:31:33.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.176+0000 7f4b49cdb700 1 -- 172.21.15.81:0/591936007 wait complete. 2024-05-08T23:31:33.192 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-08T23:31:33.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.300+0000 7f3c4712c700 1 Processor -- start 2024-05-08T23:31:33.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.300+0000 7f3c4712c700 1 -- start start 2024-05-08T23:31:33.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4712c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c40071ab0 0x7f3c400ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:33.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4712c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3c4015ac70 0x7f3c4013d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:33.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4712c700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c400cea50 con 0x7f3c40071e90 2024-05-08T23:31:33.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4712c700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c4015b0d0 con 0x7f3c4015ac70 2024-05-08T23:31:33.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4712c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400ce080 con 0x7f3c40071ab0 2024-05-08T23:31:33.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4692b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3c4015ac70 0x7f3c4013d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:33.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4692b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3c4015ac70 0x7f3c4013d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42752/0 (socket says 172.21.15.81:42752) 2024-05-08T23:31:33.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4692b700 1 -- 172.21.15.81:0/1360306290 learned_addr learned my addr 172.21.15.81:0/1360306290 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:33.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c45929700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f3c40071e90 0x7f3c4015a560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:32842/0 (socket says 172.21.15.81:32842) 2024-05-08T23:31:33.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4612a700 1 --2- 172.21.15.81:0/1360306290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c40071ab0 0x7f3c400ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:33.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4692b700 1 -- 172.21.15.81:0/1360306290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c40071ab0 msgr2=0x7f3c400ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:33.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4692b700 1 --2- 172.21.15.81:0/1360306290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c40071ab0 0x7f3c400ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4692b700 1 -- 172.21.15.81:0/1360306290 >> v1:172.21.15.81:6789/0 conn(0x7f3c40071e90 legacy=0x7f3c4015a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:33.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4692b700 1 -- 172.21.15.81:0/1360306290 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c400cd7c0 con 0x7f3c4015ac70 2024-05-08T23:31:33.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4612a700 1 --2- 172.21.15.81:0/1360306290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c40071ab0 0x7f3c400ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:33.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4692b700 1 --2- 172.21.15.81:0/1360306290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c4015ac70 0x7f3c4013d500 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800d150 tx=0x7f3c3800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37f0aa4652f0cf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:33.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c45128700 1 -- 172.21.15.81:0/1360306290 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38010070 con 0x7f3c4015ac70 2024-05-08T23:31:33.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c45128700 1 -- 172.21.15.81:0/1360306290 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3800ddc0 con 0x7f3c4015ac70 2024-05-08T23:31:33.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c45128700 1 -- 172.21.15.81:0/1360306290 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3800b310 con 0x7f3c4015ac70 2024-05-08T23:31:33.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4712c700 1 -- 172.21.15.81:0/1360306290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c4015ac70 msgr2=0x7f3c4013d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:33.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4712c700 1 --2- 172.21.15.81:0/1360306290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c4015ac70 0x7f3c4013d500 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800d150 tx=0x7f3c3800d460 comp rx=0 tx=0).stop 2024-05-08T23:31:33.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4712c700 1 -- 172.21.15.81:0/1360306290 shutdown_connections 2024-05-08T23:31:33.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4712c700 1 --2- 172.21.15.81:0/1360306290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c4015ac70 0x7f3c4013d500 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4712c700 1 --2- 172.21.15.81:0/1360306290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c40071ab0 0x7f3c400ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4712c700 1 -- 172.21.15.81:0/1360306290 >> 172.21.15.81:0/1360306290 conn(0x7f3c4006c330 msgr2=0x7f3c4006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:33.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4712c700 1 -- 172.21.15.81:0/1360306290 shutdown_connections 2024-05-08T23:31:33.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.304+0000 7f3c4712c700 1 -- 172.21.15.81:0/1360306290 wait complete. 2024-05-08T23:31:33.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4712c700 1 Processor -- start 2024-05-08T23:31:33.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4712c700 1 -- start start 2024-05-08T23:31:33.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4712c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c40071ab0 0x7f3c40153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:33.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4712c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c40071e90 0x7f3c400b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:33.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4712c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c4015ac70 0x7f3c401eea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:33.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4712c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c40141c00 con 0x7f3c4015ac70 2024-05-08T23:31:33.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4712c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c40141ee0 con 0x7f3c40071ab0 2024-05-08T23:31:33.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4712c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c401421c0 con 0x7f3c40071e90 2024-05-08T23:31:33.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4692b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c4015ac70 0x7f3c401eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:33.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4692b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c4015ac70 0x7f3c401eea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42760/0 (socket says 172.21.15.81:42760) 2024-05-08T23:31:33.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4692b700 1 -- 172.21.15.81:0/2111678839 learned_addr learned my addr 172.21.15.81:0/2111678839 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:33.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4612a700 1 --2- 172.21.15.81:0/2111678839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c40071ab0 0x7f3c40153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:33.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c45929700 1 --2- 172.21.15.81:0/2111678839 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c40071e90 0x7f3c400b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:33.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4692b700 1 -- 172.21.15.81:0/2111678839 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c40071e90 msgr2=0x7f3c400b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:33.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4692b700 1 --2- 172.21.15.81:0/2111678839 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c40071e90 0x7f3c400b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4692b700 1 -- 172.21.15.81:0/2111678839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c40071ab0 msgr2=0x7f3c40153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:33.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4692b700 1 --2- 172.21.15.81:0/2111678839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c40071ab0 0x7f3c40153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4692b700 1 -- 172.21.15.81:0/2111678839 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3800bd40 con 0x7f3c4015ac70 2024-05-08T23:31:33.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4692b700 1 --2- 172.21.15.81:0/2111678839 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c4015ac70 0x7f3c401eea70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3c4006b120 tx=0x7f3c38004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:33.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c32ffd700 1 -- 172.21.15.81:0/2111678839 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38009e40 con 0x7f3c4015ac70 2024-05-08T23:31:33.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c32ffd700 1 -- 172.21.15.81:0/2111678839 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3800bbc0 con 0x7f3c4015ac70 2024-05-08T23:31:33.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c4712c700 1 -- 172.21.15.81:0/2111678839 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c401424a0 con 0x7f3c4015ac70 2024-05-08T23:31:33.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.308+0000 7f3c32ffd700 1 -- 172.21.15.81:0/2111678839 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38004280 con 0x7f3c4015ac70 2024-05-08T23:31:33.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.312+0000 7f3c4712c700 1 -- 172.21.15.81:0/2111678839 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c401541c0 con 0x7f3c4015ac70 2024-05-08T23:31:33.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.312+0000 7f3c4712c700 1 -- 172.21.15.81:0/2111678839 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c40142960 con 0x7f3c4015ac70 2024-05-08T23:31:33.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.312+0000 7f3c32ffd700 1 -- 172.21.15.81:0/2111678839 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3c380043e0 con 0x7f3c4015ac70 2024-05-08T23:31:33.323 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.312+0000 7f3c32ffd700 1 --2- 172.21.15.81:0/2111678839 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3c24066730 0x7f3c24068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:33.323 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.312+0000 7f3c32ffd700 1 -- 172.21.15.81:0/2111678839 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3c38087770 con 0x7f3c4015ac70 2024-05-08T23:31:33.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.312+0000 7f3c4612a700 1 --2- 172.21.15.81:0/2111678839 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3c24066730 0x7f3c24068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:33.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.312+0000 7f3c4612a700 1 --2- 172.21.15.81:0/2111678839 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3c24066730 0x7f3c24068bf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3c340044b0 tx=0x7f3c34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:33.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.320+0000 7f3c32ffd700 1 -- 172.21.15.81:0/2111678839 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3c38054ec0 con 0x7f3c4015ac70 2024-05-08T23:31:33.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.548+0000 7f3c4712c700 1 -- 172.21.15.81:0/2111678839 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f3c400684b0 con 0x7f3c4015ac70 2024-05-08T23:31:33.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.556+0000 7f3c32ffd700 1 -- 172.21.15.81:0/2111678839 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3c380584e0 con 0x7f3c4015ac70 2024-05-08T23:31:33.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.560+0000 7f3c4712c700 1 -- 172.21.15.81:0/2111678839 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3c24066730 msgr2=0x7f3c24068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:33.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.560+0000 7f3c4712c700 1 --2- 172.21.15.81:0/2111678839 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3c24066730 0x7f3c24068bf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3c340044b0 tx=0x7f3c34008040 comp rx=0 tx=0).stop 2024-05-08T23:31:33.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.560+0000 7f3c4712c700 1 -- 172.21.15.81:0/2111678839 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c4015ac70 msgr2=0x7f3c401eea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:33.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.560+0000 7f3c4712c700 1 --2- 172.21.15.81:0/2111678839 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c4015ac70 0x7f3c401eea70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3c4006b120 tx=0x7f3c38004af0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.560+0000 7f3c4712c700 1 -- 172.21.15.81:0/2111678839 shutdown_connections 2024-05-08T23:31:33.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.560+0000 7f3c4712c700 1 --2- 172.21.15.81:0/2111678839 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3c24066730 0x7f3c24068bf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.560+0000 7f3c4712c700 1 --2- 172.21.15.81:0/2111678839 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c4015ac70 0x7f3c401eea70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.560+0000 7f3c4712c700 1 --2- 172.21.15.81:0/2111678839 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c40071e90 0x7f3c400b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.560+0000 7f3c4712c700 1 --2- 172.21.15.81:0/2111678839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c40071ab0 0x7f3c40153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.560+0000 7f3c4712c700 1 -- 172.21.15.81:0/2111678839 >> 172.21.15.81:0/2111678839 conn(0x7f3c4006c330 msgr2=0x7f3c4015cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:33.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.560+0000 7f3c4712c700 1 -- 172.21.15.81:0/2111678839 shutdown_connections 2024-05-08T23:31:33.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.560+0000 7f3c4712c700 1 -- 172.21.15.81:0/2111678839 wait complete. 2024-05-08T23:31:33.579 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-08T23:31:33.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.684+0000 7f885a845700 1 Processor -- start 2024-05-08T23:31:33.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.684+0000 7f885a845700 1 -- start start 2024-05-08T23:31:33.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.684+0000 7f885a845700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f88540d37c0 0x7f88540d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:33.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.684+0000 7f885a845700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f88540b2ea0 0x7f88540b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:33.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.684+0000 7f885a845700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88540d3c20 con 0x7f88540d1690 2024-05-08T23:31:33.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.684+0000 7f885a845700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88540d3e50 con 0x7f88540b2ea0 2024-05-08T23:31:33.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.684+0000 7f885a845700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88540d1460 con 0x7f88540d37c0 2024-05-08T23:31:33.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f885a044700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f88540b2ea0 0x7f88540b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:33.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f8859843700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f88540d1690 0x7f88540d1a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:32846/0 (socket says 172.21.15.81:32846) 2024-05-08T23:31:33.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f8859843700 1 -- 172.21.15.81:0/4115619248 learned_addr learned my addr 172.21.15.81:0/4115619248 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:33.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f8859042700 1 --2- 172.21.15.81:0/4115619248 >> v2:172.21.15.117:3300/0 conn(0x7f88540d37c0 0x7f88540d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:33.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f885a044700 1 -- 172.21.15.81:0/4115619248 >> v2:172.21.15.117:3300/0 conn(0x7f88540d37c0 msgr2=0x7f88540d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:33.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f885a044700 1 --2- 172.21.15.81:0/4115619248 >> v2:172.21.15.117:3300/0 conn(0x7f88540d37c0 0x7f88540d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f885a044700 1 -- 172.21.15.81:0/4115619248 >> v1:172.21.15.81:6789/0 conn(0x7f88540d1690 legacy=0x7f88540d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:33.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f885a044700 1 -- 172.21.15.81:0/4115619248 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88540ce870 con 0x7f88540b2ea0 2024-05-08T23:31:33.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f8858841700 1 -- 172.21.15.81:0/4115619248 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634057408 0 0) 0x7f88540d3c20 con 0x7f88540d1690 2024-05-08T23:31:33.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f8859042700 1 --2- 172.21.15.81:0/4115619248 >> v2:172.21.15.117:3300/0 conn(0x7f88540d37c0 0x7f88540d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:33.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f885a044700 1 --2- 172.21.15.81:0/4115619248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88540b2ea0 0x7f88540b3330 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f885408d360 tx=0x7f884c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ee27753d0a9fd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:33.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f8858841700 1 -- 172.21.15.81:0/4115619248 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c00e070 con 0x7f88540b2ea0 2024-05-08T23:31:33.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f8858841700 1 -- 172.21.15.81:0/4115619248 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c00bbd0 con 0x7f88540b2ea0 2024-05-08T23:31:33.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f8858841700 1 -- 172.21.15.81:0/4115619248 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c012430 con 0x7f88540b2ea0 2024-05-08T23:31:33.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f885a845700 1 -- 172.21.15.81:0/4115619248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88540b2ea0 msgr2=0x7f88540b3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:33.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f885a845700 1 --2- 172.21.15.81:0/4115619248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88540b2ea0 0x7f88540b3330 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f885408d360 tx=0x7f884c00b330 comp rx=0 tx=0).stop 2024-05-08T23:31:33.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f885a845700 1 -- 172.21.15.81:0/4115619248 shutdown_connections 2024-05-08T23:31:33.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f885a845700 1 --2- 172.21.15.81:0/4115619248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88540b2ea0 0x7f88540b3330 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f885a845700 1 --2- 172.21.15.81:0/4115619248 >> v2:172.21.15.117:3300/0 conn(0x7f88540d37c0 0x7f88540d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f885a845700 1 -- 172.21.15.81:0/4115619248 >> 172.21.15.81:0/4115619248 conn(0x7f885409c3c0 msgr2=0x7f88540d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:33.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f885a845700 1 -- 172.21.15.81:0/4115619248 shutdown_connections 2024-05-08T23:31:33.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.688+0000 7f885a845700 1 -- 172.21.15.81:0/4115619248 wait complete. 2024-05-08T23:31:33.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f885a845700 1 Processor -- start 2024-05-08T23:31:33.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f885a845700 1 -- start start 2024-05-08T23:31:33.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f885a845700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88540b2ea0 0x7f88541525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:33.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f885a845700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88540d1690 0x7f88540b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:33.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f885a845700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88540d37c0 0x7f88540b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:33.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f885a845700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88540cb250 con 0x7f88540b2ea0 2024-05-08T23:31:33.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f885a845700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88540b7020 con 0x7f88540d1690 2024-05-08T23:31:33.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f885a845700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88540b6780 con 0x7f88540d37c0 2024-05-08T23:31:33.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f8859843700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88540b2ea0 0x7f88541525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:33.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f8859843700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88540b2ea0 0x7f88541525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42780/0 (socket says 172.21.15.81:42780) 2024-05-08T23:31:33.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f8859843700 1 -- 172.21.15.81:0/4007720752 learned_addr learned my addr 172.21.15.81:0/4007720752 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:33.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f8859042700 1 --2- 172.21.15.81:0/4007720752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88540d1690 0x7f88540b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:33.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f885a044700 1 --2- 172.21.15.81:0/4007720752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88540d37c0 0x7f88540b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:33.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f8859843700 1 -- 172.21.15.81:0/4007720752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88540d37c0 msgr2=0x7f88540b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:33.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f8859843700 1 --2- 172.21.15.81:0/4007720752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88540d37c0 0x7f88540b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f8859843700 1 -- 172.21.15.81:0/4007720752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88540d1690 msgr2=0x7f88540b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:33.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f8859843700 1 --2- 172.21.15.81:0/4007720752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88540d1690 0x7f88540b7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f8859843700 1 -- 172.21.15.81:0/4007720752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c009a70 con 0x7f88540b2ea0 2024-05-08T23:31:33.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f8859843700 1 --2- 172.21.15.81:0/4007720752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88540b2ea0 0x7f88541525b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f884400c510 tx=0x7f884400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:33.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f884a7fc700 1 -- 172.21.15.81:0/4007720752 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844004ca0 con 0x7f88540b2ea0 2024-05-08T23:31:33.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f884a7fc700 1 -- 172.21.15.81:0/4007720752 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8844004e20 con 0x7f88540b2ea0 2024-05-08T23:31:33.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f885a845700 1 -- 172.21.15.81:0/4007720752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88540b6a60 con 0x7f88540b2ea0 2024-05-08T23:31:33.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f884a7fc700 1 -- 172.21.15.81:0/4007720752 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844007310 con 0x7f88540b2ea0 2024-05-08T23:31:33.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.692+0000 7f885a845700 1 -- 172.21.15.81:0/4007720752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8854160890 con 0x7f88540b2ea0 2024-05-08T23:31:33.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.696+0000 7f885a845700 1 -- 172.21.15.81:0/4007720752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88540684b0 con 0x7f88540b2ea0 2024-05-08T23:31:33.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.696+0000 7f884a7fc700 1 -- 172.21.15.81:0/4007720752 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8844007470 con 0x7f88540b2ea0 2024-05-08T23:31:33.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.696+0000 7f884a7fc700 1 --2- 172.21.15.81:0/4007720752 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8838066730 0x7f8838068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:33.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.696+0000 7f884a7fc700 1 -- 172.21.15.81:0/4007720752 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8844083890 con 0x7f88540b2ea0 2024-05-08T23:31:33.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.696+0000 7f8859042700 1 --2- 172.21.15.81:0/4007720752 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8838066730 0x7f8838068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:33.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.700+0000 7f8859042700 1 --2- 172.21.15.81:0/4007720752 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8838066730 0x7f8838068bf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f88541f6290 tx=0x7f8850008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:33.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.704+0000 7f884a7fc700 1 -- 172.21.15.81:0/4007720752 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8844050fe0 con 0x7f88540b2ea0 2024-05-08T23:31:33.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.932+0000 7f885a845700 1 -- 172.21.15.81:0/4007720752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f88540d3fc0 con 0x7f88540b2ea0 2024-05-08T23:31:33.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.940+0000 7f884a7fc700 1 -- 172.21.15.81:0/4007720752 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8844007720 con 0x7f88540b2ea0 2024-05-08T23:31:33.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.944+0000 7f885a845700 1 -- 172.21.15.81:0/4007720752 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8838066730 msgr2=0x7f8838068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:33.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.944+0000 7f885a845700 1 --2- 172.21.15.81:0/4007720752 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8838066730 0x7f8838068bf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f88541f6290 tx=0x7f8850008040 comp rx=0 tx=0).stop 2024-05-08T23:31:33.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.944+0000 7f885a845700 1 -- 172.21.15.81:0/4007720752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88540b2ea0 msgr2=0x7f88541525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:33.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.944+0000 7f885a845700 1 --2- 172.21.15.81:0/4007720752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88540b2ea0 0x7f88541525b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f884400c510 tx=0x7f884400c820 comp rx=0 tx=0).stop 2024-05-08T23:31:33.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.944+0000 7f885a845700 1 -- 172.21.15.81:0/4007720752 shutdown_connections 2024-05-08T23:31:33.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.944+0000 7f885a845700 1 --2- 172.21.15.81:0/4007720752 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8838066730 0x7f8838068bf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.944+0000 7f885a845700 1 --2- 172.21.15.81:0/4007720752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88540d37c0 0x7f88540b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.944+0000 7f885a845700 1 --2- 172.21.15.81:0/4007720752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88540d1690 0x7f88540b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.944+0000 7f885a845700 1 --2- 172.21.15.81:0/4007720752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88540b2ea0 0x7f88541525b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:33.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.944+0000 7f885a845700 1 -- 172.21.15.81:0/4007720752 >> 172.21.15.81:0/4007720752 conn(0x7f885409c3c0 msgr2=0x7f885406c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:33.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.944+0000 7f885a845700 1 -- 172.21.15.81:0/4007720752 shutdown_connections 2024-05-08T23:31:33.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:33.944+0000 7f885a845700 1 -- 172.21.15.81:0/4007720752 wait complete. 2024-05-08T23:31:33.962 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-08T23:31:34.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd759e700 1 Processor -- start 2024-05-08T23:31:34.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd759e700 1 -- start start 2024-05-08T23:31:34.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dd80d1690 0x7f4dd80d37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd759e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4dd80d0f20 0x7f4dd80d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd759e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd80d1fb0 con 0x7f4dd80cf6e0 2024-05-08T23:31:34.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd759e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd80d3e50 con 0x7f4dd80d0f20 2024-05-08T23:31:34.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80ce870 con 0x7f4dd80d1690 2024-05-08T23:31:34.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd659c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dd80d1690 0x7f4dd80d37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd659c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dd80d1690 0x7f4dd80d37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:38820/0 (socket says 172.21.15.81:38820) 2024-05-08T23:31:34.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd659c700 1 -- 172.21.15.81:0/109434764 learned_addr learned my addr 172.21.15.81:0/109434764 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:34.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd5d9b700 1 --2- 172.21.15.81:0/109434764 >> v2:172.21.15.117:3300/0 conn(0x7f4dd80d0f20 0x7f4dd80d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd659c700 1 -- 172.21.15.81:0/109434764 >> v2:172.21.15.117:3300/0 conn(0x7f4dd80d0f20 msgr2=0x7f4dd80d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:34.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd659c700 1 --2- 172.21.15.81:0/109434764 >> v2:172.21.15.117:3300/0 conn(0x7f4dd80d0f20 0x7f4dd80d1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd659c700 1 -- 172.21.15.81:0/109434764 >> v1:172.21.15.117:6789/0 conn(0x7f4dd80cf6e0 legacy=0x7f4dd80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd659c700 1 -- 172.21.15.81:0/109434764 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd80ceb50 con 0x7f4dd80d1690 2024-05-08T23:31:34.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd659c700 1 --2- 172.21.15.81:0/109434764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dd80d1690 0x7f4dd80d37c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009910 tx=0x7f4dd0009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=309858e0018505e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:34.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd559a700 1 -- 172.21.15.81:0/109434764 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0011830 con 0x7f4dd80d1690 2024-05-08T23:31:34.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd559a700 1 -- 172.21.15.81:0/109434764 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd0011990 con 0x7f4dd80d1690 2024-05-08T23:31:34.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.072+0000 7f4dd559a700 1 -- 172.21.15.81:0/109434764 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0011d40 con 0x7f4dd80d1690 2024-05-08T23:31:34.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.076+0000 7f4dd759e700 1 -- 172.21.15.81:0/109434764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dd80d1690 msgr2=0x7f4dd80d37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.076+0000 7f4dd759e700 1 --2- 172.21.15.81:0/109434764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dd80d1690 0x7f4dd80d37c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009910 tx=0x7f4dd0009c20 comp rx=0 tx=0).stop 2024-05-08T23:31:34.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.076+0000 7f4dd759e700 1 -- 172.21.15.81:0/109434764 shutdown_connections 2024-05-08T23:31:34.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.076+0000 7f4dd759e700 1 --2- 172.21.15.81:0/109434764 >> v2:172.21.15.117:3300/0 conn(0x7f4dd80d0f20 0x7f4dd80d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.076+0000 7f4dd759e700 1 --2- 172.21.15.81:0/109434764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dd80d1690 0x7f4dd80d37c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.076+0000 7f4dd759e700 1 -- 172.21.15.81:0/109434764 >> 172.21.15.81:0/109434764 conn(0x7f4dd809c3c0 msgr2=0x7f4dd80d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:34.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.076+0000 7f4dd759e700 1 -- 172.21.15.81:0/109434764 shutdown_connections 2024-05-08T23:31:34.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.076+0000 7f4dd759e700 1 -- 172.21.15.81:0/109434764 wait complete. 2024-05-08T23:31:34.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.076+0000 7f4dd759e700 1 Processor -- start 2024-05-08T23:31:34.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd759e700 1 -- start start 2024-05-08T23:31:34.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dd80cf6e0 0x7f4dd80b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4dd80d0f20 0x7f4dd8157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dd80d1690 0x7f4dd815ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd759e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd8158020 con 0x7f4dd80d0f20 2024-05-08T23:31:34.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd759e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd8160180 con 0x7f4dd80d1690 2024-05-08T23:31:34.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd815f570 con 0x7f4dd80cf6e0 2024-05-08T23:31:34.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd5d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4dd80d0f20 0x7f4dd8157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd5d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4dd80d0f20 0x7f4dd8157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42792/0 (socket says 172.21.15.81:42792) 2024-05-08T23:31:34.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd5d9b700 1 -- 172.21.15.81:0/1144786848 learned_addr learned my addr 172.21.15.81:0/1144786848 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:34.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd659c700 1 --2- 172.21.15.81:0/1144786848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dd80cf6e0 0x7f4dd80b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd6d9d700 1 --2- 172.21.15.81:0/1144786848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dd80d1690 0x7f4dd815ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd5d9b700 1 -- 172.21.15.81:0/1144786848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dd80cf6e0 msgr2=0x7f4dd80b8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd5d9b700 1 --2- 172.21.15.81:0/1144786848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dd80cf6e0 0x7f4dd80b8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd5d9b700 1 -- 172.21.15.81:0/1144786848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dd80d1690 msgr2=0x7f4dd815ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd5d9b700 1 --2- 172.21.15.81:0/1144786848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dd80d1690 0x7f4dd815ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd5d9b700 1 -- 172.21.15.81:0/1144786848 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd0009570 con 0x7f4dd80d0f20 2024-05-08T23:31:34.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd5d9b700 1 --2- 172.21.15.81:0/1144786848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4dd80d0f20 0x7f4dd8157ae0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00e840 tx=0x7f4dcc00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:34.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dc6ffd700 1 -- 172.21.15.81:0/1144786848 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0095a0 con 0x7f4dd80d0f20 2024-05-08T23:31:34.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dc6ffd700 1 -- 172.21.15.81:0/1144786848 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc013680 con 0x7f4dd80d0f20 2024-05-08T23:31:34.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd759e700 1 -- 172.21.15.81:0/1144786848 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd815f850 con 0x7f4dd80d0f20 2024-05-08T23:31:34.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dc6ffd700 1 -- 172.21.15.81:0/1144786848 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0139f0 con 0x7f4dd80d0f20 2024-05-08T23:31:34.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.080+0000 7f4dd759e700 1 -- 172.21.15.81:0/1144786848 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd815fe10 con 0x7f4dd80d0f20 2024-05-08T23:31:34.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.084+0000 7f4dd759e700 1 -- 172.21.15.81:0/1144786848 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd8160330 con 0x7f4dd80d0f20 2024-05-08T23:31:34.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.084+0000 7f4dc6ffd700 1 -- 172.21.15.81:0/1144786848 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4dcc013b50 con 0x7f4dd80d0f20 2024-05-08T23:31:34.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.084+0000 7f4dc6ffd700 1 --2- 172.21.15.81:0/1144786848 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4db8066730 0x7f4db8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.084+0000 7f4dd659c700 1 --2- 172.21.15.81:0/1144786848 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4db8066730 0x7f4db8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.084+0000 7f4dc6ffd700 1 -- 172.21.15.81:0/1144786848 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4dcc083cf0 con 0x7f4dd80d0f20 2024-05-08T23:31:34.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.084+0000 7f4dd659c700 1 --2- 172.21.15.81:0/1144786848 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4db8066730 0x7f4db8068bf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009860 tx=0x7f4dd0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:34.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.088+0000 7f4dc6ffd700 1 -- 172.21.15.81:0/1144786848 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4dd8160330 con 0x7f4dd80d0f20 2024-05-08T23:31:34.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.320+0000 7f4dd759e700 1 -- 172.21.15.81:0/1144786848 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f4dd81602f0 con 0x7f4dd80d0f20 2024-05-08T23:31:34.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.328+0000 7f4dc6ffd700 1 -- 172.21.15.81:0/1144786848 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4dcc088070 con 0x7f4dd80d0f20 2024-05-08T23:31:34.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.328+0000 7f4dd759e700 1 -- 172.21.15.81:0/1144786848 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4db8066730 msgr2=0x7f4db8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.328+0000 7f4dd759e700 1 --2- 172.21.15.81:0/1144786848 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4db8066730 0x7f4db8068bf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009860 tx=0x7f4dd0008030 comp rx=0 tx=0).stop 2024-05-08T23:31:34.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.328+0000 7f4dd759e700 1 -- 172.21.15.81:0/1144786848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4dd80d0f20 msgr2=0x7f4dd8157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.328+0000 7f4dd759e700 1 --2- 172.21.15.81:0/1144786848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4dd80d0f20 0x7f4dd8157ae0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00e840 tx=0x7f4dcc00eb50 comp rx=0 tx=0).stop 2024-05-08T23:31:34.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.328+0000 7f4dd759e700 1 -- 172.21.15.81:0/1144786848 shutdown_connections 2024-05-08T23:31:34.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.328+0000 7f4dd759e700 1 --2- 172.21.15.81:0/1144786848 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4db8066730 0x7f4db8068bf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.328+0000 7f4dd759e700 1 --2- 172.21.15.81:0/1144786848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dd80d1690 0x7f4dd815ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.328+0000 7f4dd759e700 1 --2- 172.21.15.81:0/1144786848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4dd80d0f20 0x7f4dd8157ae0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.328+0000 7f4dd759e700 1 --2- 172.21.15.81:0/1144786848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dd80cf6e0 0x7f4dd80b8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.328+0000 7f4dd759e700 1 -- 172.21.15.81:0/1144786848 >> 172.21.15.81:0/1144786848 conn(0x7f4dd809c3c0 msgr2=0x7f4dd80afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:34.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.328+0000 7f4dd759e700 1 -- 172.21.15.81:0/1144786848 shutdown_connections 2024-05-08T23:31:34.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.328+0000 7f4dd759e700 1 -- 172.21.15.81:0/1144786848 wait complete. 2024-05-08T23:31:34.348 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:34.458 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c8895700 1 Processor -- start 2024-05-08T23:31:34.458 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c8895700 1 -- start start 2024-05-08T23:31:34.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c8895700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c415afa0 0x7fa6c415b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c8895700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa6c40ce990 0x7fa6c40b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c8895700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c415eda0 con 0x7fa6c415b8c0 2024-05-08T23:31:34.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c8895700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c415ea70 con 0x7fa6c40ce990 2024-05-08T23:31:34.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c8895700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c415bd40 con 0x7fa6c415afa0 2024-05-08T23:31:34.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c359e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa6c40ce990 0x7fa6c40b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c259c700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa6c415b8c0 0x7fa6c415f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:54982/0 (socket says 172.21.15.81:54982) 2024-05-08T23:31:34.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c259c700 1 -- 172.21.15.81:0/695141084 learned_addr learned my addr 172.21.15.81:0/695141084 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:34.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c2d9d700 1 --2- 172.21.15.81:0/695141084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c415afa0 0x7fa6c415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c1d9b700 1 -- 172.21.15.81:0/695141084 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383288460 0 0) 0x7fa6c415eda0 con 0x7fa6c415b8c0 2024-05-08T23:31:34.461 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c1d9b700 1 -- 172.21.15.81:0/695141084 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6b00035d0 con 0x7fa6c415b8c0 2024-05-08T23:31:34.461 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c359e700 1 -- 172.21.15.81:0/695141084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c415afa0 msgr2=0x7fa6c415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:34.461 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c359e700 1 --2- 172.21.15.81:0/695141084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c415afa0 0x7fa6c415b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.461 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c359e700 1 -- 172.21.15.81:0/695141084 >> v1:172.21.15.117:6789/0 conn(0x7fa6c415b8c0 legacy=0x7fa6c415f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.461 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.452+0000 7fa6c359e700 1 -- 172.21.15.81:0/695141084 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c415c020 con 0x7fa6c40ce990 2024-05-08T23:31:34.461 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.456+0000 7fa6c359e700 1 --2- 172.21.15.81:0/695141084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6c40ce990 0x7fa6c40b4270 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009d90 tx=0x7fa6bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355ad9e969c63406 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:34.461 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.456+0000 7fa6c1d9b700 1 -- 172.21.15.81:0/695141084 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc00e070 con 0x7fa6c40ce990 2024-05-08T23:31:34.461 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.456+0000 7fa6c1d9b700 1 -- 172.21.15.81:0/695141084 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc00bb20 con 0x7fa6c40ce990 2024-05-08T23:31:34.461 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.456+0000 7fa6c1d9b700 1 -- 172.21.15.81:0/695141084 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc00be90 con 0x7fa6c40ce990 2024-05-08T23:31:34.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.456+0000 7fa6c8895700 1 -- 172.21.15.81:0/695141084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6c40ce990 msgr2=0x7fa6c40b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.456+0000 7fa6c8895700 1 --2- 172.21.15.81:0/695141084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6c40ce990 0x7fa6c40b4270 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009d90 tx=0x7fa6bc00b330 comp rx=0 tx=0).stop 2024-05-08T23:31:34.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.456+0000 7fa6c8895700 1 -- 172.21.15.81:0/695141084 shutdown_connections 2024-05-08T23:31:34.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.456+0000 7fa6c8895700 1 --2- 172.21.15.81:0/695141084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6c40ce990 0x7fa6c40b4270 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.456+0000 7fa6c8895700 1 --2- 172.21.15.81:0/695141084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c415afa0 0x7fa6c415b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.456+0000 7fa6c8895700 1 -- 172.21.15.81:0/695141084 >> 172.21.15.81:0/695141084 conn(0x7fa6c406c330 msgr2=0x7fa6c406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:34.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.456+0000 7fa6c8895700 1 -- 172.21.15.81:0/695141084 shutdown_connections 2024-05-08T23:31:34.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.456+0000 7fa6c8895700 1 -- 172.21.15.81:0/695141084 wait complete. 2024-05-08T23:31:34.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c8895700 1 Processor -- start 2024-05-08T23:31:34.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c8895700 1 -- start start 2024-05-08T23:31:34.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c8895700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6c40ce990 0x7fa6c40b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c8895700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c415afa0 0x7fa6c4149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c8895700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c415b8c0 0x7fa6c4159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c8895700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40c4a30 con 0x7fa6c40ce990 2024-05-08T23:31:34.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c8895700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c4149d80 con 0x7fa6c415afa0 2024-05-08T23:31:34.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c8895700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40b2bf0 con 0x7fa6c415b8c0 2024-05-08T23:31:34.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c2d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6c40ce990 0x7fa6c40b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c2d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6c40ce990 0x7fa6c40b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42812/0 (socket says 172.21.15.81:42812) 2024-05-08T23:31:34.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c2d9d700 1 -- 172.21.15.81:0/2557162194 learned_addr learned my addr 172.21.15.81:0/2557162194 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:34.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c259c700 1 --2- 172.21.15.81:0/2557162194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c415afa0 0x7fa6c4149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c359e700 1 --2- 172.21.15.81:0/2557162194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c415b8c0 0x7fa6c4159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c2d9d700 1 -- 172.21.15.81:0/2557162194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c415b8c0 msgr2=0x7fa6c4159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c2d9d700 1 --2- 172.21.15.81:0/2557162194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c415b8c0 0x7fa6c4159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c2d9d700 1 -- 172.21.15.81:0/2557162194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c415afa0 msgr2=0x7fa6c4149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c2d9d700 1 --2- 172.21.15.81:0/2557162194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c415afa0 0x7fa6c4149840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c2d9d700 1 -- 172.21.15.81:0/2557162194 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc009a70 con 0x7fa6c40ce990 2024-05-08T23:31:34.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c259c700 1 --2- 172.21.15.81:0/2557162194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c415afa0 0x7fa6c4149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:34.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c2d9d700 1 --2- 172.21.15.81:0/2557162194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6c40ce990 0x7fa6c40b3c70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400e9d0 tx=0x7fa6b400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:34.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6bb7fe700 1 -- 172.21.15.81:0/2557162194 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4009720 con 0x7fa6c40ce990 2024-05-08T23:31:34.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6bb7fe700 1 -- 172.21.15.81:0/2557162194 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b400c850 con 0x7fa6c40ce990 2024-05-08T23:31:34.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6bb7fe700 1 -- 172.21.15.81:0/2557162194 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b400cbc0 con 0x7fa6c40ce990 2024-05-08T23:31:34.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c8895700 1 -- 172.21.15.81:0/2557162194 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c40b2eb0 con 0x7fa6c40ce990 2024-05-08T23:31:34.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c8895700 1 -- 172.21.15.81:0/2557162194 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c40b3470 con 0x7fa6c40ce990 2024-05-08T23:31:34.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.460+0000 7fa6c8895700 1 -- 172.21.15.81:0/2557162194 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c40684b0 con 0x7fa6c40ce990 2024-05-08T23:31:34.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.464+0000 7fa6bb7fe700 1 -- 172.21.15.81:0/2557162194 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa6b400cd20 con 0x7fa6c40ce990 2024-05-08T23:31:34.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.464+0000 7fa6bb7fe700 1 --2- 172.21.15.81:0/2557162194 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa6a4066730 0x7fa6a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.464+0000 7fa6bb7fe700 1 -- 172.21.15.81:0/2557162194 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa6b4084720 con 0x7fa6c40ce990 2024-05-08T23:31:34.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.464+0000 7fa6c259c700 1 --2- 172.21.15.81:0/2557162194 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa6a4066730 0x7fa6a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.464+0000 7fa6c259c700 1 --2- 172.21.15.81:0/2557162194 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa6a4066730 0x7fa6a4068bf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa6c41f63c0 tx=0x7fa6ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:34.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.468+0000 7fa6bb7fe700 1 -- 172.21.15.81:0/2557162194 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa6b4051dc0 con 0x7fa6c40ce990 2024-05-08T23:31:34.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.628+0000 7fa6c8895700 1 -- 172.21.15.81:0/2557162194 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6c406e220 con 0x7fa6c40ce990 2024-05-08T23:31:34.635 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.628+0000 7fa6bb7fe700 1 -- 172.21.15.81:0/2557162194 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+12389 (secure 0 0 0) 0x7fa6b4017090 con 0x7fa6c40ce990 2024-05-08T23:31:34.635 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:34.635 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":32,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:31.610382+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:34.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.632+0000 7fa6c8895700 1 -- 172.21.15.81:0/2557162194 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa6a4066730 msgr2=0x7fa6a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.632+0000 7fa6c8895700 1 --2- 172.21.15.81:0/2557162194 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa6a4066730 0x7fa6a4068bf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa6c41f63c0 tx=0x7fa6ac009040 comp rx=0 tx=0).stop 2024-05-08T23:31:34.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.632+0000 7fa6c8895700 1 -- 172.21.15.81:0/2557162194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6c40ce990 msgr2=0x7fa6c40b3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.632+0000 7fa6c8895700 1 --2- 172.21.15.81:0/2557162194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6c40ce990 0x7fa6c40b3c70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400e9d0 tx=0x7fa6b400ed90 comp rx=0 tx=0).stop 2024-05-08T23:31:34.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.632+0000 7fa6c8895700 1 -- 172.21.15.81:0/2557162194 shutdown_connections 2024-05-08T23:31:34.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.632+0000 7fa6c8895700 1 --2- 172.21.15.81:0/2557162194 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa6a4066730 0x7fa6a4068bf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.632+0000 7fa6c8895700 1 --2- 172.21.15.81:0/2557162194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c415b8c0 0x7fa6c4159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.632+0000 7fa6c8895700 1 --2- 172.21.15.81:0/2557162194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c415afa0 0x7fa6c4149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.632+0000 7fa6c8895700 1 --2- 172.21.15.81:0/2557162194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6c40ce990 0x7fa6c40b3c70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.632+0000 7fa6c8895700 1 -- 172.21.15.81:0/2557162194 >> 172.21.15.81:0/2557162194 conn(0x7fa6c406c330 msgr2=0x7fa6c40af680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:34.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.632+0000 7fa6c8895700 1 -- 172.21.15.81:0/2557162194 shutdown_connections 2024-05-08T23:31:34.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.632+0000 7fa6c8895700 1 -- 172.21.15.81:0/2557162194 wait complete. 2024-05-08T23:31:34.648 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:34.649 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:34.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.760+0000 7ff30e3f6700 1 Processor -- start 2024-05-08T23:31:34.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.760+0000 7ff30e3f6700 1 -- start start 2024-05-08T23:31:34.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff308143850 0x7ff30815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff30815e600 0x7ff3080cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff30815d9d0 0x7ff30815de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3080cfd50 con 0x7ff30815d9d0 2024-05-08T23:31:34.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff30815e3a0 con 0x7ff30815e600 2024-05-08T23:31:34.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff308143c30 con 0x7ff308143850 2024-05-08T23:31:34.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30dbf5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff30815d9d0 0x7ff30815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30dbf5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff30815d9d0 0x7ff30815de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42818/0 (socket says 172.21.15.81:42818) 2024-05-08T23:31:34.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30dbf5700 1 -- 172.21.15.81:0/4065851987 learned_addr learned my addr 172.21.15.81:0/4065851987 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:34.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30cbf3700 1 --2- 172.21.15.81:0/4065851987 >> v2:172.21.15.117:3300/0 conn(0x7ff30815e600 0x7ff3080cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30dbf5700 1 -- 172.21.15.81:0/4065851987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff308143850 msgr2=0x7ff30815a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:34.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30d3f4700 1 --2- 172.21.15.81:0/4065851987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff308143850 0x7ff30815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30dbf5700 1 --2- 172.21.15.81:0/4065851987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff308143850 0x7ff30815a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30dbf5700 1 -- 172.21.15.81:0/4065851987 >> v2:172.21.15.117:3300/0 conn(0x7ff30815e600 msgr2=0x7ff3080cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:34.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30dbf5700 1 --2- 172.21.15.81:0/4065851987 >> v2:172.21.15.117:3300/0 conn(0x7ff30815e600 0x7ff3080cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30dbf5700 1 -- 172.21.15.81:0/4065851987 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3080cfa70 con 0x7ff30815d9d0 2024-05-08T23:31:34.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30cbf3700 1 --2- 172.21.15.81:0/4065851987 >> v2:172.21.15.117:3300/0 conn(0x7ff30815e600 0x7ff3080cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:34.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30dbf5700 1 --2- 172.21.15.81:0/4065851987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff30815d9d0 0x7ff30815de60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff308070d50 tx=0x7ff30000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168b0653286ce76a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:34.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff2fffff700 1 -- 172.21.15.81:0/4065851987 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300010070 con 0x7ff30815d9d0 2024-05-08T23:31:34.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff2fffff700 1 -- 172.21.15.81:0/4065851987 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30000dd90 con 0x7ff30815d9d0 2024-05-08T23:31:34.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff2fffff700 1 -- 172.21.15.81:0/4065851987 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30000b310 con 0x7ff30815d9d0 2024-05-08T23:31:34.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 -- 172.21.15.81:0/4065851987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff30815d9d0 msgr2=0x7ff30815de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 --2- 172.21.15.81:0/4065851987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff30815d9d0 0x7ff30815de60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff308070d50 tx=0x7ff30000d430 comp rx=0 tx=0).stop 2024-05-08T23:31:34.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 -- 172.21.15.81:0/4065851987 shutdown_connections 2024-05-08T23:31:34.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 --2- 172.21.15.81:0/4065851987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff30815d9d0 0x7ff30815de60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 --2- 172.21.15.81:0/4065851987 >> v2:172.21.15.117:3300/0 conn(0x7ff30815e600 0x7ff3080cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 --2- 172.21.15.81:0/4065851987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff308143850 0x7ff30815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 -- 172.21.15.81:0/4065851987 >> 172.21.15.81:0/4065851987 conn(0x7ff3080b1c70 msgr2=0x7ff3080b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:34.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 -- 172.21.15.81:0/4065851987 shutdown_connections 2024-05-08T23:31:34.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 -- 172.21.15.81:0/4065851987 wait complete. 2024-05-08T23:31:34.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 Processor -- start 2024-05-08T23:31:34.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 -- start start 2024-05-08T23:31:34.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff308143850 0x7ff30813df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff30815d9d0 0x7ff3080ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff30815e600 0x7ff30813d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30815cc60 con 0x7ff308143850 2024-05-08T23:31:34.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30815c050 con 0x7ff30815e600 2024-05-08T23:31:34.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30815c330 con 0x7ff30815d9d0 2024-05-08T23:31:34.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30d3f4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff308143850 0x7ff30813df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30d3f4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff308143850 0x7ff30813df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42824/0 (socket says 172.21.15.81:42824) 2024-05-08T23:31:34.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30d3f4700 1 -- 172.21.15.81:0/1314323910 learned_addr learned my addr 172.21.15.81:0/1314323910 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:34.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30cbf3700 1 --2- 172.21.15.81:0/1314323910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff30815d9d0 0x7ff3080ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30dbf5700 1 --2- 172.21.15.81:0/1314323910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff30815e600 0x7ff30813d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30d3f4700 1 -- 172.21.15.81:0/1314323910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff30815d9d0 msgr2=0x7ff3080ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30d3f4700 1 --2- 172.21.15.81:0/1314323910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff30815d9d0 0x7ff3080ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30d3f4700 1 -- 172.21.15.81:0/1314323910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff30815e600 msgr2=0x7ff30813d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30d3f4700 1 --2- 172.21.15.81:0/1314323910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff30815e600 0x7ff30813d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30d3f4700 1 -- 172.21.15.81:0/1314323910 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30000bd40 con 0x7ff308143850 2024-05-08T23:31:34.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30d3f4700 1 --2- 172.21.15.81:0/1314323910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff308143850 0x7ff30813df80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff30400c740 tx=0x7ff30400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:34.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff2fdffb700 1 -- 172.21.15.81:0/1314323910 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304004220 con 0x7ff308143850 2024-05-08T23:31:34.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff2fdffb700 1 -- 172.21.15.81:0/1314323910 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff304004380 con 0x7ff308143850 2024-05-08T23:31:34.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 -- 172.21.15.81:0/1314323910 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30815c610 con 0x7ff308143850 2024-05-08T23:31:34.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff2fdffb700 1 -- 172.21.15.81:0/1314323910 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3040036c0 con 0x7ff308143850 2024-05-08T23:31:34.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 -- 172.21.15.81:0/1314323910 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3080ade50 con 0x7ff308143850 2024-05-08T23:31:34.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.768+0000 7ff30e3f6700 1 -- 172.21.15.81:0/1314323910 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3080ae0d0 con 0x7ff308143850 2024-05-08T23:31:34.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.772+0000 7ff2fdffb700 1 -- 172.21.15.81:0/1314323910 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff304003820 con 0x7ff308143850 2024-05-08T23:31:34.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.772+0000 7ff2fdffb700 1 --2- 172.21.15.81:0/1314323910 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff2ec066730 0x7ff2ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:34.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.772+0000 7ff2fdffb700 1 -- 172.21.15.81:0/1314323910 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff304084190 con 0x7ff308143850 2024-05-08T23:31:34.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.772+0000 7ff30cbf3700 1 --2- 172.21.15.81:0/1314323910 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff2ec066730 0x7ff2ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:34.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.772+0000 7ff30cbf3700 1 --2- 172.21.15.81:0/1314323910 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff2ec066730 0x7ff2ec068bf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8006fd0 tx=0x7ff2f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:34.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.784+0000 7ff2fdffb700 1 -- 172.21.15.81:0/1314323910 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff304051860 con 0x7ff308143850 2024-05-08T23:31:34.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.936+0000 7ff30e3f6700 1 -- 172.21.15.81:0/1314323910 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3080672a0 con 0x7ff308143850 2024-05-08T23:31:34.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.936+0000 7ff2fdffb700 1 -- 172.21.15.81:0/1314323910 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff304054e80 con 0x7ff308143850 2024-05-08T23:31:34.942 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:34.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.940+0000 7ff30e3f6700 1 -- 172.21.15.81:0/1314323910 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff2ec066730 msgr2=0x7ff2ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.940+0000 7ff30e3f6700 1 --2- 172.21.15.81:0/1314323910 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff2ec066730 0x7ff2ec068bf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8006fd0 tx=0x7ff2f8008040 comp rx=0 tx=0).stop 2024-05-08T23:31:34.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.940+0000 7ff30e3f6700 1 -- 172.21.15.81:0/1314323910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff308143850 msgr2=0x7ff30813df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:34.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.940+0000 7ff30e3f6700 1 --2- 172.21.15.81:0/1314323910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff308143850 0x7ff30813df80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff30400c740 tx=0x7ff30400cb00 comp rx=0 tx=0).stop 2024-05-08T23:31:34.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.940+0000 7ff30e3f6700 1 -- 172.21.15.81:0/1314323910 shutdown_connections 2024-05-08T23:31:34.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.940+0000 7ff30e3f6700 1 --2- 172.21.15.81:0/1314323910 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff2ec066730 0x7ff2ec068bf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.940+0000 7ff30e3f6700 1 --2- 172.21.15.81:0/1314323910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff30815e600 0x7ff30813d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.940+0000 7ff30e3f6700 1 --2- 172.21.15.81:0/1314323910 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff30815d9d0 0x7ff3080ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.940+0000 7ff30e3f6700 1 --2- 172.21.15.81:0/1314323910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff308143850 0x7ff30813df80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:34.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.940+0000 7ff30e3f6700 1 -- 172.21.15.81:0/1314323910 >> 172.21.15.81:0/1314323910 conn(0x7ff3080b1c70 msgr2=0x7ff3080b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:34.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.940+0000 7ff30e3f6700 1 -- 172.21.15.81:0/1314323910 shutdown_connections 2024-05-08T23:31:34.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:34.940+0000 7ff30e3f6700 1 -- 172.21.15.81:0/1314323910 wait complete. 2024-05-08T23:31:34.956 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T23:31:35.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5656bdd700 1 Processor -- start 2024-05-08T23:31:35.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5656bdd700 1 -- start start 2024-05-08T23:31:35.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5656bdd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f56500ce990 0x7f565015f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:35.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5656bdd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f565015f770 0x7f56500b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:35.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5656bdd700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f565015eda0 con 0x7f565015afa0 2024-05-08T23:31:35.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5656bdd700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565015ea70 con 0x7f56500ce990 2024-05-08T23:31:35.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5656bdd700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565015ba90 con 0x7f565015f770 2024-05-08T23:31:35.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5655bdb700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f565015afa0 0x7f565015b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:32852/0 (socket says 172.21.15.81:32852) 2024-05-08T23:31:35.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5655bdb700 1 -- 172.21.15.81:0/1929024116 learned_addr learned my addr 172.21.15.81:0/1929024116 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:35.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f56553da700 1 --2- 172.21.15.81:0/1929024116 >> v2:172.21.15.117:3300/0 conn(0x7f56500ce990 0x7f565015f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:35.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f56563dc700 1 --2- 172.21.15.81:0/1929024116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f565015f770 0x7f56500b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:35.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 -- 172.21.15.81:0/1929024116 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472555666 0 0) 0x7f565015eda0 con 0x7f565015afa0 2024-05-08T23:31:35.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 -- 172.21.15.81:0/1929024116 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f563c0035d0 con 0x7f565015afa0 2024-05-08T23:31:35.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 -- 172.21.15.81:0/1929024116 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2731555869 0 0) 0x7f563c0035d0 con 0x7f565015afa0 2024-05-08T23:31:35.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 -- 172.21.15.81:0/1929024116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f565015f770 msgr2=0x7f56500b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:35.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 --2- 172.21.15.81:0/1929024116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f565015f770 0x7f56500b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 -- 172.21.15.81:0/1929024116 >> v2:172.21.15.117:3300/0 conn(0x7f56500ce990 msgr2=0x7f565015f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:35.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 --2- 172.21.15.81:0/1929024116 >> v2:172.21.15.117:3300/0 conn(0x7f56500ce990 0x7f565015f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 -- 172.21.15.81:0/1929024116 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565015bd40 con 0x7f565015afa0 2024-05-08T23:31:35.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 -- 172.21.15.81:0/1929024116 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f564c001670 con 0x7f565015afa0 2024-05-08T23:31:35.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f56553da700 1 --2- 172.21.15.81:0/1929024116 >> v2:172.21.15.117:3300/0 conn(0x7f56500ce990 0x7f565015f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:35.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 -- 172.21.15.81:0/1929024116 >> v1:172.21.15.81:6789/0 conn(0x7f565015afa0 legacy=0x7f565015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:35.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 --2- 172.21.15.81:0/1929024116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f563c003a20 0x7f563c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:35.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 --2- 172.21.15.81:0/1929024116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f565015f770 0x7f563c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:35.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 --2- 172.21.15.81:0/1929024116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f563c004db0 0x7f563c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:35.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 -- 172.21.15.81:0/1929024116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565015ea70 con 0x7f563c003a20 2024-05-08T23:31:35.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 -- 172.21.15.81:0/1929024116 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565015ba90 con 0x7f563c004db0 2024-05-08T23:31:35.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 -- 172.21.15.81:0/1929024116 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c00ba30 con 0x7f565015f770 2024-05-08T23:31:35.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5655bdb700 1 --2- 172.21.15.81:0/1929024116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f563c003a20 0x7f563c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:35.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f56553da700 1 --2- 172.21.15.81:0/1929024116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f565015f770 0x7f563c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:35.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f56563dc700 1 --2- 172.21.15.81:0/1929024116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f563c004db0 0x7f563c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:35.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5655bdb700 1 -- 172.21.15.81:0/1929024116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f565015f770 msgr2=0x7f563c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:35.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5655bdb700 1 --2- 172.21.15.81:0/1929024116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f565015f770 0x7f563c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5655bdb700 1 -- 172.21.15.81:0/1929024116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f563c004db0 msgr2=0x7f563c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:35.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5655bdb700 1 --2- 172.21.15.81:0/1929024116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f563c004db0 0x7f563c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5655bdb700 1 -- 172.21.15.81:0/1929024116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c0035d0 con 0x7f563c003a20 2024-05-08T23:31:35.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f56553da700 1 --2- 172.21.15.81:0/1929024116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f565015f770 0x7f563c004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:35.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5655bdb700 1 --2- 172.21.15.81:0/1929024116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f563c003a20 0x7f563c003e00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f564c001eb0 tx=0x7f563c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b27fce9affcf247 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:35.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 -- 172.21.15.81:0/1929024116 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c004c90 con 0x7f563c003a20 2024-05-08T23:31:35.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 -- 172.21.15.81:0/1929024116 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c005b80 con 0x7f563c003a20 2024-05-08T23:31:35.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5654bd9700 1 -- 172.21.15.81:0/1929024116 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c0080d0 con 0x7f563c003a20 2024-05-08T23:31:35.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5656bdd700 1 -- 172.21.15.81:0/1929024116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f563c003a20 msgr2=0x7f563c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:35.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5656bdd700 1 --2- 172.21.15.81:0/1929024116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f563c003a20 0x7f563c003e00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f564c001eb0 tx=0x7f563c0058a0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f56563dc700 1 -- 172.21.15.81:0/1929024116 reap_dead start 2024-05-08T23:31:35.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5656bdd700 1 -- 172.21.15.81:0/1929024116 shutdown_connections 2024-05-08T23:31:35.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5656bdd700 1 -- 172.21.15.81:0/1929024116 >> 172.21.15.81:0/1929024116 conn(0x7f565006c330 msgr2=0x7f565006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:35.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5656bdd700 1 -- 172.21.15.81:0/1929024116 shutdown_connections 2024-05-08T23:31:35.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.064+0000 7f5656bdd700 1 -- 172.21.15.81:0/1929024116 wait complete. 2024-05-08T23:31:35.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f5656bdd700 1 Processor -- start 2024-05-08T23:31:35.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f5656bdd700 1 -- start start 2024-05-08T23:31:35.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f5656bdd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f563c003a20 0x7f5650156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:35.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f5656bdd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56500b4720 0x7f56500b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:35.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f5656bdd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56501effe0 0x7f56501f0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:35.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f5656bdd700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56500b50e0 con 0x7f56500b4720 2024-05-08T23:31:35.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f5656bdd700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5650157850 con 0x7f563c003a20 2024-05-08T23:31:35.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f5656bdd700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56501f09a0 con 0x7f56501effe0 2024-05-08T23:31:35.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56553da700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56500b4720 0x7f56500b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:35.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56553da700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56500b4720 0x7f56500b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42844/0 (socket says 172.21.15.81:42844) 2024-05-08T23:31:35.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56553da700 1 -- 172.21.15.81:0/817140306 learned_addr learned my addr 172.21.15.81:0/817140306 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:35.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f5655bdb700 1 --2- 172.21.15.81:0/817140306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f563c003a20 0x7f5650156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:35.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56563dc700 1 --2- 172.21.15.81:0/817140306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56501effe0 0x7f56501f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:35.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56553da700 1 -- 172.21.15.81:0/817140306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56501effe0 msgr2=0x7f56501f0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:35.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56553da700 1 --2- 172.21.15.81:0/817140306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56501effe0 0x7f56501f0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56553da700 1 -- 172.21.15.81:0/817140306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f563c003a20 msgr2=0x7f5650156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:35.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56553da700 1 --2- 172.21.15.81:0/817140306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f563c003a20 0x7f5650156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56553da700 1 -- 172.21.15.81:0/817140306 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56501f17a0 con 0x7f56500b4720 2024-05-08T23:31:35.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f5655bdb700 1 --2- 172.21.15.81:0/817140306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f563c003a20 0x7f5650156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:35.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56553da700 1 --2- 172.21.15.81:0/817140306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56500b4720 0x7f56500b4ba0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5640010c60 tx=0x7f564000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:35.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56467fc700 1 -- 172.21.15.81:0/817140306 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564000eac0 con 0x7f56500b4720 2024-05-08T23:31:35.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56467fc700 1 -- 172.21.15.81:0/817140306 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564000ec20 con 0x7f56500b4720 2024-05-08T23:31:35.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56467fc700 1 -- 172.21.15.81:0/817140306 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640016460 con 0x7f56500b4720 2024-05-08T23:31:35.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f5656bdd700 1 -- 172.21.15.81:0/817140306 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56501f0c20 con 0x7f56500b4720 2024-05-08T23:31:35.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f5656bdd700 1 -- 172.21.15.81:0/817140306 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56501f11e0 con 0x7f56500b4720 2024-05-08T23:31:35.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56467fc700 1 -- 172.21.15.81:0/817140306 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f56400165c0 con 0x7f56500b4720 2024-05-08T23:31:35.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f5656bdd700 1 -- 172.21.15.81:0/817140306 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56500684b0 con 0x7f56500b4720 2024-05-08T23:31:35.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56467fc700 1 --2- 172.21.15.81:0/817140306 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5634066730 0x7f5634068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:35.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f56467fc700 1 -- 172.21.15.81:0/817140306 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5640084300 con 0x7f56500b4720 2024-05-08T23:31:35.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f5655bdb700 1 --2- 172.21.15.81:0/817140306 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5634066730 0x7f5634068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:35.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.072+0000 7f5655bdb700 1 --2- 172.21.15.81:0/817140306 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5634066730 0x7f5634068bf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f564c001eb0 tx=0x7f564c00a070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:35.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.084+0000 7f56467fc700 1 -- 172.21.15.81:0/817140306 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5640051930 con 0x7f56500b4720 2024-05-08T23:31:35.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.248+0000 7f5656bdd700 1 -- 172.21.15.81:0/817140306 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f565006e220 con 0x7f56500b4720 2024-05-08T23:31:35.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.252+0000 7f56467fc700 1 -- 172.21.15.81:0/817140306 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5640054f50 con 0x7f56500b4720 2024-05-08T23:31:35.256 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:35.256 INFO:teuthology.orchestra.run.smithi081.stdout:[ 2024-05-08T23:31:35.256 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:31:35.256 INFO:teuthology.orchestra.run.smithi081.stdout: "poolnum": 1, 2024-05-08T23:31:35.256 INFO:teuthology.orchestra.run.smithi081.stdout: "poolname": ".mgr" 2024-05-08T23:31:35.256 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:35.256 INFO:teuthology.orchestra.run.smithi081.stdout:] 2024-05-08T23:31:35.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.252+0000 7f5656bdd700 1 -- 172.21.15.81:0/817140306 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5634066730 msgr2=0x7f5634068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:35.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.252+0000 7f5656bdd700 1 --2- 172.21.15.81:0/817140306 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5634066730 0x7f5634068bf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f564c001eb0 tx=0x7f564c00a070 comp rx=0 tx=0).stop 2024-05-08T23:31:35.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.252+0000 7f5656bdd700 1 -- 172.21.15.81:0/817140306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56500b4720 msgr2=0x7f56500b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:35.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.252+0000 7f5656bdd700 1 --2- 172.21.15.81:0/817140306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56500b4720 0x7f56500b4ba0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5640010c60 tx=0x7f564000e4b0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.252+0000 7f5656bdd700 1 -- 172.21.15.81:0/817140306 shutdown_connections 2024-05-08T23:31:35.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.252+0000 7f5656bdd700 1 --2- 172.21.15.81:0/817140306 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5634066730 0x7f5634068bf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.252+0000 7f5656bdd700 1 --2- 172.21.15.81:0/817140306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56501effe0 0x7f56501f0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.252+0000 7f5656bdd700 1 --2- 172.21.15.81:0/817140306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56500b4720 0x7f56500b4ba0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.252+0000 7f5656bdd700 1 --2- 172.21.15.81:0/817140306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f563c003a20 0x7f5650156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.252+0000 7f5656bdd700 1 -- 172.21.15.81:0/817140306 >> 172.21.15.81:0/817140306 conn(0x7f565006c330 msgr2=0x7f56500aec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:35.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.252+0000 7f5656bdd700 1 -- 172.21.15.81:0/817140306 shutdown_connections 2024-05-08T23:31:35.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.252+0000 7f5656bdd700 1 -- 172.21.15.81:0/817140306 wait complete. 2024-05-08T23:31:35.271 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T23:31:35.271 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T23:31:35.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.384+0000 7f89911b0700 1 Processor -- start 2024-05-08T23:31:35.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.384+0000 7f89911b0700 1 -- start start 2024-05-08T23:31:35.392 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f89911b0700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f898c0d0c30 0x7f898c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:35.392 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f89911b0700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0d02b0 0x7f898c0b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:35.392 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f89911b0700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898c0b3030 con 0x7f898c15d300 2024-05-08T23:31:35.392 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f89911b0700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898c0d1210 con 0x7f898c0d0c30 2024-05-08T23:31:35.393 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f89911b0700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c0b29e0 con 0x7f898c0d02b0 2024-05-08T23:31:35.393 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f898b7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f898c0d0c30 0x7f898c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:35.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f89909af700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0d02b0 0x7f898c0b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:35.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f898bfff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f898c15d300 0x7f898c0d14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:54998/0 (socket says 172.21.15.81:54998) 2024-05-08T23:31:35.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f898b7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f898c0d0c30 0x7f898c0d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:36400/0 (socket says 172.21.15.81:36400) 2024-05-08T23:31:35.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f89909af700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0d02b0 0x7f898c0b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:38882/0 (socket says 172.21.15.81:38882) 2024-05-08T23:31:35.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f898bfff700 1 -- 172.21.15.81:0/2052167951 learned_addr learned my addr 172.21.15.81:0/2052167951 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:35.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f898b7fe700 1 -- 172.21.15.81:0/2052167951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0d02b0 msgr2=0x7f898c0b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:35.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f898b7fe700 1 --2- 172.21.15.81:0/2052167951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0d02b0 0x7f898c0b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f898b7fe700 1 -- 172.21.15.81:0/2052167951 >> v1:172.21.15.117:6789/0 conn(0x7f898c15d300 legacy=0x7f898c0d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:35.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f898b7fe700 1 -- 172.21.15.81:0/2052167951 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c0b2cc0 con 0x7f898c0d0c30 2024-05-08T23:31:35.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f898b7fe700 1 --2- 172.21.15.81:0/2052167951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0d0c30 0x7f898c0d1c00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f897c009850 tx=0x7f897c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3179781b7593807f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:35.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f898affd700 1 -- 172.21.15.81:0/2052167951 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c011830 con 0x7f898c0d0c30 2024-05-08T23:31:35.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f898affd700 1 -- 172.21.15.81:0/2052167951 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c011990 con 0x7f898c0d0c30 2024-05-08T23:31:35.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f898affd700 1 -- 172.21.15.81:0/2052167951 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c011d40 con 0x7f898c0d0c30 2024-05-08T23:31:35.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f89911b0700 1 -- 172.21.15.81:0/2052167951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0d0c30 msgr2=0x7f898c0d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:35.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f89911b0700 1 --2- 172.21.15.81:0/2052167951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0d0c30 0x7f898c0d1c00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f897c009850 tx=0x7f897c009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:35.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f89911b0700 1 -- 172.21.15.81:0/2052167951 shutdown_connections 2024-05-08T23:31:35.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f89911b0700 1 --2- 172.21.15.81:0/2052167951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0d02b0 0x7f898c0b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f89911b0700 1 --2- 172.21.15.81:0/2052167951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0d0c30 0x7f898c0d1c00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f89911b0700 1 -- 172.21.15.81:0/2052167951 >> 172.21.15.81:0/2052167951 conn(0x7f898c09c3b0 msgr2=0x7f898c0d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:35.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f89911b0700 1 -- 172.21.15.81:0/2052167951 shutdown_connections 2024-05-08T23:31:35.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.388+0000 7f89911b0700 1 -- 172.21.15.81:0/2052167951 wait complete. 2024-05-08T23:31:35.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89911b0700 1 Processor -- start 2024-05-08T23:31:35.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89911b0700 1 -- start start 2024-05-08T23:31:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89911b0700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0d02b0 0x7f898c156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89911b0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0d0c30 0x7f898c14d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89911b0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c15d300 0x7f898c0b7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89911b0700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c14d970 con 0x7f898c15d300 2024-05-08T23:31:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89911b0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c15e050 con 0x7f898c0d0c30 2024-05-08T23:31:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89911b0700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c0b7e90 con 0x7f898c0d02b0 2024-05-08T23:31:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89909af700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c15d300 0x7f898c0b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89909af700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c15d300 0x7f898c0b7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42852/0 (socket says 172.21.15.81:42852) 2024-05-08T23:31:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89909af700 1 -- 172.21.15.81:0/2652575358 learned_addr learned my addr 172.21.15.81:0/2652575358 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f898bfff700 1 --2- 172.21.15.81:0/2652575358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0d02b0 0x7f898c156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f898b7fe700 1 --2- 172.21.15.81:0/2652575358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0d0c30 0x7f898c14d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89909af700 1 -- 172.21.15.81:0/2652575358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0d02b0 msgr2=0x7f898c156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89909af700 1 --2- 172.21.15.81:0/2652575358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0d02b0 0x7f898c156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89909af700 1 -- 172.21.15.81:0/2652575358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0d0c30 msgr2=0x7f898c14d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89909af700 1 --2- 172.21.15.81:0/2652575358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0d0c30 0x7f898c14d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89909af700 1 -- 172.21.15.81:0/2652575358 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c0094b0 con 0x7f898c15d300 2024-05-08T23:31:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f898b7fe700 1 --2- 172.21.15.81:0/2652575358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0d0c30 0x7f898c14d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.392+0000 7f89909af700 1 --2- 172.21.15.81:0/2652575358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c15d300 0x7f898c0b7670 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f898400efc0 tx=0x7f898400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.396+0000 7f8988ff9700 1 -- 172.21.15.81:0/2652575358 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898400cd40 con 0x7f898c15d300 2024-05-08T23:31:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.396+0000 7f8988ff9700 1 -- 172.21.15.81:0/2652575358 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898400f040 con 0x7f898c15d300 2024-05-08T23:31:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.396+0000 7f89911b0700 1 -- 172.21.15.81:0/2652575358 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898c0b8170 con 0x7f898c15d300 2024-05-08T23:31:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.396+0000 7f8988ff9700 1 -- 172.21.15.81:0/2652575358 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89840146c0 con 0x7f898c15d300 2024-05-08T23:31:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.396+0000 7f89911b0700 1 -- 172.21.15.81:0/2652575358 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c15e2a0 con 0x7f898c15d300 2024-05-08T23:31:35.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.396+0000 7f89911b0700 1 -- 172.21.15.81:0/2652575358 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898c0684b0 con 0x7f898c15d300 2024-05-08T23:31:35.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.396+0000 7f8988ff9700 1 -- 172.21.15.81:0/2652575358 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8984014820 con 0x7f898c15d300 2024-05-08T23:31:35.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.396+0000 7f8988ff9700 1 --2- 172.21.15.81:0/2652575358 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f896c066730 0x7f896c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:35.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.396+0000 7f8988ff9700 1 -- 172.21.15.81:0/2652575358 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8984084700 con 0x7f898c15d300 2024-05-08T23:31:35.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.400+0000 7f898bfff700 1 --2- 172.21.15.81:0/2652575358 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f896c066730 0x7f896c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:35.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.400+0000 7f898bfff700 1 --2- 172.21.15.81:0/2652575358 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f896c066730 0x7f896c068bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8974005930 tx=0x7f8974009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:35.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.408+0000 7f8988ff9700 1 -- 172.21.15.81:0/2652575358 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8984051e50 con 0x7f898c15d300 2024-05-08T23:31:35.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:35.572+0000 7f89911b0700 1 -- 172.21.15.81:0/2652575358 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f898c0b7bb0 con 0x7f898c15d300 2024-05-08T23:31:36.066 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:31:36.069 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:31:36.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.508+0000 7f8988ff9700 1 -- 172.21.15.81:0/2652575358 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) v1 ==== 138+0+0 (secure 0 0 0) 0x7f898400cea0 con 0x7f898c15d300 2024-05-08T23:31:36.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.512+0000 7f89911b0700 1 -- 172.21.15.81:0/2652575358 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f896c066730 msgr2=0x7f896c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:36.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.512+0000 7f89911b0700 1 --2- 172.21.15.81:0/2652575358 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f896c066730 0x7f896c068bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8974005930 tx=0x7f8974009040 comp rx=0 tx=0).stop 2024-05-08T23:31:36.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.512+0000 7f89911b0700 1 -- 172.21.15.81:0/2652575358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c15d300 msgr2=0x7f898c0b7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:36.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.512+0000 7f89911b0700 1 --2- 172.21.15.81:0/2652575358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c15d300 0x7f898c0b7670 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f898400efc0 tx=0x7f898400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:31:36.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.512+0000 7f89911b0700 1 -- 172.21.15.81:0/2652575358 shutdown_connections 2024-05-08T23:31:36.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.512+0000 7f89911b0700 1 --2- 172.21.15.81:0/2652575358 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f896c066730 0x7f896c068bf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:36.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.512+0000 7f89911b0700 1 --2- 172.21.15.81:0/2652575358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c15d300 0x7f898c0b7670 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:36.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.512+0000 7f89911b0700 1 --2- 172.21.15.81:0/2652575358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0d0c30 0x7f898c14d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:36.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.512+0000 7f89911b0700 1 --2- 172.21.15.81:0/2652575358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0d02b0 0x7f898c156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:36.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.512+0000 7f89911b0700 1 -- 172.21.15.81:0/2652575358 >> 172.21.15.81:0/2652575358 conn(0x7f898c09c3b0 msgr2=0x7f898c15a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:36.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.512+0000 7f89911b0700 1 -- 172.21.15.81:0/2652575358 shutdown_connections 2024-05-08T23:31:36.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.512+0000 7f89911b0700 1 -- 172.21.15.81:0/2652575358 wait complete. 2024-05-08T23:31:36.517 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' created 2024-05-08T23:31:36.528 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T23:31:36.651 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.648+0000 7fd83c92f700 1 Processor -- start 2024-05-08T23:31:36.651 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.648+0000 7fd83c92f700 1 -- start start 2024-05-08T23:31:36.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83c92f700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd83815b080 0x7fd838099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:36.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83c92f700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd8380b53c0 0x7fd8380b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:36.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83c92f700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83809a660 con 0x7fd8380b21b0 2024-05-08T23:31:36.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83c92f700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8380b2640 con 0x7fd83815b080 2024-05-08T23:31:36.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83c92f700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83815b460 con 0x7fd8380b53c0 2024-05-08T23:31:36.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd836d9d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd83815b080 0x7fd838099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:36.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd82ffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd8380b53c0 0x7fd8380b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:36.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd836d9d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd83815b080 0x7fd838099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60404/0 (socket says 172.21.15.81:60404) 2024-05-08T23:31:36.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd836d9d700 1 -- 172.21.15.81:0/965877719 learned_addr learned my addr 172.21.15.81:0/965877719 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:36.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd836d9d700 1 -- 172.21.15.81:0/965877719 >> v2:172.21.15.117:3300/0 conn(0x7fd8380b53c0 msgr2=0x7fd8380b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:36.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd836d9d700 1 --2- 172.21.15.81:0/965877719 >> v2:172.21.15.117:3300/0 conn(0x7fd8380b53c0 0x7fd8380b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:36.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd836d9d700 1 -- 172.21.15.81:0/965877719 >> v1:172.21.15.117:6789/0 conn(0x7fd8380b21b0 legacy=0x7fd83815ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:36.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd836d9d700 1 -- 172.21.15.81:0/965877719 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83815b700 con 0x7fd83815b080 2024-05-08T23:31:36.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd836d9d700 1 --2- 172.21.15.81:0/965877719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd83815b080 0x7fd838099a70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd8380d1e60 tx=0x7fd828009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae40e018116fcede server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:36.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83659c700 1 -- 172.21.15.81:0/965877719 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828011830 con 0x7fd83815b080 2024-05-08T23:31:36.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83659c700 1 -- 172.21.15.81:0/965877719 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd828011990 con 0x7fd83815b080 2024-05-08T23:31:36.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83659c700 1 -- 172.21.15.81:0/965877719 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828011d20 con 0x7fd83815b080 2024-05-08T23:31:36.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83c92f700 1 -- 172.21.15.81:0/965877719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd83815b080 msgr2=0x7fd838099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:36.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83c92f700 1 --2- 172.21.15.81:0/965877719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd83815b080 0x7fd838099a70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd8380d1e60 tx=0x7fd828009bf0 comp rx=0 tx=0).stop 2024-05-08T23:31:36.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83c92f700 1 -- 172.21.15.81:0/965877719 shutdown_connections 2024-05-08T23:31:36.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83c92f700 1 --2- 172.21.15.81:0/965877719 >> v2:172.21.15.117:3300/0 conn(0x7fd8380b53c0 0x7fd8380b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:36.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83c92f700 1 --2- 172.21.15.81:0/965877719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd83815b080 0x7fd838099a70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:36.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83c92f700 1 -- 172.21.15.81:0/965877719 >> 172.21.15.81:0/965877719 conn(0x7fd8380b2ae0 msgr2=0x7fd8380b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:36.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83c92f700 1 -- 172.21.15.81:0/965877719 shutdown_connections 2024-05-08T23:31:36.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.652+0000 7fd83c92f700 1 -- 172.21.15.81:0/965877719 wait complete. 2024-05-08T23:31:36.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd83c92f700 1 Processor -- start 2024-05-08T23:31:36.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd83c92f700 1 -- start start 2024-05-08T23:31:36.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd83c92f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8380b21b0 0x7fd83806f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:36.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd83c92f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8380b53c0 0x7fd83814b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:36.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd83c92f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd83815b080 0x7fd8381595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:36.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd83c92f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83806fd30 con 0x7fd8380b21b0 2024-05-08T23:31:36.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd83c92f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8381f40f0 con 0x7fd8380b53c0 2024-05-08T23:31:36.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd83c92f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8381f43d0 con 0x7fd83815b080 2024-05-08T23:31:36.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd836d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8380b21b0 0x7fd83806f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:36.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd836d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8380b21b0 0x7fd83806f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60408/0 (socket says 172.21.15.81:60408) 2024-05-08T23:31:36.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd836d9d700 1 -- 172.21.15.81:0/2814048504 learned_addr learned my addr 172.21.15.81:0/2814048504 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:36.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd82ffff700 1 --2- 172.21.15.81:0/2814048504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8380b53c0 0x7fd83814b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:36.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd83759e700 1 --2- 172.21.15.81:0/2814048504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd83815b080 0x7fd8381595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:36.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd836d9d700 1 -- 172.21.15.81:0/2814048504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd83815b080 msgr2=0x7fd8381595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:36.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd836d9d700 1 --2- 172.21.15.81:0/2814048504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd83815b080 0x7fd8381595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:36.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd836d9d700 1 -- 172.21.15.81:0/2814048504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8380b53c0 msgr2=0x7fd83814b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:36.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd836d9d700 1 --2- 172.21.15.81:0/2814048504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8380b53c0 0x7fd83814b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:36.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd836d9d700 1 -- 172.21.15.81:0/2814048504 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd828009570 con 0x7fd8380b21b0 2024-05-08T23:31:36.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd836d9d700 1 --2- 172.21.15.81:0/2814048504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8380b21b0 0x7fd83806f4c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd83806b440 tx=0x7fd828004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:36.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd82f7fe700 1 -- 172.21.15.81:0/2814048504 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828004450 con 0x7fd8380b21b0 2024-05-08T23:31:36.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd82f7fe700 1 -- 172.21.15.81:0/2814048504 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82801f070 con 0x7fd8380b21b0 2024-05-08T23:31:36.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd83c92f700 1 -- 172.21.15.81:0/2814048504 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8381f46b0 con 0x7fd8380b21b0 2024-05-08T23:31:36.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd82f7fe700 1 -- 172.21.15.81:0/2814048504 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828003900 con 0x7fd8380b21b0 2024-05-08T23:31:36.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.656+0000 7fd83c92f700 1 -- 172.21.15.81:0/2814048504 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83806fa00 con 0x7fd8380b21b0 2024-05-08T23:31:36.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.660+0000 7fd83c92f700 1 -- 172.21.15.81:0/2814048504 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8380672a0 con 0x7fd8380b21b0 2024-05-08T23:31:36.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.660+0000 7fd82f7fe700 1 -- 172.21.15.81:0/2814048504 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd828003a60 con 0x7fd8380b21b0 2024-05-08T23:31:36.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.660+0000 7fd82f7fe700 1 --2- 172.21.15.81:0/2814048504 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd818066730 0x7fd818068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:36.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.660+0000 7fd82ffff700 1 --2- 172.21.15.81:0/2814048504 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd818066730 0x7fd818068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:36.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.660+0000 7fd82f7fe700 1 -- 172.21.15.81:0/2814048504 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fd828021070 con 0x7fd8380b21b0 2024-05-08T23:31:36.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.660+0000 7fd82ffff700 1 --2- 172.21.15.81:0/2814048504 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd818066730 0x7fd818068bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd838156d40 tx=0x7fd820009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:36.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.668+0000 7fd82f7fe700 1 -- 172.21.15.81:0/2814048504 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd8280528b0 con 0x7fd8380b21b0 2024-05-08T23:31:36.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:36.836+0000 7fd83c92f700 1 -- 172.21.15.81:0/2814048504 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd83815dee0 con 0x7fd8380b21b0 2024-05-08T23:31:37.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.516+0000 7fd82f7fe700 1 -- 172.21.15.81:0/2814048504 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd8280045d0 con 0x7fd8380b21b0 2024-05-08T23:31:37.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.536+0000 7fd83c92f700 1 -- 172.21.15.81:0/2814048504 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd818066730 msgr2=0x7fd818068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:37.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.536+0000 7fd83c92f700 1 --2- 172.21.15.81:0/2814048504 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd818066730 0x7fd818068bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd838156d40 tx=0x7fd820009240 comp rx=0 tx=0).stop 2024-05-08T23:31:37.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.536+0000 7fd83c92f700 1 -- 172.21.15.81:0/2814048504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8380b21b0 msgr2=0x7fd83806f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:37.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.536+0000 7fd83c92f700 1 --2- 172.21.15.81:0/2814048504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8380b21b0 0x7fd83806f4c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd83806b440 tx=0x7fd828004770 comp rx=0 tx=0).stop 2024-05-08T23:31:37.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.536+0000 7fd83c92f700 1 -- 172.21.15.81:0/2814048504 shutdown_connections 2024-05-08T23:31:37.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.536+0000 7fd83c92f700 1 --2- 172.21.15.81:0/2814048504 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd818066730 0x7fd818068bf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:37.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.536+0000 7fd83c92f700 1 --2- 172.21.15.81:0/2814048504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd83815b080 0x7fd8381595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:37.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.536+0000 7fd83c92f700 1 --2- 172.21.15.81:0/2814048504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8380b53c0 0x7fd83814b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:37.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.536+0000 7fd83c92f700 1 --2- 172.21.15.81:0/2814048504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8380b21b0 0x7fd83806f4c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:37.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.536+0000 7fd83c92f700 1 -- 172.21.15.81:0/2814048504 >> 172.21.15.81:0/2814048504 conn(0x7fd8380b2ae0 msgr2=0x7fd838155930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:37.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.536+0000 7fd83c92f700 1 -- 172.21.15.81:0/2814048504 shutdown_connections 2024-05-08T23:31:37.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.536+0000 7fd83c92f700 1 -- 172.21.15.81:0/2814048504 wait complete. 2024-05-08T23:31:37.538 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' created 2024-05-08T23:31:37.549 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T23:31:37.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.656+0000 7f529b645700 1 Processor -- start 2024-05-08T23:31:37.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.656+0000 7f529b645700 1 -- start start 2024-05-08T23:31:37.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f529b645700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f529413cda0 0x7f529406c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:37.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f529b645700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529413d180 0x7f529406c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:37.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f529b645700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52940b3380 con 0x7f529415a950 2024-05-08T23:31:37.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f529b645700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529415b6a0 con 0x7f529413cda0 2024-05-08T23:31:37.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f529b645700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52940b3660 con 0x7f529413d180 2024-05-08T23:31:37.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f52993e1700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f529413cda0 0x7f529406c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:37.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f5299be2700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f529415a950 0x7f529415ade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58190/0 (socket says 172.21.15.81:58190) 2024-05-08T23:31:37.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f5299be2700 1 -- 172.21.15.81:0/1794474170 learned_addr learned my addr 172.21.15.81:0/1794474170 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:37.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f5298be0700 1 --2- 172.21.15.81:0/1794474170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529413d180 0x7f529406c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:37.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f528bfff700 1 -- 172.21.15.81:0/1794474170 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1781875772 0 0) 0x7f52940b3380 con 0x7f529415a950 2024-05-08T23:31:37.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f52993e1700 1 -- 172.21.15.81:0/1794474170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529413d180 msgr2=0x7f529406c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:37.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f52993e1700 1 --2- 172.21.15.81:0/1794474170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529413d180 0x7f529406c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:37.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f52993e1700 1 -- 172.21.15.81:0/1794474170 >> v1:172.21.15.81:6789/0 conn(0x7f529415a950 legacy=0x7f529415ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:37.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f52993e1700 1 -- 172.21.15.81:0/1794474170 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52940b3940 con 0x7f529413cda0 2024-05-08T23:31:37.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f52993e1700 1 --2- 172.21.15.81:0/1794474170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529413cda0 0x7f529406c350 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f529000baf0 tx=0x7f529000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a918154db3df8b8a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:37.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f528bfff700 1 -- 172.21.15.81:0/1794474170 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529000dc70 con 0x7f529413cda0 2024-05-08T23:31:37.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f528bfff700 1 -- 172.21.15.81:0/1794474170 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529000ddd0 con 0x7f529413cda0 2024-05-08T23:31:37.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f528bfff700 1 -- 172.21.15.81:0/1794474170 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290010690 con 0x7f529413cda0 2024-05-08T23:31:37.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f529b645700 1 -- 172.21.15.81:0/1794474170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529413cda0 msgr2=0x7f529406c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:37.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f529b645700 1 --2- 172.21.15.81:0/1794474170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529413cda0 0x7f529406c350 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f529000baf0 tx=0x7f529000be00 comp rx=0 tx=0).stop 2024-05-08T23:31:37.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f529b645700 1 -- 172.21.15.81:0/1794474170 shutdown_connections 2024-05-08T23:31:37.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f529b645700 1 --2- 172.21.15.81:0/1794474170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529413d180 0x7f529406c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:37.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f529b645700 1 --2- 172.21.15.81:0/1794474170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529413cda0 0x7f529406c350 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:37.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f529b645700 1 -- 172.21.15.81:0/1794474170 >> 172.21.15.81:0/1794474170 conn(0x7f529406a360 msgr2=0x7f529406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:37.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f529b645700 1 -- 172.21.15.81:0/1794474170 shutdown_connections 2024-05-08T23:31:37.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.660+0000 7f529b645700 1 -- 172.21.15.81:0/1794474170 wait complete. 2024-05-08T23:31:37.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f529b645700 1 Processor -- start 2024-05-08T23:31:37.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f529b645700 1 -- start start 2024-05-08T23:31:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f529b645700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f529413cda0 0x7f529414a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f529b645700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529413d180 0x7f52940b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f529b645700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529415a950 0x7f529415b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f529b645700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52940b0290 con 0x7f529415a950 2024-05-08T23:31:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f529b645700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52940affb0 con 0x7f529413cda0 2024-05-08T23:31:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f529b645700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52940b0670 con 0x7f529413d180 2024-05-08T23:31:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f5299be2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529415a950 0x7f529415b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f5299be2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529415a950 0x7f529415b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60416/0 (socket says 172.21.15.81:60416) 2024-05-08T23:31:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f5299be2700 1 -- 172.21.15.81:0/1750285757 learned_addr learned my addr 172.21.15.81:0/1750285757 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f52993e1700 1 --2- 172.21.15.81:0/1750285757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f529413cda0 0x7f529414a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f5298be0700 1 --2- 172.21.15.81:0/1750285757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529413d180 0x7f52940b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:37.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f5299be2700 1 -- 172.21.15.81:0/1750285757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529413d180 msgr2=0x7f52940b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:37.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f5299be2700 1 --2- 172.21.15.81:0/1750285757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529413d180 0x7f52940b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:37.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f5299be2700 1 -- 172.21.15.81:0/1750285757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f529413cda0 msgr2=0x7f529414a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:37.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f5299be2700 1 --2- 172.21.15.81:0/1750285757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f529413cda0 0x7f529414a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:37.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.664+0000 7f5299be2700 1 -- 172.21.15.81:0/1750285757 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529000b750 con 0x7f529415a950 2024-05-08T23:31:37.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.668+0000 7f5299be2700 1 --2- 172.21.15.81:0/1750285757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529415a950 0x7f529415b7e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f528c00cba0 tx=0x7f528c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:37.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.668+0000 7f5289ffb700 1 -- 172.21.15.81:0/1750285757 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c00aaa0 con 0x7f529415a950 2024-05-08T23:31:37.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.668+0000 7f5289ffb700 1 -- 172.21.15.81:0/1750285757 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c00ac20 con 0x7f529415a950 2024-05-08T23:31:37.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.668+0000 7f529b645700 1 -- 172.21.15.81:0/1750285757 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52940b0950 con 0x7f529415a950 2024-05-08T23:31:37.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.668+0000 7f5289ffb700 1 -- 172.21.15.81:0/1750285757 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c0070d0 con 0x7f529415a950 2024-05-08T23:31:37.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.668+0000 7f529b645700 1 -- 172.21.15.81:0/1750285757 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52940afc40 con 0x7f529415a950 2024-05-08T23:31:37.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.668+0000 7f529b645700 1 -- 172.21.15.81:0/1750285757 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52940b0e70 con 0x7f529415a950 2024-05-08T23:31:37.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.668+0000 7f5289ffb700 1 -- 172.21.15.81:0/1750285757 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f528c007230 con 0x7f529415a950 2024-05-08T23:31:37.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.672+0000 7f5289ffb700 1 --2- 172.21.15.81:0/1750285757 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5278066730 0x7f5278068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:37.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.672+0000 7f52993e1700 1 --2- 172.21.15.81:0/1750285757 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5278066730 0x7f5278068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:37.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.672+0000 7f5289ffb700 1 -- 172.21.15.81:0/1750285757 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f528c083e80 con 0x7f529415a950 2024-05-08T23:31:37.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.672+0000 7f52993e1700 1 --2- 172.21.15.81:0/1750285757 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5278066730 0x7f5278068bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f529000b2c0 tx=0x7f5290013000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:37.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.676+0000 7f5289ffb700 1 -- 172.21.15.81:0/1750285757 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f52940b0e70 con 0x7f529415a950 2024-05-08T23:31:37.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.884+0000 7f529b645700 1 -- 172.21.15.81:0/1750285757 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f52940b0e70 con 0x7f529415a950 2024-05-08T23:31:37.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.896+0000 7f5289ffb700 1 -- 172.21.15.81:0/1750285757 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f52940b0e70 con 0x7f529415a950 2024-05-08T23:31:37.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.900+0000 7f529b645700 1 -- 172.21.15.81:0/1750285757 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5278066730 msgr2=0x7f5278068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:37.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.900+0000 7f529b645700 1 --2- 172.21.15.81:0/1750285757 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5278066730 0x7f5278068bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f529000b2c0 tx=0x7f5290013000 comp rx=0 tx=0).stop 2024-05-08T23:31:37.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.900+0000 7f529b645700 1 -- 172.21.15.81:0/1750285757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529415a950 msgr2=0x7f529415b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:37.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.900+0000 7f529b645700 1 --2- 172.21.15.81:0/1750285757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529415a950 0x7f529415b7e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f528c00cba0 tx=0x7f528c00ceb0 comp rx=0 tx=0).stop 2024-05-08T23:31:37.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.900+0000 7f529b645700 1 -- 172.21.15.81:0/1750285757 shutdown_connections 2024-05-08T23:31:37.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.900+0000 7f529b645700 1 --2- 172.21.15.81:0/1750285757 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5278066730 0x7f5278068bf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:37.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.900+0000 7f529b645700 1 --2- 172.21.15.81:0/1750285757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f529415a950 0x7f529415b7e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:37.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.900+0000 7f529b645700 1 --2- 172.21.15.81:0/1750285757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f529413d180 0x7f52940b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:37.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.900+0000 7f529b645700 1 --2- 172.21.15.81:0/1750285757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f529413cda0 0x7f529414a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:37.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.900+0000 7f529b645700 1 -- 172.21.15.81:0/1750285757 >> 172.21.15.81:0/1750285757 conn(0x7f529406a360 msgr2=0x7f5294159ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:37.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.900+0000 7f529b645700 1 -- 172.21.15.81:0/1750285757 shutdown_connections 2024-05-08T23:31:37.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:37.900+0000 7f529b645700 1 -- 172.21.15.81:0/1750285757 wait complete. 2024-05-08T23:31:37.902 INFO:teuthology.orchestra.run.smithi081.stderr:new fs with metadata pool 4 and data pool 5 2024-05-08T23:31:37.919 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:38.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.020+0000 7fb25178a700 1 Processor -- start 2024-05-08T23:31:38.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.020+0000 7fb25178a700 1 -- start start 2024-05-08T23:31:38.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb25178a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb24c15c800 0x7fb24c06f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb25178a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb24c15cc80 0x7fb24c06fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb25178a700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb24c06f2a0 con 0x7fb24c155690 2024-05-08T23:31:38.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb25178a700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb24c06e690 con 0x7fb24c15cc80 2024-05-08T23:31:38.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb25178a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c06e970 con 0x7fb24c15c800 2024-05-08T23:31:38.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb250f89700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb24c15cc80 0x7fb24c06fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb250f89700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb24c15cc80 0x7fb24c06fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60426/0 (socket says 172.21.15.81:60426) 2024-05-08T23:31:38.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb250f89700 1 -- 172.21.15.81:0/596165581 learned_addr learned my addr 172.21.15.81:0/596165581 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:38.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb2437fe700 1 --2- 172.21.15.81:0/596165581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb24c15c800 0x7fb24c06f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb250f89700 1 -- 172.21.15.81:0/596165581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb24c15c800 msgr2=0x7fb24c06f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:38.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb250f89700 1 --2- 172.21.15.81:0/596165581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb24c15c800 0x7fb24c06f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb250f89700 1 -- 172.21.15.81:0/596165581 >> v1:172.21.15.117:6789/0 conn(0x7fb24c155690 legacy=0x7fb24c15c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb250f89700 1 -- 172.21.15.81:0/596165581 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c06ec50 con 0x7fb24c15cc80 2024-05-08T23:31:38.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb250f89700 1 --2- 172.21.15.81:0/596165581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb24c15cc80 0x7fb24c06fc70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb244009e50 tx=0x7fb24400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96de98aa8479c715 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:38.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb24b7fe700 1 -- 172.21.15.81:0/596165581 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24400e070 con 0x7fb24c15cc80 2024-05-08T23:31:38.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb24b7fe700 1 -- 172.21.15.81:0/596165581 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24400b9c0 con 0x7fb24c15cc80 2024-05-08T23:31:38.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb24b7fe700 1 -- 172.21.15.81:0/596165581 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24400bd70 con 0x7fb24c15cc80 2024-05-08T23:31:38.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb25178a700 1 -- 172.21.15.81:0/596165581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb24c15cc80 msgr2=0x7fb24c06fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb25178a700 1 --2- 172.21.15.81:0/596165581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb24c15cc80 0x7fb24c06fc70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb244009e50 tx=0x7fb24400b330 comp rx=0 tx=0).stop 2024-05-08T23:31:38.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb25178a700 1 -- 172.21.15.81:0/596165581 shutdown_connections 2024-05-08T23:31:38.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb25178a700 1 --2- 172.21.15.81:0/596165581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb24c15cc80 0x7fb24c06fc70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb25178a700 1 --2- 172.21.15.81:0/596165581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb24c15c800 0x7fb24c06f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb25178a700 1 -- 172.21.15.81:0/596165581 >> 172.21.15.81:0/596165581 conn(0x7fb24c099a70 msgr2=0x7fb24c099e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:38.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb25178a700 1 -- 172.21.15.81:0/596165581 shutdown_connections 2024-05-08T23:31:38.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.024+0000 7fb25178a700 1 -- 172.21.15.81:0/596165581 wait complete. 2024-05-08T23:31:38.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.028+0000 7fb25178a700 1 Processor -- start 2024-05-08T23:31:38.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.028+0000 7fb25178a700 1 -- start start 2024-05-08T23:31:38.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.028+0000 7fb25178a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb24c155690 0x7fb24c0b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.028+0000 7fb25178a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb24c15c800 0x7fb24c14f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.028+0000 7fb25178a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb24c15cc80 0x7fb24c155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.028+0000 7fb25178a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c14fc70 con 0x7fb24c155690 2024-05-08T23:31:38.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.028+0000 7fb25178a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c0b0db0 con 0x7fb24c15c800 2024-05-08T23:31:38.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.028+0000 7fb25178a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c0b1090 con 0x7fb24c15cc80 2024-05-08T23:31:38.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.028+0000 7fb24bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb24c155690 0x7fb24c0b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.028+0000 7fb24bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb24c155690 0x7fb24c0b0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60440/0 (socket says 172.21.15.81:60440) 2024-05-08T23:31:38.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.028+0000 7fb24bfff700 1 -- 172.21.15.81:0/718590267 learned_addr learned my addr 172.21.15.81:0/718590267 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:38.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.028+0000 7fb2437fe700 1 --2- 172.21.15.81:0/718590267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb24c15c800 0x7fb24c14f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.028+0000 7fb250f89700 1 --2- 172.21.15.81:0/718590267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb24c15cc80 0x7fb24c155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.028+0000 7fb24bfff700 1 -- 172.21.15.81:0/718590267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb24c15cc80 msgr2=0x7fb24c155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.028+0000 7fb24bfff700 1 --2- 172.21.15.81:0/718590267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb24c15cc80 0x7fb24c155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.032+0000 7fb24bfff700 1 -- 172.21.15.81:0/718590267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb24c15c800 msgr2=0x7fb24c14f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.032+0000 7fb24bfff700 1 --2- 172.21.15.81:0/718590267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb24c15c800 0x7fb24c14f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.032+0000 7fb24bfff700 1 -- 172.21.15.81:0/718590267 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb244009b30 con 0x7fb24c155690 2024-05-08T23:31:38.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.032+0000 7fb24bfff700 1 --2- 172.21.15.81:0/718590267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb24c155690 0x7fb24c0b0390 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00c500 tx=0x7fb23c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:38.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.032+0000 7fb2497fa700 1 -- 172.21.15.81:0/718590267 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c0052d0 con 0x7fb24c155690 2024-05-08T23:31:38.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.032+0000 7fb2497fa700 1 -- 172.21.15.81:0/718590267 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23c005430 con 0x7fb24c155690 2024-05-08T23:31:38.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.032+0000 7fb25178a700 1 -- 172.21.15.81:0/718590267 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb24c0b1370 con 0x7fb24c155690 2024-05-08T23:31:38.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.032+0000 7fb2497fa700 1 -- 172.21.15.81:0/718590267 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c0055a0 con 0x7fb24c155690 2024-05-08T23:31:38.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.032+0000 7fb25178a700 1 -- 172.21.15.81:0/718590267 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24c0b1930 con 0x7fb24c155690 2024-05-08T23:31:38.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.032+0000 7fb2497fa700 1 -- 172.21.15.81:0/718590267 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb23c0038e0 con 0x7fb24c155690 2024-05-08T23:31:38.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.032+0000 7fb25178a700 1 -- 172.21.15.81:0/718590267 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb24c0684b0 con 0x7fb24c155690 2024-05-08T23:31:38.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.032+0000 7fb2497fa700 1 --2- 172.21.15.81:0/718590267 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb22c066730 0x7fb22c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.032+0000 7fb2497fa700 1 -- 172.21.15.81:0/718590267 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb23c0834e0 con 0x7fb24c155690 2024-05-08T23:31:38.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.032+0000 7fb2437fe700 1 --2- 172.21.15.81:0/718590267 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb22c066730 0x7fb22c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.036+0000 7fb2437fe700 1 --2- 172.21.15.81:0/718590267 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb22c066730 0x7fb22c068bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0ae050 tx=0x7fb234008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:38.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.044+0000 7fb2497fa700 1 -- 172.21.15.81:0/718590267 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb23c050850 con 0x7fb24c155690 2024-05-08T23:31:38.218 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.216+0000 7fb25178a700 1 -- 172.21.15.81:0/718590267 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb24c09a5b0 con 0x7fb24c155690 2024-05-08T23:31:38.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.220+0000 7fb2497fa700 1 -- 172.21.15.81:0/718590267 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16114 (secure 0 0 0) 0x7fb23c053e70 con 0x7fb24c155690 2024-05-08T23:31:38.221 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:38.221 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":35,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:37.888485+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:31:35.620423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:31:36.878524+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:38.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.220+0000 7fb25178a700 1 -- 172.21.15.81:0/718590267 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb22c066730 msgr2=0x7fb22c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.220+0000 7fb25178a700 1 --2- 172.21.15.81:0/718590267 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb22c066730 0x7fb22c068bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0ae050 tx=0x7fb234008040 comp rx=0 tx=0).stop 2024-05-08T23:31:38.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.220+0000 7fb25178a700 1 -- 172.21.15.81:0/718590267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb24c155690 msgr2=0x7fb24c0b0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.220+0000 7fb25178a700 1 --2- 172.21.15.81:0/718590267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb24c155690 0x7fb24c0b0390 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00c500 tx=0x7fb23c00c810 comp rx=0 tx=0).stop 2024-05-08T23:31:38.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.220+0000 7fb25178a700 1 -- 172.21.15.81:0/718590267 shutdown_connections 2024-05-08T23:31:38.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.220+0000 7fb25178a700 1 --2- 172.21.15.81:0/718590267 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb22c066730 0x7fb22c068bf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.220+0000 7fb25178a700 1 --2- 172.21.15.81:0/718590267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb24c15cc80 0x7fb24c155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.220+0000 7fb25178a700 1 --2- 172.21.15.81:0/718590267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb24c15c800 0x7fb24c14f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.220+0000 7fb25178a700 1 --2- 172.21.15.81:0/718590267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb24c155690 0x7fb24c0b0390 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.220+0000 7fb25178a700 1 -- 172.21.15.81:0/718590267 >> 172.21.15.81:0/718590267 conn(0x7fb24c099a70 msgr2=0x7fb24c153350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:38.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.220+0000 7fb25178a700 1 -- 172.21.15.81:0/718590267 shutdown_connections 2024-05-08T23:31:38.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.220+0000 7fb25178a700 1 -- 172.21.15.81:0/718590267 wait complete. 2024-05-08T23:31:38.237 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:38.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf3498f700 1 Processor -- start 2024-05-08T23:31:38.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf3498f700 1 -- start start 2024-05-08T23:31:38.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf3498f700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdf300b3c20 0x7fdf301624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf3498f700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdf30071e70 0x7fdf300b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf3498f700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf30158960 con 0x7fdf30165070 2024-05-08T23:31:38.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf3498f700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf300b1f60 con 0x7fdf30071e70 2024-05-08T23:31:38.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf3498f700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf30071120 con 0x7fdf300b3c20 2024-05-08T23:31:38.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf26ffd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdf30071e70 0x7fdf300b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf26ffd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdf30071e70 0x7fdf300b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60452/0 (socket says 172.21.15.81:60452) 2024-05-08T23:31:38.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf26ffd700 1 -- 172.21.15.81:0/2320830260 learned_addr learned my addr 172.21.15.81:0/2320830260 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:38.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf2f7fe700 1 --2- 172.21.15.81:0/2320830260 >> v2:172.21.15.117:3300/0 conn(0x7fdf300b3c20 0x7fdf301624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf26ffd700 1 -- 172.21.15.81:0/2320830260 >> v2:172.21.15.117:3300/0 conn(0x7fdf300b3c20 msgr2=0x7fdf301624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:38.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf26ffd700 1 --2- 172.21.15.81:0/2320830260 >> v2:172.21.15.117:3300/0 conn(0x7fdf300b3c20 0x7fdf301624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf26ffd700 1 -- 172.21.15.81:0/2320830260 >> v1:172.21.15.81:6789/0 conn(0x7fdf30165070 legacy=0x7fdf301654e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf26ffd700 1 -- 172.21.15.81:0/2320830260 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf300b4540 con 0x7fdf30071e70 2024-05-08T23:31:38.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf2f7fe700 1 --2- 172.21.15.81:0/2320830260 >> v2:172.21.15.117:3300/0 conn(0x7fdf300b3c20 0x7fdf301624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:38.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf26ffd700 1 --2- 172.21.15.81:0/2320830260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf30071e70 0x7fdf300b4000 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdf3006eba0 tx=0x7fdf18009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0229ee48164fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:38.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf2effd700 1 -- 172.21.15.81:0/2320830260 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800b8a0 con 0x7fdf30071e70 2024-05-08T23:31:38.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf2effd700 1 -- 172.21.15.81:0/2320830260 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1800ba00 con 0x7fdf30071e70 2024-05-08T23:31:38.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.348+0000 7fdf2effd700 1 -- 172.21.15.81:0/2320830260 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800bd90 con 0x7fdf30071e70 2024-05-08T23:31:38.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 -- 172.21.15.81:0/2320830260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf30071e70 msgr2=0x7fdf300b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 --2- 172.21.15.81:0/2320830260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf30071e70 0x7fdf300b4000 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdf3006eba0 tx=0x7fdf18009bf0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 -- 172.21.15.81:0/2320830260 shutdown_connections 2024-05-08T23:31:38.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 --2- 172.21.15.81:0/2320830260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf30071e70 0x7fdf300b4000 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 --2- 172.21.15.81:0/2320830260 >> v2:172.21.15.117:3300/0 conn(0x7fdf300b3c20 0x7fdf301624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 -- 172.21.15.81:0/2320830260 >> 172.21.15.81:0/2320830260 conn(0x7fdf3006c330 msgr2=0x7fdf3006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:38.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 -- 172.21.15.81:0/2320830260 shutdown_connections 2024-05-08T23:31:38.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 -- 172.21.15.81:0/2320830260 wait complete. 2024-05-08T23:31:38.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 Processor -- start 2024-05-08T23:31:38.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 -- start start 2024-05-08T23:31:38.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf30071e70 0x7fdf301467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf300b3c20 0x7fdf301ecde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf30165070 0x7fdf30145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf30146cf0 con 0x7fdf30165070 2024-05-08T23:31:38.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf301ec480 con 0x7fdf300b3c20 2024-05-08T23:31:38.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf30158bb0 con 0x7fdf30071e70 2024-05-08T23:31:38.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf2ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf30165070 0x7fdf30145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf2ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf30165070 0x7fdf30145b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60468/0 (socket says 172.21.15.81:60468) 2024-05-08T23:31:38.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf2ffff700 1 -- 172.21.15.81:0/1126279385 learned_addr learned my addr 172.21.15.81:0/1126279385 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:38.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf26ffd700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf300b3c20 0x7fdf301ecde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf2f7fe700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf30071e70 0x7fdf301467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf2ffff700 1 -- 172.21.15.81:0/1126279385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf30071e70 msgr2=0x7fdf301467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf2ffff700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf30071e70 0x7fdf301467b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf2ffff700 1 -- 172.21.15.81:0/1126279385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf300b3c20 msgr2=0x7fdf301ecde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf2ffff700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf300b3c20 0x7fdf301ecde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf2ffff700 1 -- 172.21.15.81:0/1126279385 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf18009570 con 0x7fdf30165070 2024-05-08T23:31:38.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf26ffd700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf300b3c20 0x7fdf301ecde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:38.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf2f7fe700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf30071e70 0x7fdf301467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:38.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf2ffff700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf30165070 0x7fdf30145b80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800cbc0 tx=0x7fdf2800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:38.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf2cff9700 1 -- 172.21.15.81:0/1126279385 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28003c20 con 0x7fdf30165070 2024-05-08T23:31:38.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf2cff9700 1 -- 172.21.15.81:0/1126279385 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf28003d80 con 0x7fdf30165070 2024-05-08T23:31:38.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 -- 172.21.15.81:0/1126279385 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf30158e90 con 0x7fdf30165070 2024-05-08T23:31:38.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf2cff9700 1 -- 172.21.15.81:0/1126279385 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2800ab30 con 0x7fdf30165070 2024-05-08T23:31:38.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.352+0000 7fdf3498f700 1 -- 172.21.15.81:0/1126279385 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf30159450 con 0x7fdf30165070 2024-05-08T23:31:38.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.356+0000 7fdf3498f700 1 -- 172.21.15.81:0/1126279385 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf30159640 con 0x7fdf30165070 2024-05-08T23:31:38.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.356+0000 7fdf2cff9700 1 -- 172.21.15.81:0/1126279385 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf2800ac90 con 0x7fdf30165070 2024-05-08T23:31:38.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.356+0000 7fdf2cff9700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdf10066730 0x7fdf10068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.356+0000 7fdf2cff9700 1 -- 172.21.15.81:0/1126279385 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf28084280 con 0x7fdf30165070 2024-05-08T23:31:38.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.356+0000 7fdf2f7fe700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdf10066730 0x7fdf10068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.356+0000 7fdf2f7fe700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdf10066730 0x7fdf10068bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdf20008930 tx=0x7fdf20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:38.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.364+0000 7fdf2cff9700 1 -- 172.21.15.81:0/1126279385 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdf280514e0 con 0x7fdf30165070 2024-05-08T23:31:38.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.536+0000 7fdf3498f700 1 -- 172.21.15.81:0/1126279385 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf300684b0 con 0x7fdf30165070 2024-05-08T23:31:38.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.536+0000 7fdf2cff9700 1 -- 172.21.15.81:0/1126279385 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16114 (secure 0 0 0) 0x7fdf28054b00 con 0x7fdf30165070 2024-05-08T23:31:38.539 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:38.539 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":35,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:37.888485+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:31:35.620423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:31:36.878524+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:38.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.540+0000 7fdf3498f700 1 -- 172.21.15.81:0/1126279385 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdf10066730 msgr2=0x7fdf10068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.540+0000 7fdf3498f700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdf10066730 0x7fdf10068bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdf20008930 tx=0x7fdf20008040 comp rx=0 tx=0).stop 2024-05-08T23:31:38.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.540+0000 7fdf3498f700 1 -- 172.21.15.81:0/1126279385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf30165070 msgr2=0x7fdf30145b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.540+0000 7fdf3498f700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf30165070 0x7fdf30145b80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800cbc0 tx=0x7fdf2800cf80 comp rx=0 tx=0).stop 2024-05-08T23:31:38.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.540+0000 7fdf3498f700 1 -- 172.21.15.81:0/1126279385 shutdown_connections 2024-05-08T23:31:38.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.540+0000 7fdf3498f700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdf10066730 0x7fdf10068bf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.540+0000 7fdf3498f700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdf30165070 0x7fdf30145b80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.540+0000 7fdf3498f700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf300b3c20 0x7fdf301ecde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.540+0000 7fdf3498f700 1 --2- 172.21.15.81:0/1126279385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf30071e70 0x7fdf301467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.540+0000 7fdf3498f700 1 -- 172.21.15.81:0/1126279385 >> 172.21.15.81:0/1126279385 conn(0x7fdf3006c330 msgr2=0x7fdf300ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:38.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.540+0000 7fdf3498f700 1 -- 172.21.15.81:0/1126279385 shutdown_connections 2024-05-08T23:31:38.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.540+0000 7fdf3498f700 1 -- 172.21.15.81:0/1126279385 wait complete. 2024-05-08T23:31:38.553 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T23:31:38.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd3345f700 1 Processor -- start 2024-05-08T23:31:38.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd3345f700 1 -- start start 2024-05-08T23:31:38.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd3345f700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbd2c15b860 0x7fbd2c0ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd3345f700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbd2c162bc0 0x7fbd2c164fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd3345f700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd2c155ad0 con 0x7fbd2c0aed90 2024-05-08T23:31:38.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd3345f700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd2c0cf2d0 con 0x7fbd2c15b860 2024-05-08T23:31:38.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd3345f700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd2c0b3df0 con 0x7fbd2c162bc0 2024-05-08T23:31:38.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd3245d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbd2c15b860 0x7fbd2c0ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd31c5c700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fbd2c0aed90 0x7fbd2c15bc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58198/0 (socket says 172.21.15.81:58198) 2024-05-08T23:31:38.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd31c5c700 1 -- 172.21.15.81:0/378093138 learned_addr learned my addr 172.21.15.81:0/378093138 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:38.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd32c5e700 1 --2- 172.21.15.81:0/378093138 >> v2:172.21.15.117:3300/0 conn(0x7fbd2c162bc0 0x7fbd2c164fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd3245d700 1 -- 172.21.15.81:0/378093138 >> v2:172.21.15.117:3300/0 conn(0x7fbd2c162bc0 msgr2=0x7fbd2c164fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:38.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd3245d700 1 --2- 172.21.15.81:0/378093138 >> v2:172.21.15.117:3300/0 conn(0x7fbd2c162bc0 0x7fbd2c164fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd3145b700 1 -- 172.21.15.81:0/378093138 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2982647273 0 0) 0x7fbd1c001110 con 0x7fbd2c0aed90 2024-05-08T23:31:38.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd3245d700 1 -- 172.21.15.81:0/378093138 >> v1:172.21.15.81:6789/0 conn(0x7fbd2c0aed90 legacy=0x7fbd2c15bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd3245d700 1 -- 172.21.15.81:0/378093138 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2c15d780 con 0x7fbd2c15b860 2024-05-08T23:31:38.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.668+0000 7fbd3245d700 1 --2- 172.21.15.81:0/378093138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd2c15b860 0x7fbd2c0ae850 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0b4c40 tx=0x7fbd28009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be313733a97c6150 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:38.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.672+0000 7fbd3145b700 1 -- 172.21.15.81:0/378093138 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28011830 con 0x7fbd2c15b860 2024-05-08T23:31:38.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.672+0000 7fbd3145b700 1 -- 172.21.15.81:0/378093138 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd28011990 con 0x7fbd2c15b860 2024-05-08T23:31:38.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.672+0000 7fbd3145b700 1 -- 172.21.15.81:0/378093138 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28011d20 con 0x7fbd2c15b860 2024-05-08T23:31:38.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.672+0000 7fbd3345f700 1 -- 172.21.15.81:0/378093138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd2c15b860 msgr2=0x7fbd2c0ae850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.672+0000 7fbd3345f700 1 --2- 172.21.15.81:0/378093138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd2c15b860 0x7fbd2c0ae850 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0b4c40 tx=0x7fbd28009b30 comp rx=0 tx=0).stop 2024-05-08T23:31:38.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.672+0000 7fbd3345f700 1 -- 172.21.15.81:0/378093138 shutdown_connections 2024-05-08T23:31:38.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.672+0000 7fbd3345f700 1 --2- 172.21.15.81:0/378093138 >> v2:172.21.15.117:3300/0 conn(0x7fbd2c162bc0 0x7fbd2c164fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.672+0000 7fbd3345f700 1 --2- 172.21.15.81:0/378093138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd2c15b860 0x7fbd2c0ae850 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.672+0000 7fbd3345f700 1 -- 172.21.15.81:0/378093138 >> 172.21.15.81:0/378093138 conn(0x7fbd2c0717b0 msgr2=0x7fbd2c06a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:38.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.672+0000 7fbd3345f700 1 -- 172.21.15.81:0/378093138 shutdown_connections 2024-05-08T23:31:38.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.672+0000 7fbd3345f700 1 -- 172.21.15.81:0/378093138 wait complete. 2024-05-08T23:31:38.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd3345f700 1 Processor -- start 2024-05-08T23:31:38.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd3345f700 1 -- start start 2024-05-08T23:31:38.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd3345f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd2c0aed90 0x7fbd2c14b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd3345f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd2c15b860 0x7fbd2c15f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd3345f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd2c162bc0 0x7fbd2c14a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd3345f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c14c0c0 con 0x7fbd2c162bc0 2024-05-08T23:31:38.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd3345f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c15fa80 con 0x7fbd2c0aed90 2024-05-08T23:31:38.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd3345f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c14bde0 con 0x7fbd2c15b860 2024-05-08T23:31:38.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd32c5e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd2c162bc0 0x7fbd2c14a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd32c5e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd2c162bc0 0x7fbd2c14a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60492/0 (socket says 172.21.15.81:60492) 2024-05-08T23:31:38.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd32c5e700 1 -- 172.21.15.81:0/1674493190 learned_addr learned my addr 172.21.15.81:0/1674493190 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:38.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd3245d700 1 --2- 172.21.15.81:0/1674493190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd2c0aed90 0x7fbd2c14b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd31c5c700 1 --2- 172.21.15.81:0/1674493190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd2c15b860 0x7fbd2c15f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd32c5e700 1 -- 172.21.15.81:0/1674493190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd2c15b860 msgr2=0x7fbd2c15f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd32c5e700 1 --2- 172.21.15.81:0/1674493190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd2c15b860 0x7fbd2c15f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd32c5e700 1 -- 172.21.15.81:0/1674493190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd2c0aed90 msgr2=0x7fbd2c14b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:38.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd32c5e700 1 --2- 172.21.15.81:0/1674493190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd2c0aed90 0x7fbd2c14b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:38.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd32c5e700 1 -- 172.21.15.81:0/1674493190 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd280094b0 con 0x7fbd2c162bc0 2024-05-08T23:31:38.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd31c5c700 1 --2- 172.21.15.81:0/1674493190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd2c15b860 0x7fbd2c15f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:38.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd32c5e700 1 --2- 172.21.15.81:0/1674493190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd2c162bc0 0x7fbd2c14a1d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400efc0 tx=0x7fbd2400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:38.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.676+0000 7fbd22ffd700 1 -- 172.21.15.81:0/1674493190 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2400cb80 con 0x7fbd2c162bc0 2024-05-08T23:31:38.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.680+0000 7fbd22ffd700 1 -- 172.21.15.81:0/1674493190 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2400f040 con 0x7fbd2c162bc0 2024-05-08T23:31:38.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.680+0000 7fbd22ffd700 1 -- 172.21.15.81:0/1674493190 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd24014480 con 0x7fbd2c162bc0 2024-05-08T23:31:38.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.680+0000 7fbd3345f700 1 -- 172.21.15.81:0/1674493190 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2c1611b0 con 0x7fbd2c162bc0 2024-05-08T23:31:38.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.680+0000 7fbd3345f700 1 -- 172.21.15.81:0/1674493190 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2c161770 con 0x7fbd2c162bc0 2024-05-08T23:31:38.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.680+0000 7fbd3345f700 1 -- 172.21.15.81:0/1674493190 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd2c160ff0 con 0x7fbd2c162bc0 2024-05-08T23:31:38.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.680+0000 7fbd22ffd700 1 -- 172.21.15.81:0/1674493190 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbd240145e0 con 0x7fbd2c162bc0 2024-05-08T23:31:38.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.680+0000 7fbd22ffd700 1 --2- 172.21.15.81:0/1674493190 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbd10066730 0x7fbd10068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:38.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.680+0000 7fbd3245d700 1 --2- 172.21.15.81:0/1674493190 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbd10066730 0x7fbd10068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:38.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.680+0000 7fbd22ffd700 1 -- 172.21.15.81:0/1674493190 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd24084930 con 0x7fbd2c162bc0 2024-05-08T23:31:38.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.680+0000 7fbd3245d700 1 --2- 172.21.15.81:0/1674493190 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbd10066730 0x7fbd10068bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbd280090b0 tx=0x7fbd28008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:38.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.684+0000 7fbd22ffd700 1 -- 172.21.15.81:0/1674493190 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbd24051d30 con 0x7fbd2c162bc0 2024-05-08T23:31:38.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:38.892+0000 7fbd3345f700 1 -- 172.21.15.81:0/1674493190 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbd2c0672a0 con 0x7fbd2c162bc0 2024-05-08T23:31:39.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:39.900+0000 7fbd22ffd700 1 -- 172.21.15.81:0/1674493190 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbd240180b0 con 0x7fbd2c162bc0 2024-05-08T23:31:39.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:39.904+0000 7fbd3345f700 1 -- 172.21.15.81:0/1674493190 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbd10066730 msgr2=0x7fbd10068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:39.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:39.904+0000 7fbd3345f700 1 --2- 172.21.15.81:0/1674493190 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbd10066730 0x7fbd10068bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbd280090b0 tx=0x7fbd28008030 comp rx=0 tx=0).stop 2024-05-08T23:31:39.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:39.904+0000 7fbd3345f700 1 -- 172.21.15.81:0/1674493190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd2c162bc0 msgr2=0x7fbd2c14a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:39.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:39.904+0000 7fbd3345f700 1 --2- 172.21.15.81:0/1674493190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd2c162bc0 0x7fbd2c14a1d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400efc0 tx=0x7fbd2400c0c0 comp rx=0 tx=0).stop 2024-05-08T23:31:39.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:39.904+0000 7fbd3345f700 1 -- 172.21.15.81:0/1674493190 shutdown_connections 2024-05-08T23:31:39.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:39.904+0000 7fbd3345f700 1 --2- 172.21.15.81:0/1674493190 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbd10066730 0x7fbd10068bf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:39.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:39.904+0000 7fbd3345f700 1 --2- 172.21.15.81:0/1674493190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd2c162bc0 0x7fbd2c14a1d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:39.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:39.904+0000 7fbd3345f700 1 --2- 172.21.15.81:0/1674493190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd2c15b860 0x7fbd2c15f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:39.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:39.904+0000 7fbd3345f700 1 --2- 172.21.15.81:0/1674493190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd2c0aed90 0x7fbd2c14b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:39.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:39.904+0000 7fbd3345f700 1 -- 172.21.15.81:0/1674493190 >> 172.21.15.81:0/1674493190 conn(0x7fbd2c0717b0 msgr2=0x7fbd2c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:39.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:39.904+0000 7fbd3345f700 1 -- 172.21.15.81:0/1674493190 shutdown_connections 2024-05-08T23:31:39.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:39.904+0000 7fbd3345f700 1 -- 172.21.15.81:0/1674493190 wait complete. 2024-05-08T23:31:39.922 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T23:31:39.922 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T23:31:40.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.044+0000 7fe4f63e2700 1 Processor -- start 2024-05-08T23:31:40.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.044+0000 7fe4f63e2700 1 -- start start 2024-05-08T23:31:40.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.044+0000 7fe4f63e2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe4f0160920 0x7fe4f00ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:40.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.044+0000 7fe4f63e2700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe4f00d0050 0x7fe4f00b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:40.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.044+0000 7fe4f63e2700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f00d3d40 con 0x7fe4f00d1da0 2024-05-08T23:31:40.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.044+0000 7fe4f63e2700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f00d0d00 con 0x7fe4f0160920 2024-05-08T23:31:40.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.044+0000 7fe4f63e2700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f00d0fe0 con 0x7fe4f00d0050 2024-05-08T23:31:40.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.044+0000 7fe4f53e0700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe4f0160920 0x7fe4f00ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:40.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.044+0000 7fe4f4bdf700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe4f00d1da0 0x7fe4f00cf940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58212/0 (socket says 172.21.15.81:58212) 2024-05-08T23:31:40.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.044+0000 7fe4f4bdf700 1 -- 172.21.15.81:0/3818284157 learned_addr learned my addr 172.21.15.81:0/3818284157 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:40.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f5be1700 1 --2- 172.21.15.81:0/3818284157 >> v2:172.21.15.117:3300/0 conn(0x7fe4f00d0050 0x7fe4f00b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:40.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f53e0700 1 -- 172.21.15.81:0/3818284157 >> v2:172.21.15.117:3300/0 conn(0x7fe4f00d0050 msgr2=0x7fe4f00b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:40.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f53e0700 1 --2- 172.21.15.81:0/3818284157 >> v2:172.21.15.117:3300/0 conn(0x7fe4f00d0050 0x7fe4f00b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:40.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f53e0700 1 -- 172.21.15.81:0/3818284157 >> v1:172.21.15.81:6789/0 conn(0x7fe4f00d1da0 legacy=0x7fe4f00cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:40.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4e7fff700 1 -- 172.21.15.81:0/3818284157 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3290378260 0 0) 0x7fe4f00d3d40 con 0x7fe4f00d1da0 2024-05-08T23:31:40.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f53e0700 1 -- 172.21.15.81:0/3818284157 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f00d12c0 con 0x7fe4f0160920 2024-05-08T23:31:40.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f53e0700 1 --2- 172.21.15.81:0/3818284157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4f0160920 0x7fe4f00ce6f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00cd6e0 tx=0x7fe4e0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d0b084aeaa2e2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:40.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4e7fff700 1 -- 172.21.15.81:0/3818284157 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0011830 con 0x7fe4f0160920 2024-05-08T23:31:40.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4e7fff700 1 -- 172.21.15.81:0/3818284157 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e0011990 con 0x7fe4f0160920 2024-05-08T23:31:40.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4e7fff700 1 -- 172.21.15.81:0/3818284157 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0011d20 con 0x7fe4f0160920 2024-05-08T23:31:40.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f63e2700 1 -- 172.21.15.81:0/3818284157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4f0160920 msgr2=0x7fe4f00ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:40.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f63e2700 1 --2- 172.21.15.81:0/3818284157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4f0160920 0x7fe4f00ce6f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00cd6e0 tx=0x7fe4e0009b30 comp rx=0 tx=0).stop 2024-05-08T23:31:40.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f63e2700 1 -- 172.21.15.81:0/3818284157 shutdown_connections 2024-05-08T23:31:40.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f63e2700 1 --2- 172.21.15.81:0/3818284157 >> v2:172.21.15.117:3300/0 conn(0x7fe4f00d0050 0x7fe4f00b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:40.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f63e2700 1 --2- 172.21.15.81:0/3818284157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4f0160920 0x7fe4f00ce6f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:40.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f63e2700 1 -- 172.21.15.81:0/3818284157 >> 172.21.15.81:0/3818284157 conn(0x7fe4f009c340 msgr2=0x7fe4f00d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:40.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f63e2700 1 -- 172.21.15.81:0/3818284157 shutdown_connections 2024-05-08T23:31:40.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f63e2700 1 -- 172.21.15.81:0/3818284157 wait complete. 2024-05-08T23:31:40.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f63e2700 1 Processor -- start 2024-05-08T23:31:40.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f63e2700 1 -- start start 2024-05-08T23:31:40.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f63e2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4f00d0050 0x7fe4f015b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:40.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.048+0000 7fe4f63e2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4f00d1da0 0x7fe4f00b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:40.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f63e2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4f0160920 0x7fe4f00b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:40.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f63e2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00b1360 con 0x7fe4f0160920 2024-05-08T23:31:40.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f63e2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00b06e0 con 0x7fe4f00d0050 2024-05-08T23:31:40.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f63e2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00b09c0 con 0x7fe4f00d1da0 2024-05-08T23:31:40.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f5be1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4f0160920 0x7fe4f00b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:40.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f5be1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4f0160920 0x7fe4f00b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60510/0 (socket says 172.21.15.81:60510) 2024-05-08T23:31:40.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f5be1700 1 -- 172.21.15.81:0/3516146400 learned_addr learned my addr 172.21.15.81:0/3516146400 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:40.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f53e0700 1 --2- 172.21.15.81:0/3516146400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4f00d0050 0x7fe4f015b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:40.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f4bdf700 1 --2- 172.21.15.81:0/3516146400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4f00d1da0 0x7fe4f00b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:40.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f5be1700 1 -- 172.21.15.81:0/3516146400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4f00d1da0 msgr2=0x7fe4f00b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:40.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f5be1700 1 --2- 172.21.15.81:0/3516146400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4f00d1da0 0x7fe4f00b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:40.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f5be1700 1 -- 172.21.15.81:0/3516146400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4f00d0050 msgr2=0x7fe4f015b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f5be1700 1 --2- 172.21.15.81:0/3516146400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4f00d0050 0x7fe4f015b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f5be1700 1 -- 172.21.15.81:0/3516146400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e00094b0 con 0x7fe4f0160920 2024-05-08T23:31:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f4bdf700 1 --2- 172.21.15.81:0/3516146400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4f00d1da0 0x7fe4f00b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f5be1700 1 --2- 172.21.15.81:0/3516146400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4f0160920 0x7fe4f00b1db0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800efc0 tx=0x7fe4e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4e5ffb700 1 -- 172.21.15.81:0/3516146400 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8009b20 con 0x7fe4f0160920 2024-05-08T23:31:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4e5ffb700 1 -- 172.21.15.81:0/3516146400 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e800f040 con 0x7fe4f0160920 2024-05-08T23:31:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4e5ffb700 1 -- 172.21.15.81:0/3516146400 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8009e50 con 0x7fe4f0160920 2024-05-08T23:31:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f63e2700 1 -- 172.21.15.81:0/3516146400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f0160e80 con 0x7fe4f0160920 2024-05-08T23:31:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f63e2700 1 -- 172.21.15.81:0/3516146400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f00b0f80 con 0x7fe4f0160920 2024-05-08T23:31:40.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f63e2700 1 -- 172.21.15.81:0/3516146400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f00684b0 con 0x7fe4f0160920 2024-05-08T23:31:40.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4e5ffb700 1 -- 172.21.15.81:0/3516146400 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe4e8014480 con 0x7fe4f0160920 2024-05-08T23:31:40.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4e5ffb700 1 --2- 172.21.15.81:0/3516146400 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe4d4066730 0x7fe4d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4f53e0700 1 --2- 172.21.15.81:0/3516146400 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe4d4066730 0x7fe4d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.052+0000 7fe4e5ffb700 1 -- 172.21.15.81:0/3516146400 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4e8084c90 con 0x7fe4f0160920 2024-05-08T23:31:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.056+0000 7fe4f53e0700 1 --2- 172.21.15.81:0/3516146400 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe4d4066730 0x7fe4d4068bf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0000c00 tx=0x7fe4e001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:40.065 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.064+0000 7fe4e5ffb700 1 -- 172.21.15.81:0/3516146400 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe4e80520b0 con 0x7fe4f0160920 2024-05-08T23:31:40.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.276+0000 7fe4f63e2700 1 -- 172.21.15.81:0/3516146400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe4f00cf510 con 0x7fe4f0160920 2024-05-08T23:31:40.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.908+0000 7fe4e5ffb700 1 -- 172.21.15.81:0/3516146400 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe4e8018070 con 0x7fe4f0160920 2024-05-08T23:31:40.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.912+0000 7fe4f63e2700 1 -- 172.21.15.81:0/3516146400 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe4d4066730 msgr2=0x7fe4d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:40.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.912+0000 7fe4f63e2700 1 --2- 172.21.15.81:0/3516146400 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe4d4066730 0x7fe4d4068bf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0000c00 tx=0x7fe4e001c040 comp rx=0 tx=0).stop 2024-05-08T23:31:40.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.912+0000 7fe4f63e2700 1 -- 172.21.15.81:0/3516146400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4f0160920 msgr2=0x7fe4f00b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:40.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.912+0000 7fe4f63e2700 1 --2- 172.21.15.81:0/3516146400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4f0160920 0x7fe4f00b1db0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800efc0 tx=0x7fe4e800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:31:40.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.912+0000 7fe4f63e2700 1 -- 172.21.15.81:0/3516146400 shutdown_connections 2024-05-08T23:31:40.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.912+0000 7fe4f63e2700 1 --2- 172.21.15.81:0/3516146400 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe4d4066730 0x7fe4d4068bf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:40.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.912+0000 7fe4f63e2700 1 --2- 172.21.15.81:0/3516146400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4f0160920 0x7fe4f00b1db0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:40.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.912+0000 7fe4f63e2700 1 --2- 172.21.15.81:0/3516146400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4f00d1da0 0x7fe4f00b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:40.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.912+0000 7fe4f63e2700 1 --2- 172.21.15.81:0/3516146400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4f00d0050 0x7fe4f015b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:40.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.912+0000 7fe4f63e2700 1 -- 172.21.15.81:0/3516146400 >> 172.21.15.81:0/3516146400 conn(0x7fe4f009c340 msgr2=0x7fe4f00b7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:40.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.912+0000 7fe4f63e2700 1 -- 172.21.15.81:0/3516146400 shutdown_connections 2024-05-08T23:31:40.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:40.912+0000 7fe4f63e2700 1 -- 172.21.15.81:0/3516146400 wait complete. 2024-05-08T23:31:40.929 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T23:31:40.929 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:31:41.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716f59e700 1 Processor -- start 2024-05-08T23:31:41.049 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716f59e700 1 -- start start 2024-05-08T23:31:41.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716f59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7170160ab0 0x7f71700b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716f59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71700b4610 0x7f71700cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716f59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f71700cff00 0x7f71700d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716f59e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71700d0390 con 0x7f7170160ab0 2024-05-08T23:31:41.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716f59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7170160e90 con 0x7f71700cff00 2024-05-08T23:31:41.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716f59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71700d3e80 con 0x7f71700b4610 2024-05-08T23:31:41.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716e59c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7170160ab0 0x7f71700b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716e59c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7170160ab0 0x7f71700b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60512/0 (socket says 172.21.15.81:60512) 2024-05-08T23:31:41.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716e59c700 1 -- 172.21.15.81:0/2821907102 learned_addr learned my addr 172.21.15.81:0/2821907102 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:41.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716ed9d700 1 --2- 172.21.15.81:0/2821907102 >> v2:172.21.15.117:3300/0 conn(0x7f71700cff00 0x7f71700d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716dd9b700 1 --2- 172.21.15.81:0/2821907102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71700b4610 0x7f71700cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716e59c700 1 -- 172.21.15.81:0/2821907102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71700b4610 msgr2=0x7f71700cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:41.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716e59c700 1 --2- 172.21.15.81:0/2821907102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71700b4610 0x7f71700cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716e59c700 1 -- 172.21.15.81:0/2821907102 >> v2:172.21.15.117:3300/0 conn(0x7f71700cff00 msgr2=0x7f71700d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716e59c700 1 --2- 172.21.15.81:0/2821907102 >> v2:172.21.15.117:3300/0 conn(0x7f71700cff00 0x7f71700d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716e59c700 1 -- 172.21.15.81:0/2821907102 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71700ce780 con 0x7f7170160ab0 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716e59c700 1 --2- 172.21.15.81:0/2821907102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7170160ab0 0x7f71700b1dc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f717008d890 tx=0x7f715800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9399a7ae4b759ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716d59a700 1 -- 172.21.15.81:0/2821907102 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715800e070 con 0x7f7170160ab0 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716d59a700 1 -- 172.21.15.81:0/2821907102 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715800b910 con 0x7f7170160ab0 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716d59a700 1 -- 172.21.15.81:0/2821907102 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715800bcc0 con 0x7f7170160ab0 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716f59e700 1 -- 172.21.15.81:0/2821907102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7170160ab0 msgr2=0x7f71700b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716f59e700 1 --2- 172.21.15.81:0/2821907102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7170160ab0 0x7f71700b1dc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f717008d890 tx=0x7f715800b330 comp rx=0 tx=0).stop 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716f59e700 1 -- 172.21.15.81:0/2821907102 shutdown_connections 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716f59e700 1 --2- 172.21.15.81:0/2821907102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71700b4610 0x7f71700cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716f59e700 1 --2- 172.21.15.81:0/2821907102 >> v2:172.21.15.117:3300/0 conn(0x7f71700cff00 0x7f71700d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716f59e700 1 --2- 172.21.15.81:0/2821907102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7170160ab0 0x7f71700b1dc0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.048+0000 7f716f59e700 1 -- 172.21.15.81:0/2821907102 >> 172.21.15.81:0/2821907102 conn(0x7f717009c390 msgr2=0x7f71700d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716f59e700 1 -- 172.21.15.81:0/2821907102 shutdown_connections 2024-05-08T23:31:41.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716f59e700 1 -- 172.21.15.81:0/2821907102 wait complete. 2024-05-08T23:31:41.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716f59e700 1 Processor -- start 2024-05-08T23:31:41.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716f59e700 1 -- start start 2024-05-08T23:31:41.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716f59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71700b4610 0x7f71700b6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716f59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71700cff00 0x7f71700b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716f59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7170160ab0 0x7f71700b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716f59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717009e060 con 0x7f71700cff00 2024-05-08T23:31:41.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716f59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717009e570 con 0x7f7170160ab0 2024-05-08T23:31:41.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716f59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7170152e80 con 0x7f71700b4610 2024-05-08T23:31:41.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716dd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71700cff00 0x7f71700b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716dd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71700cff00 0x7f71700b6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60528/0 (socket says 172.21.15.81:60528) 2024-05-08T23:31:41.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716dd9b700 1 -- 172.21.15.81:0/1454023943 learned_addr learned my addr 172.21.15.81:0/1454023943 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:41.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716e59c700 1 --2- 172.21.15.81:0/1454023943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71700b4610 0x7f71700b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716ed9d700 1 --2- 172.21.15.81:0/1454023943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7170160ab0 0x7f71700b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716dd9b700 1 -- 172.21.15.81:0/1454023943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71700b4610 msgr2=0x7f71700b6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716dd9b700 1 --2- 172.21.15.81:0/1454023943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71700b4610 0x7f71700b6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716dd9b700 1 -- 172.21.15.81:0/1454023943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7170160ab0 msgr2=0x7f71700b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716dd9b700 1 --2- 172.21.15.81:0/1454023943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7170160ab0 0x7f71700b72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716dd9b700 1 -- 172.21.15.81:0/1454023943 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7158009b30 con 0x7f71700cff00 2024-05-08T23:31:41.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f716dd9b700 1 --2- 172.21.15.81:0/1454023943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71700cff00 0x7f71700b6da0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f716000e910 tx=0x7f716000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:41.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.052+0000 7f7166ffd700 1 -- 172.21.15.81:0/1454023943 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160013680 con 0x7f71700cff00 2024-05-08T23:31:41.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.056+0000 7f7166ffd700 1 -- 172.21.15.81:0/1454023943 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71600137e0 con 0x7f71700cff00 2024-05-08T23:31:41.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.056+0000 7f716f59e700 1 -- 172.21.15.81:0/1454023943 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71700ca400 con 0x7f71700cff00 2024-05-08T23:31:41.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.056+0000 7f7166ffd700 1 -- 172.21.15.81:0/1454023943 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160013b90 con 0x7f71700cff00 2024-05-08T23:31:41.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.056+0000 7f716f59e700 1 -- 172.21.15.81:0/1454023943 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71700b6340 con 0x7f71700cff00 2024-05-08T23:31:41.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.056+0000 7f716f59e700 1 -- 172.21.15.81:0/1454023943 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71700684b0 con 0x7f71700cff00 2024-05-08T23:31:41.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.056+0000 7f7166ffd700 1 -- 172.21.15.81:0/1454023943 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7160013cf0 con 0x7f71700cff00 2024-05-08T23:31:41.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.056+0000 7f7166ffd700 1 --2- 172.21.15.81:0/1454023943 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7154066730 0x7f7154068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.056+0000 7f716e59c700 1 --2- 172.21.15.81:0/1454023943 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7154066730 0x7f7154068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.056+0000 7f7166ffd700 1 -- 172.21.15.81:0/1454023943 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7160084110 con 0x7f71700cff00 2024-05-08T23:31:41.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.056+0000 7f716e59c700 1 --2- 172.21.15.81:0/1454023943 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7154066730 0x7f7154068bf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f715800b370 tx=0x7f7158009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:41.065 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.064+0000 7f7166ffd700 1 -- 172.21.15.81:0/1454023943 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f71600513e0 con 0x7f71700cff00 2024-05-08T23:31:41.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.268+0000 7f716f59e700 1 -- 172.21.15.81:0/1454023943 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f717015e090 con 0x7f71700cff00 2024-05-08T23:31:41.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.268+0000 7f7166ffd700 1 -- 172.21.15.81:0/1454023943 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f7160054a00 con 0x7f71700cff00 2024-05-08T23:31:41.271 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:41.271 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1243037934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1243037934},{"type":"v1","addr":"172.21.15.81:6835","nonce":1243037934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3124115182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3124115182},{"type":"v1","addr":"172.21.15.81:6837","nonce":3124115182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1423349691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":1423349691},{"type":"v1","addr":"172.21.15.117:6834","nonce":1423349691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:31:37.889298+0000","modified":"2024-05-08T23:31:40.903919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14852":{"gid":14852,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/3091184157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3091184157},{"type":"v1","addr":"172.21.15.117:6835","nonce":3091184157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:31:41.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.272+0000 7f716f59e700 1 -- 172.21.15.81:0/1454023943 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7154066730 msgr2=0x7f7154068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.272+0000 7f716f59e700 1 --2- 172.21.15.81:0/1454023943 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7154066730 0x7f7154068bf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f715800b370 tx=0x7f7158009000 comp rx=0 tx=0).stop 2024-05-08T23:31:41.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.272+0000 7f716f59e700 1 -- 172.21.15.81:0/1454023943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71700cff00 msgr2=0x7f71700b6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.272+0000 7f716f59e700 1 --2- 172.21.15.81:0/1454023943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71700cff00 0x7f71700b6da0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f716000e910 tx=0x7f716000ec20 comp rx=0 tx=0).stop 2024-05-08T23:31:41.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.272+0000 7f716f59e700 1 -- 172.21.15.81:0/1454023943 shutdown_connections 2024-05-08T23:31:41.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.272+0000 7f716f59e700 1 --2- 172.21.15.81:0/1454023943 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7154066730 0x7f7154068bf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.272+0000 7f716f59e700 1 --2- 172.21.15.81:0/1454023943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7170160ab0 0x7f71700b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.272+0000 7f716f59e700 1 --2- 172.21.15.81:0/1454023943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71700cff00 0x7f71700b6da0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.272+0000 7f716f59e700 1 --2- 172.21.15.81:0/1454023943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71700b4610 0x7f71700b6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.272+0000 7f716f59e700 1 -- 172.21.15.81:0/1454023943 >> 172.21.15.81:0/1454023943 conn(0x7f717009c390 msgr2=0x7f71700d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:41.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.272+0000 7f716f59e700 1 -- 172.21.15.81:0/1454023943 shutdown_connections 2024-05-08T23:31:41.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.272+0000 7f716f59e700 1 -- 172.21.15.81:0/1454023943 wait complete. 2024-05-08T23:31:41.274 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 14 2024-05-08T23:31:41.285 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:41.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.392+0000 7fcb56d23700 1 Processor -- start 2024-05-08T23:31:41.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.392+0000 7fcb56d23700 1 -- start start 2024-05-08T23:31:41.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb56d23700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb50071ab0 0x7fcb500ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb56d23700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcb50071e90 0x7fcb5015a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb56d23700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb500cea50 con 0x7fcb5015aaa0 2024-05-08T23:31:41.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb56d23700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb500ce080 con 0x7fcb50071e90 2024-05-08T23:31:41.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb56d23700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb500cd7c0 con 0x7fcb50071ab0 2024-05-08T23:31:41.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb55520700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcb50071e90 0x7fcb5015a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb55520700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcb50071e90 0x7fcb5015a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60538/0 (socket says 172.21.15.81:60538) 2024-05-08T23:31:41.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb55520700 1 -- 172.21.15.81:0/443531436 learned_addr learned my addr 172.21.15.81:0/443531436 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:41.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb55d21700 1 --2- 172.21.15.81:0/443531436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb50071ab0 0x7fcb500ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb55520700 1 -- 172.21.15.81:0/443531436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb50071ab0 msgr2=0x7fcb500ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:41.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb55520700 1 --2- 172.21.15.81:0/443531436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb50071ab0 0x7fcb500ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb55520700 1 -- 172.21.15.81:0/443531436 >> v1:172.21.15.117:6789/0 conn(0x7fcb5015aaa0 legacy=0x7fcb5013d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb54d1f700 1 -- 172.21.15.81:0/443531436 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472225962 0 0) 0x7fcb48001110 con 0x7fcb5015aaa0 2024-05-08T23:31:41.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb55520700 1 -- 172.21.15.81:0/443531436 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb500cda30 con 0x7fcb50071e90 2024-05-08T23:31:41.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb55520700 1 --2- 172.21.15.81:0/443531436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb50071e90 0x7fcb5015a560 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009e50 tx=0x7fcb4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb39ee92a3d7b15 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:41.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb54d1f700 1 -- 172.21.15.81:0/443531436 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c00e070 con 0x7fcb50071e90 2024-05-08T23:31:41.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb54d1f700 1 -- 172.21.15.81:0/443531436 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c00b910 con 0x7fcb50071e90 2024-05-08T23:31:41.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb54d1f700 1 -- 172.21.15.81:0/443531436 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c00bcc0 con 0x7fcb50071e90 2024-05-08T23:31:41.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb56d23700 1 -- 172.21.15.81:0/443531436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb50071e90 msgr2=0x7fcb5015a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb56d23700 1 --2- 172.21.15.81:0/443531436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb50071e90 0x7fcb5015a560 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009e50 tx=0x7fcb4c00b330 comp rx=0 tx=0).stop 2024-05-08T23:31:41.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb56d23700 1 -- 172.21.15.81:0/443531436 shutdown_connections 2024-05-08T23:31:41.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb56d23700 1 --2- 172.21.15.81:0/443531436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb50071e90 0x7fcb5015a560 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb56d23700 1 --2- 172.21.15.81:0/443531436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb50071ab0 0x7fcb500ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb56d23700 1 -- 172.21.15.81:0/443531436 >> 172.21.15.81:0/443531436 conn(0x7fcb5006c330 msgr2=0x7fcb5006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:41.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb56d23700 1 -- 172.21.15.81:0/443531436 shutdown_connections 2024-05-08T23:31:41.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.396+0000 7fcb56d23700 1 -- 172.21.15.81:0/443531436 wait complete. 2024-05-08T23:31:41.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56d23700 1 Processor -- start 2024-05-08T23:31:41.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56d23700 1 -- start start 2024-05-08T23:31:41.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56d23700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb50071ab0 0x7fcb50157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56d23700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb50071e90 0x7fcb501598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56d23700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb5015aaa0 0x7fcb50159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56d23700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb50158190 con 0x7fcb5015aaa0 2024-05-08T23:31:41.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56d23700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb500b3c90 con 0x7fcb50071ab0 2024-05-08T23:31:41.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56d23700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb500b3f70 con 0x7fcb50071e90 2024-05-08T23:31:41.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56522700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb5015aaa0 0x7fcb50159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56522700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb5015aaa0 0x7fcb50159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60550/0 (socket says 172.21.15.81:60550) 2024-05-08T23:31:41.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56522700 1 -- 172.21.15.81:0/3917824391 learned_addr learned my addr 172.21.15.81:0/3917824391 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:41.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb55d21700 1 --2- 172.21.15.81:0/3917824391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb50071ab0 0x7fcb50157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb55520700 1 --2- 172.21.15.81:0/3917824391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb50071e90 0x7fcb501598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56522700 1 -- 172.21.15.81:0/3917824391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb50071e90 msgr2=0x7fcb501598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56522700 1 --2- 172.21.15.81:0/3917824391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb50071e90 0x7fcb501598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56522700 1 -- 172.21.15.81:0/3917824391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb50071ab0 msgr2=0x7fcb50157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56522700 1 --2- 172.21.15.81:0/3917824391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb50071ab0 0x7fcb50157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56522700 1 -- 172.21.15.81:0/3917824391 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c009b30 con 0x7fcb5015aaa0 2024-05-08T23:31:41.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56522700 1 --2- 172.21.15.81:0/3917824391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb5015aaa0 0x7fcb50159df0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcb48007fd0 tx=0x7fcb4800c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:41.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb467fc700 1 -- 172.21.15.81:0/3917824391 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480054e0 con 0x7fcb5015aaa0 2024-05-08T23:31:41.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb467fc700 1 -- 172.21.15.81:0/3917824391 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4800ce30 con 0x7fcb5015aaa0 2024-05-08T23:31:41.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb467fc700 1 -- 172.21.15.81:0/3917824391 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480038e0 con 0x7fcb5015aaa0 2024-05-08T23:31:41.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56d23700 1 -- 172.21.15.81:0/3917824391 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb500b4250 con 0x7fcb5015aaa0 2024-05-08T23:31:41.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56d23700 1 -- 172.21.15.81:0/3917824391 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb500b4810 con 0x7fcb5015aaa0 2024-05-08T23:31:41.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.400+0000 7fcb56d23700 1 -- 172.21.15.81:0/3917824391 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb501439f0 con 0x7fcb5015aaa0 2024-05-08T23:31:41.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.404+0000 7fcb467fc700 1 -- 172.21.15.81:0/3917824391 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcb48003a40 con 0x7fcb5015aaa0 2024-05-08T23:31:41.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.404+0000 7fcb467fc700 1 --2- 172.21.15.81:0/3917824391 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcb34066730 0x7fcb34068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.404+0000 7fcb55d21700 1 --2- 172.21.15.81:0/3917824391 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcb34066730 0x7fcb34068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.404+0000 7fcb467fc700 1 -- 172.21.15.81:0/3917824391 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb480833b0 con 0x7fcb5015aaa0 2024-05-08T23:31:41.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.404+0000 7fcb55d21700 1 --2- 172.21.15.81:0/3917824391 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcb34066730 0x7fcb34068bf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009bf0 tx=0x7fcb40009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:41.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.408+0000 7fcb467fc700 1 -- 172.21.15.81:0/3917824391 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcb480506a0 con 0x7fcb5015aaa0 2024-05-08T23:31:41.469 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-05-08T23:31:41.465+0000 7fa112c82700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:31:41.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.580+0000 7fcb56d23700 1 -- 172.21.15.81:0/3917824391 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb500cea20 con 0x7fcb5015aaa0 2024-05-08T23:31:41.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.584+0000 7fcb467fc700 1 -- 172.21.15.81:0/3917824391 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16101 (secure 0 0 0) 0x7fcb48053cc0 con 0x7fcb5015aaa0 2024-05-08T23:31:41.584 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:41.584 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":36,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:38.892842+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:31:35.620423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:31:36.878524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:41.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.584+0000 7fcb56d23700 1 -- 172.21.15.81:0/3917824391 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcb34066730 msgr2=0x7fcb34068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.584+0000 7fcb56d23700 1 --2- 172.21.15.81:0/3917824391 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcb34066730 0x7fcb34068bf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009bf0 tx=0x7fcb40009240 comp rx=0 tx=0).stop 2024-05-08T23:31:41.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.584+0000 7fcb56d23700 1 -- 172.21.15.81:0/3917824391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb5015aaa0 msgr2=0x7fcb50159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.584+0000 7fcb56d23700 1 --2- 172.21.15.81:0/3917824391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb5015aaa0 0x7fcb50159df0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcb48007fd0 tx=0x7fcb4800c730 comp rx=0 tx=0).stop 2024-05-08T23:31:41.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.584+0000 7fcb56d23700 1 -- 172.21.15.81:0/3917824391 shutdown_connections 2024-05-08T23:31:41.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.584+0000 7fcb56d23700 1 --2- 172.21.15.81:0/3917824391 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcb34066730 0x7fcb34068bf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.584+0000 7fcb56d23700 1 --2- 172.21.15.81:0/3917824391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb5015aaa0 0x7fcb50159df0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.584+0000 7fcb56d23700 1 --2- 172.21.15.81:0/3917824391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb50071e90 0x7fcb501598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.584+0000 7fcb56d23700 1 --2- 172.21.15.81:0/3917824391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb50071ab0 0x7fcb50157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.584+0000 7fcb56d23700 1 -- 172.21.15.81:0/3917824391 >> 172.21.15.81:0/3917824391 conn(0x7fcb5006c330 msgr2=0x7fcb50142690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:41.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.584+0000 7fcb56d23700 1 -- 172.21.15.81:0/3917824391 shutdown_connections 2024-05-08T23:31:41.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.584+0000 7fcb56d23700 1 -- 172.21.15.81:0/3917824391 wait complete. 2024-05-08T23:31:41.600 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T23:31:41.600 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:31:41.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb501880700 1 Processor -- start 2024-05-08T23:31:41.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb501880700 1 -- start start 2024-05-08T23:31:41.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb501880700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb4fc159a80 0x7fb4fc159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb501880700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb4fc0709a0 0x7fb4fc070e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb501880700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4fc0b3590 con 0x7fb4fc1596a0 2024-05-08T23:31:41.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb501880700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4fc0b5020 con 0x7fb4fc159a80 2024-05-08T23:31:41.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb501880700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4fc0b24f0 con 0x7fb4fc0709a0 2024-05-08T23:31:41.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb50087e700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb4fc1596a0 0x7fb4fc0b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58228/0 (socket says 172.21.15.81:58228) 2024-05-08T23:31:41.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb4fbfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb4fc159a80 0x7fb4fc159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb50087e700 1 -- 172.21.15.81:0/2312847008 learned_addr learned my addr 172.21.15.81:0/2312847008 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:41.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb50107f700 1 --2- 172.21.15.81:0/2312847008 >> v2:172.21.15.117:3300/0 conn(0x7fb4fc0709a0 0x7fb4fc070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb4fbfff700 1 -- 172.21.15.81:0/2312847008 >> v2:172.21.15.117:3300/0 conn(0x7fb4fc0709a0 msgr2=0x7fb4fc070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:41.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb4fbfff700 1 --2- 172.21.15.81:0/2312847008 >> v2:172.21.15.117:3300/0 conn(0x7fb4fc0709a0 0x7fb4fc070e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb4fbfff700 1 -- 172.21.15.81:0/2312847008 >> v1:172.21.15.81:6789/0 conn(0x7fb4fc1596a0 legacy=0x7fb4fc0b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb4fbfff700 1 -- 172.21.15.81:0/2312847008 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc0b27d0 con 0x7fb4fc159a80 2024-05-08T23:31:41.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.704+0000 7fb4fb7fe700 1 -- 172.21.15.81:0/2312847008 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573942696 0 0) 0x7fb4fc0b3590 con 0x7fb4fc1596a0 2024-05-08T23:31:41.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.708+0000 7fb4fbfff700 1 --2- 172.21.15.81:0/2312847008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4fc159a80 0x7fb4fc159ef0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc15e0f0 tx=0x7fb4ec009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e955dca8661118 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:41.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.708+0000 7fb4fb7fe700 1 -- 172.21.15.81:0/2312847008 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec011830 con 0x7fb4fc159a80 2024-05-08T23:31:41.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.708+0000 7fb4fb7fe700 1 -- 172.21.15.81:0/2312847008 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec011990 con 0x7fb4fc159a80 2024-05-08T23:31:41.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.708+0000 7fb4fb7fe700 1 -- 172.21.15.81:0/2312847008 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec011d20 con 0x7fb4fc159a80 2024-05-08T23:31:41.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.708+0000 7fb501880700 1 -- 172.21.15.81:0/2312847008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4fc159a80 msgr2=0x7fb4fc159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.708+0000 7fb501880700 1 --2- 172.21.15.81:0/2312847008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4fc159a80 0x7fb4fc159ef0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc15e0f0 tx=0x7fb4ec009b30 comp rx=0 tx=0).stop 2024-05-08T23:31:41.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.708+0000 7fb501880700 1 -- 172.21.15.81:0/2312847008 shutdown_connections 2024-05-08T23:31:41.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.708+0000 7fb501880700 1 --2- 172.21.15.81:0/2312847008 >> v2:172.21.15.117:3300/0 conn(0x7fb4fc0709a0 0x7fb4fc070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.708+0000 7fb501880700 1 --2- 172.21.15.81:0/2312847008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4fc159a80 0x7fb4fc159ef0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.708+0000 7fb501880700 1 -- 172.21.15.81:0/2312847008 >> 172.21.15.81:0/2312847008 conn(0x7fb4fc06c330 msgr2=0x7fb4fc06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:41.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.708+0000 7fb501880700 1 -- 172.21.15.81:0/2312847008 shutdown_connections 2024-05-08T23:31:41.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.708+0000 7fb501880700 1 -- 172.21.15.81:0/2312847008 wait complete. 2024-05-08T23:31:41.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.708+0000 7fb501880700 1 Processor -- start 2024-05-08T23:31:41.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.708+0000 7fb501880700 1 -- start start 2024-05-08T23:31:41.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb501880700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4fc0709a0 0x7fb4fc1431a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.714 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb501880700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4fc1596a0 0x7fb4fc141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb501880700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4fc159a80 0x7fb4fc14edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb501880700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc145800 con 0x7fb4fc1596a0 2024-05-08T23:31:41.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb501880700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc144b80 con 0x7fb4fc159a80 2024-05-08T23:31:41.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb501880700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc144e60 con 0x7fb4fc0709a0 2024-05-08T23:31:41.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb4fbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4fc1596a0 0x7fb4fc141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb4fbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4fc1596a0 0x7fb4fc141ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60566/0 (socket says 172.21.15.81:60566) 2024-05-08T23:31:41.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb4fbfff700 1 -- 172.21.15.81:0/1182017790 learned_addr learned my addr 172.21.15.81:0/1182017790 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:41.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb50107f700 1 --2- 172.21.15.81:0/1182017790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4fc159a80 0x7fb4fc14edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb50087e700 1 --2- 172.21.15.81:0/1182017790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4fc0709a0 0x7fb4fc1431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb4fbfff700 1 -- 172.21.15.81:0/1182017790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4fc0709a0 msgr2=0x7fb4fc1431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb4fbfff700 1 --2- 172.21.15.81:0/1182017790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4fc0709a0 0x7fb4fc1431a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb4fbfff700 1 -- 172.21.15.81:0/1182017790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4fc159a80 msgr2=0x7fb4fc14edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb4fbfff700 1 --2- 172.21.15.81:0/1182017790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4fc159a80 0x7fb4fc14edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb4fbfff700 1 -- 172.21.15.81:0/1182017790 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ec0094b0 con 0x7fb4fc1596a0 2024-05-08T23:31:41.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb4fbfff700 1 --2- 172.21.15.81:0/1182017790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4fc1596a0 0x7fb4fc141ae0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009fd0 tx=0x7fb4ec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:41.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb4f97fa700 1 -- 172.21.15.81:0/1182017790 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec004520 con 0x7fb4fc1596a0 2024-05-08T23:31:41.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb4f97fa700 1 -- 172.21.15.81:0/1182017790 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec01f070 con 0x7fb4fc1596a0 2024-05-08T23:31:41.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb4f97fa700 1 -- 172.21.15.81:0/1182017790 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec003900 con 0x7fb4fc1596a0 2024-05-08T23:31:41.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb501880700 1 -- 172.21.15.81:0/1182017790 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4fc145140 con 0x7fb4fc1596a0 2024-05-08T23:31:41.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.712+0000 7fb501880700 1 -- 172.21.15.81:0/1182017790 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc142020 con 0x7fb4fc1596a0 2024-05-08T23:31:41.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.716+0000 7fb501880700 1 -- 172.21.15.81:0/1182017790 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4fc145b20 con 0x7fb4fc1596a0 2024-05-08T23:31:41.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.716+0000 7fb4f97fa700 1 -- 172.21.15.81:0/1182017790 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb4ec003a60 con 0x7fb4fc1596a0 2024-05-08T23:31:41.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.716+0000 7fb4f97fa700 1 --2- 172.21.15.81:0/1182017790 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb4e8066730 0x7fb4e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:41.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.716+0000 7fb4f97fa700 1 -- 172.21.15.81:0/1182017790 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4ec021070 con 0x7fb4fc1596a0 2024-05-08T23:31:41.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.716+0000 7fb50087e700 1 --2- 172.21.15.81:0/1182017790 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb4e8066730 0x7fb4e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:41.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.716+0000 7fb50087e700 1 --2- 172.21.15.81:0/1182017790 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb4e8066730 0x7fb4e8068bf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0000f60 tx=0x7fb4f0007e90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:41.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.724+0000 7fb4f97fa700 1 -- 172.21.15.81:0/1182017790 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb4ec052990 con 0x7fb4fc1596a0 2024-05-08T23:31:41.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.888+0000 7fb501880700 1 -- 172.21.15.81:0/1182017790 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4fc067b40 con 0x7fb4e8066730 2024-05-08T23:31:41.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.904+0000 7fb4f97fa700 1 -- 172.21.15.81:0/1182017790 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288005 (secure 0 0 0) 0x7fb4fc067b40 con 0x7fb4e8066730 2024-05-08T23:31:41.912 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:41.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.912+0000 7fb501880700 1 -- 172.21.15.81:0/1182017790 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb4e8066730 msgr2=0x7fb4e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.912+0000 7fb501880700 1 --2- 172.21.15.81:0/1182017790 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb4e8066730 0x7fb4e8068bf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0000f60 tx=0x7fb4f0007e90 comp rx=0 tx=0).stop 2024-05-08T23:31:41.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.912+0000 7fb501880700 1 -- 172.21.15.81:0/1182017790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4fc1596a0 msgr2=0x7fb4fc141ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:41.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.912+0000 7fb501880700 1 --2- 172.21.15.81:0/1182017790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4fc1596a0 0x7fb4fc141ae0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009fd0 tx=0x7fb4ec004770 comp rx=0 tx=0).stop 2024-05-08T23:31:41.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.912+0000 7fb501880700 1 -- 172.21.15.81:0/1182017790 shutdown_connections 2024-05-08T23:31:41.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.912+0000 7fb501880700 1 --2- 172.21.15.81:0/1182017790 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb4e8066730 0x7fb4e8068bf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.912+0000 7fb501880700 1 --2- 172.21.15.81:0/1182017790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4fc159a80 0x7fb4fc14edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.912+0000 7fb501880700 1 --2- 172.21.15.81:0/1182017790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4fc1596a0 0x7fb4fc141ae0 secure :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009fd0 tx=0x7fb4ec004770 comp rx=0 tx=0).stop 2024-05-08T23:31:41.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.912+0000 7fb501880700 1 --2- 172.21.15.81:0/1182017790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4fc0709a0 0x7fb4fc1431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:41.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.912+0000 7fb501880700 1 -- 172.21.15.81:0/1182017790 >> 172.21.15.81:0/1182017790 conn(0x7fb4fc06c330 msgr2=0x7fb4fc0b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:41.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.912+0000 7fb501880700 1 -- 172.21.15.81:0/1182017790 shutdown_connections 2024-05-08T23:31:41.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:41.912+0000 7fb501880700 1 -- 172.21.15.81:0/1182017790 wait complete. 2024-05-08T23:31:41.916 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:31:41.929 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-05-08T23:31:41.345079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220512,"kb_used_data":4336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754208,"statfs":{"total":773094113280,"available":772868308992,"internally_reserved":0,"allocated":4440064,"data_stored":2435348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.841901"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971251+0000","last_change":"2024-05-08T23:31:37.917280+0000","last_active":"2024-05-08T23:31:38.971251+0000","last_peered":"2024-05-08T23:31:38.971251+0000","last_clean":"2024-05-08T23:31:38.971251+0000","last_became_active":"2024-05-08T23:31:37.917030+0000","last_became_peered":"2024-05-08T23:31:37.917030+0000","last_unstale":"2024-05-08T23:31:38.971251+0000","last_undegraded":"2024-05-08T23:31:38.971251+0000","last_fullsized":"2024-05-08T23:31:38.971251+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:52.226149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970776+0000","last_change":"2024-05-08T23:31:37.924768+0000","last_active":"2024-05-08T23:31:38.970776+0000","last_peered":"2024-05-08T23:31:38.970776+0000","last_clean":"2024-05-08T23:31:38.970776+0000","last_became_active":"2024-05-08T23:31:37.924526+0000","last_became_peered":"2024-05-08T23:31:37.924526+0000","last_unstale":"2024-05-08T23:31:38.970776+0000","last_undegraded":"2024-05-08T23:31:38.970776+0000","last_fullsized":"2024-05-08T23:31:38.970776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:57.215271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969190+0000","last_change":"2024-05-08T23:31:37.932086+0000","last_active":"2024-05-08T23:31:38.969190+0000","last_peered":"2024-05-08T23:31:38.969190+0000","last_clean":"2024-05-08T23:31:38.969190+0000","last_became_active":"2024-05-08T23:31:37.931426+0000","last_became_peered":"2024-05-08T23:31:37.931426+0000","last_unstale":"2024-05-08T23:31:38.969190+0000","last_undegraded":"2024-05-08T23:31:38.969190+0000","last_fullsized":"2024-05-08T23:31:38.969190+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:18.676110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968039+0000","last_change":"2024-05-08T23:31:37.923671+0000","last_active":"2024-05-08T23:31:38.968039+0000","last_peered":"2024-05-08T23:31:38.968039+0000","last_clean":"2024-05-08T23:31:38.968039+0000","last_became_active":"2024-05-08T23:31:37.922905+0000","last_became_peered":"2024-05-08T23:31:37.922905+0000","last_unstale":"2024-05-08T23:31:38.968039+0000","last_undegraded":"2024-05-08T23:31:38.968039+0000","last_fullsized":"2024-05-08T23:31:38.968039+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:30:52.429721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.904935+0000","last_change":"2024-05-08T23:31:37.922675+0000","last_active":"2024-05-08T23:31:38.904935+0000","last_peered":"2024-05-08T23:31:38.904935+0000","last_clean":"2024-05-08T23:31:38.904935+0000","last_became_active":"2024-05-08T23:31:37.922431+0000","last_became_peered":"2024-05-08T23:31:37.922431+0000","last_unstale":"2024-05-08T23:31:38.904935+0000","last_undegraded":"2024-05-08T23:31:38.904935+0000","last_fullsized":"2024-05-08T23:31:38.904935+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:25.434258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969900+0000","last_change":"2024-05-08T23:31:37.917619+0000","last_active":"2024-05-08T23:31:38.969900+0000","last_peered":"2024-05-08T23:31:38.969900+0000","last_clean":"2024-05-08T23:31:38.969900+0000","last_became_active":"2024-05-08T23:31:37.917303+0000","last_became_peered":"2024-05-08T23:31:37.917303+0000","last_unstale":"2024-05-08T23:31:38.969900+0000","last_undegraded":"2024-05-08T23:31:38.969900+0000","last_fullsized":"2024-05-08T23:31:38.969900+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:06.725435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906357+0000","last_change":"2024-05-08T23:31:37.924580+0000","last_active":"2024-05-08T23:31:38.906357+0000","last_peered":"2024-05-08T23:31:38.906357+0000","last_clean":"2024-05-08T23:31:38.906357+0000","last_became_active":"2024-05-08T23:31:37.924326+0000","last_became_peered":"2024-05-08T23:31:37.924326+0000","last_unstale":"2024-05-08T23:31:38.906357+0000","last_undegraded":"2024-05-08T23:31:38.906357+0000","last_fullsized":"2024-05-08T23:31:38.906357+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:14.919240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906169+0000","last_change":"2024-05-08T23:31:37.919200+0000","last_active":"2024-05-08T23:31:38.906169+0000","last_peered":"2024-05-08T23:31:38.906169+0000","last_clean":"2024-05-08T23:31:38.906169+0000","last_became_active":"2024-05-08T23:31:37.918871+0000","last_became_peered":"2024-05-08T23:31:37.918871+0000","last_unstale":"2024-05-08T23:31:38.906169+0000","last_undegraded":"2024-05-08T23:31:38.906169+0000","last_fullsized":"2024-05-08T23:31:38.906169+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:31:53.198457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.972230+0000","last_change":"2024-05-08T23:31:37.925863+0000","last_active":"2024-05-08T23:31:38.972230+0000","last_peered":"2024-05-08T23:31:38.972230+0000","last_clean":"2024-05-08T23:31:38.972230+0000","last_became_active":"2024-05-08T23:31:37.925297+0000","last_became_peered":"2024-05-08T23:31:37.925297+0000","last_unstale":"2024-05-08T23:31:38.972230+0000","last_undegraded":"2024-05-08T23:31:38.972230+0000","last_fullsized":"2024-05-08T23:31:38.972230+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:26.477816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968176+0000","last_change":"2024-05-08T23:31:37.923544+0000","last_active":"2024-05-08T23:31:38.968176+0000","last_peered":"2024-05-08T23:31:38.968176+0000","last_clean":"2024-05-08T23:31:38.968176+0000","last_became_active":"2024-05-08T23:31:37.923174+0000","last_became_peered":"2024-05-08T23:31:37.923174+0000","last_unstale":"2024-05-08T23:31:38.968176+0000","last_undegraded":"2024-05-08T23:31:38.968176+0000","last_fullsized":"2024-05-08T23:31:38.968176+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:00.231440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906830+0000","last_change":"2024-05-08T23:31:37.923737+0000","last_active":"2024-05-08T23:31:38.906830+0000","last_peered":"2024-05-08T23:31:38.906830+0000","last_clean":"2024-05-08T23:31:38.906830+0000","last_became_active":"2024-05-08T23:31:37.923445+0000","last_became_peered":"2024-05-08T23:31:37.923445+0000","last_unstale":"2024-05-08T23:31:38.906830+0000","last_undegraded":"2024-05-08T23:31:38.906830+0000","last_fullsized":"2024-05-08T23:31:38.906830+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:01.173647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971070+0000","last_change":"2024-05-08T23:31:37.922985+0000","last_active":"2024-05-08T23:31:38.971070+0000","last_peered":"2024-05-08T23:31:38.971070+0000","last_clean":"2024-05-08T23:31:38.971070+0000","last_became_active":"2024-05-08T23:31:37.922703+0000","last_became_peered":"2024-05-08T23:31:37.922703+0000","last_unstale":"2024-05-08T23:31:38.971070+0000","last_undegraded":"2024-05-08T23:31:38.971070+0000","last_fullsized":"2024-05-08T23:31:38.971070+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:20:43.227330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905064+0000","last_change":"2024-05-08T23:31:37.931800+0000","last_active":"2024-05-08T23:31:38.905064+0000","last_peered":"2024-05-08T23:31:38.905064+0000","last_clean":"2024-05-08T23:31:38.905064+0000","last_became_active":"2024-05-08T23:31:37.931449+0000","last_became_peered":"2024-05-08T23:31:37.931449+0000","last_unstale":"2024-05-08T23:31:38.905064+0000","last_undegraded":"2024-05-08T23:31:38.905064+0000","last_fullsized":"2024-05-08T23:31:38.905064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:02.052324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970055+0000","last_change":"2024-05-08T23:31:37.917578+0000","last_active":"2024-05-08T23:31:38.970055+0000","last_peered":"2024-05-08T23:31:38.970055+0000","last_clean":"2024-05-08T23:31:38.970055+0000","last_became_active":"2024-05-08T23:31:37.917301+0000","last_became_peered":"2024-05-08T23:31:37.917301+0000","last_unstale":"2024-05-08T23:31:38.970055+0000","last_undegraded":"2024-05-08T23:31:38.970055+0000","last_fullsized":"2024-05-08T23:31:38.970055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:26.623286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968944+0000","last_change":"2024-05-08T23:31:37.922951+0000","last_active":"2024-05-08T23:31:38.968944+0000","last_peered":"2024-05-08T23:31:38.968944+0000","last_clean":"2024-05-08T23:31:38.968944+0000","last_became_active":"2024-05-08T23:31:37.922681+0000","last_became_peered":"2024-05-08T23:31:37.922681+0000","last_unstale":"2024-05-08T23:31:38.968944+0000","last_undegraded":"2024-05-08T23:31:38.968944+0000","last_fullsized":"2024-05-08T23:31:38.968944+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:26.623286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906218+0000","last_change":"2024-05-08T23:31:37.923756+0000","last_active":"2024-05-08T23:31:38.906218+0000","last_peered":"2024-05-08T23:31:38.906218+0000","last_clean":"2024-05-08T23:31:38.906218+0000","last_became_active":"2024-05-08T23:31:37.923586+0000","last_became_peered":"2024-05-08T23:31:37.923586+0000","last_unstale":"2024-05-08T23:31:38.906218+0000","last_undegraded":"2024-05-08T23:31:38.906218+0000","last_fullsized":"2024-05-08T23:31:38.906218+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:05.927456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.907246+0000","last_change":"2024-05-08T23:31:37.919928+0000","last_active":"2024-05-08T23:31:38.907246+0000","last_peered":"2024-05-08T23:31:38.907246+0000","last_clean":"2024-05-08T23:31:38.907246+0000","last_became_active":"2024-05-08T23:31:37.919638+0000","last_became_peered":"2024-05-08T23:31:37.919638+0000","last_unstale":"2024-05-08T23:31:38.907246+0000","last_undegraded":"2024-05-08T23:31:38.907246+0000","last_fullsized":"2024-05-08T23:31:38.907246+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:35.727462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970842+0000","last_change":"2024-05-08T23:31:37.920467+0000","last_active":"2024-05-08T23:31:38.970842+0000","last_peered":"2024-05-08T23:31:38.970842+0000","last_clean":"2024-05-08T23:31:38.970842+0000","last_became_active":"2024-05-08T23:31:37.920270+0000","last_became_peered":"2024-05-08T23:31:37.920270+0000","last_unstale":"2024-05-08T23:31:38.970842+0000","last_undegraded":"2024-05-08T23:31:38.970842+0000","last_fullsized":"2024-05-08T23:31:38.970842+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:57.561916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970454+0000","last_change":"2024-05-08T23:31:37.923846+0000","last_active":"2024-05-08T23:31:38.970454+0000","last_peered":"2024-05-08T23:31:38.970454+0000","last_clean":"2024-05-08T23:31:38.970454+0000","last_became_active":"2024-05-08T23:31:37.923555+0000","last_became_peered":"2024-05-08T23:31:37.923555+0000","last_unstale":"2024-05-08T23:31:38.970454+0000","last_undegraded":"2024-05-08T23:31:38.970454+0000","last_fullsized":"2024-05-08T23:31:38.970454+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:42.138710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905598+0000","last_change":"2024-05-08T23:31:37.917032+0000","last_active":"2024-05-08T23:31:38.905598+0000","last_peered":"2024-05-08T23:31:38.905598+0000","last_clean":"2024-05-08T23:31:38.905598+0000","last_became_active":"2024-05-08T23:31:37.916751+0000","last_became_peered":"2024-05-08T23:31:37.916751+0000","last_unstale":"2024-05-08T23:31:38.905598+0000","last_undegraded":"2024-05-08T23:31:38.905598+0000","last_fullsized":"2024-05-08T23:31:38.905598+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:39.604074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970071+0000","last_change":"2024-05-08T23:31:37.928252+0000","last_active":"2024-05-08T23:31:38.970071+0000","last_peered":"2024-05-08T23:31:38.970071+0000","last_clean":"2024-05-08T23:31:38.970071+0000","last_became_active":"2024-05-08T23:31:37.925818+0000","last_became_peered":"2024-05-08T23:31:37.925818+0000","last_unstale":"2024-05-08T23:31:38.970071+0000","last_undegraded":"2024-05-08T23:31:38.970071+0000","last_fullsized":"2024-05-08T23:31:38.970071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:52.226149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969619+0000","last_change":"2024-05-08T23:31:37.924930+0000","last_active":"2024-05-08T23:31:38.969619+0000","last_peered":"2024-05-08T23:31:38.969619+0000","last_clean":"2024-05-08T23:31:38.969619+0000","last_became_active":"2024-05-08T23:31:37.924672+0000","last_became_peered":"2024-05-08T23:31:37.924672+0000","last_unstale":"2024-05-08T23:31:38.969619+0000","last_undegraded":"2024-05-08T23:31:38.969619+0000","last_fullsized":"2024-05-08T23:31:38.969619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:49.865105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905058+0000","last_change":"2024-05-08T23:31:37.922277+0000","last_active":"2024-05-08T23:31:38.905058+0000","last_peered":"2024-05-08T23:31:38.905058+0000","last_clean":"2024-05-08T23:31:38.905058+0000","last_became_active":"2024-05-08T23:31:37.921985+0000","last_became_peered":"2024-05-08T23:31:37.921985+0000","last_unstale":"2024-05-08T23:31:38.905058+0000","last_undegraded":"2024-05-08T23:31:38.905058+0000","last_fullsized":"2024-05-08T23:31:38.905058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:32.867982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968841+0000","last_change":"2024-05-08T23:31:37.920044+0000","last_active":"2024-05-08T23:31:38.968841+0000","last_peered":"2024-05-08T23:31:38.968841+0000","last_clean":"2024-05-08T23:31:38.968841+0000","last_became_active":"2024-05-08T23:31:37.919598+0000","last_became_peered":"2024-05-08T23:31:37.919598+0000","last_unstale":"2024-05-08T23:31:38.968841+0000","last_undegraded":"2024-05-08T23:31:38.968841+0000","last_fullsized":"2024-05-08T23:31:38.968841+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:04.278422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968270+0000","last_change":"2024-05-08T23:31:37.931317+0000","last_active":"2024-05-08T23:31:38.968270+0000","last_peered":"2024-05-08T23:31:38.968270+0000","last_clean":"2024-05-08T23:31:38.968270+0000","last_became_active":"2024-05-08T23:31:37.931059+0000","last_became_peered":"2024-05-08T23:31:37.931059+0000","last_unstale":"2024-05-08T23:31:38.968270+0000","last_undegraded":"2024-05-08T23:31:38.968270+0000","last_fullsized":"2024-05-08T23:31:38.968270+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:19.037204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970868+0000","last_change":"2024-05-08T23:31:37.930968+0000","last_active":"2024-05-08T23:31:38.970868+0000","last_peered":"2024-05-08T23:31:38.970868+0000","last_clean":"2024-05-08T23:31:38.970868+0000","last_became_active":"2024-05-08T23:31:37.930671+0000","last_became_peered":"2024-05-08T23:31:37.930671+0000","last_unstale":"2024-05-08T23:31:38.970868+0000","last_undegraded":"2024-05-08T23:31:38.970868+0000","last_fullsized":"2024-05-08T23:31:38.970868+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:18.676110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968760+0000","last_change":"2024-05-08T23:31:37.930499+0000","last_active":"2024-05-08T23:31:38.968760+0000","last_peered":"2024-05-08T23:31:38.968760+0000","last_clean":"2024-05-08T23:31:38.968760+0000","last_became_active":"2024-05-08T23:31:37.930212+0000","last_became_peered":"2024-05-08T23:31:37.930212+0000","last_unstale":"2024-05-08T23:31:38.968760+0000","last_undegraded":"2024-05-08T23:31:38.968760+0000","last_fullsized":"2024-05-08T23:31:38.968760+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:00.197636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971198+0000","last_change":"2024-05-08T23:31:37.918834+0000","last_active":"2024-05-08T23:31:38.971198+0000","last_peered":"2024-05-08T23:31:38.971198+0000","last_clean":"2024-05-08T23:31:38.971198+0000","last_became_active":"2024-05-08T23:31:37.918578+0000","last_became_peered":"2024-05-08T23:31:37.918578+0000","last_unstale":"2024-05-08T23:31:38.971198+0000","last_undegraded":"2024-05-08T23:31:38.971198+0000","last_fullsized":"2024-05-08T23:31:38.971198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:19.037204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970273+0000","last_change":"2024-05-08T23:31:37.925892+0000","last_active":"2024-05-08T23:31:38.970273+0000","last_peered":"2024-05-08T23:31:38.970273+0000","last_clean":"2024-05-08T23:31:38.970273+0000","last_became_active":"2024-05-08T23:31:37.925620+0000","last_became_peered":"2024-05-08T23:31:37.925620+0000","last_unstale":"2024-05-08T23:31:38.970273+0000","last_undegraded":"2024-05-08T23:31:38.970273+0000","last_fullsized":"2024-05-08T23:31:38.970273+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:16.487914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906325+0000","last_change":"2024-05-08T23:31:37.919954+0000","last_active":"2024-05-08T23:31:38.906325+0000","last_peered":"2024-05-08T23:31:38.906325+0000","last_clean":"2024-05-08T23:31:38.906325+0000","last_became_active":"2024-05-08T23:31:37.919693+0000","last_became_peered":"2024-05-08T23:31:37.919693+0000","last_unstale":"2024-05-08T23:31:38.906325+0000","last_undegraded":"2024-05-08T23:31:38.906325+0000","last_fullsized":"2024-05-08T23:31:38.906325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:17.662962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970054+0000","last_change":"2024-05-08T23:31:37.918774+0000","last_active":"2024-05-08T23:31:38.970054+0000","last_peered":"2024-05-08T23:31:38.970054+0000","last_clean":"2024-05-08T23:31:38.970054+0000","last_became_active":"2024-05-08T23:31:37.918490+0000","last_became_peered":"2024-05-08T23:31:37.918490+0000","last_unstale":"2024-05-08T23:31:38.970054+0000","last_undegraded":"2024-05-08T23:31:38.970054+0000","last_fullsized":"2024-05-08T23:31:38.970054+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:37.500132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906044+0000","last_change":"2024-05-08T23:31:37.922333+0000","last_active":"2024-05-08T23:31:38.906044+0000","last_peered":"2024-05-08T23:31:38.906044+0000","last_clean":"2024-05-08T23:31:38.906044+0000","last_became_active":"2024-05-08T23:31:37.922073+0000","last_became_peered":"2024-05-08T23:31:37.922073+0000","last_unstale":"2024-05-08T23:31:38.906044+0000","last_undegraded":"2024-05-08T23:31:38.906044+0000","last_fullsized":"2024-05-08T23:31:38.906044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:04.613680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905981+0000","last_change":"2024-05-08T23:31:37.911527+0000","last_active":"2024-05-08T23:31:38.905981+0000","last_peered":"2024-05-08T23:31:38.905981+0000","last_clean":"2024-05-08T23:31:38.905981+0000","last_became_active":"2024-05-08T23:31:37.910910+0000","last_became_peered":"2024-05-08T23:31:37.910910+0000","last_unstale":"2024-05-08T23:31:38.905981+0000","last_undegraded":"2024-05-08T23:31:38.905981+0000","last_fullsized":"2024-05-08T23:31:38.905981+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:01.173647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971503+0000","last_change":"2024-05-08T23:31:37.531421+0000","last_active":"2024-05-08T23:31:38.971503+0000","last_peered":"2024-05-08T23:31:38.971503+0000","last_clean":"2024-05-08T23:31:38.971503+0000","last_became_active":"2024-05-08T23:31:37.530860+0000","last_became_peered":"2024-05-08T23:31:37.530860+0000","last_unstale":"2024-05-08T23:31:38.971503+0000","last_undegraded":"2024-05-08T23:31:38.971503+0000","last_fullsized":"2024-05-08T23:31:38.971503+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:44.074366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971317+0000","last_change":"2024-05-08T23:31:37.922943+0000","last_active":"2024-05-08T23:31:38.971317+0000","last_peered":"2024-05-08T23:31:38.971317+0000","last_clean":"2024-05-08T23:31:38.971317+0000","last_became_active":"2024-05-08T23:31:37.922709+0000","last_became_peered":"2024-05-08T23:31:37.922709+0000","last_unstale":"2024-05-08T23:31:38.971317+0000","last_undegraded":"2024-05-08T23:31:38.971317+0000","last_fullsized":"2024-05-08T23:31:38.971317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:26.477816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905929+0000","last_change":"2024-05-08T23:31:37.534937+0000","last_active":"2024-05-08T23:31:38.905929+0000","last_peered":"2024-05-08T23:31:38.905929+0000","last_clean":"2024-05-08T23:31:38.905929+0000","last_became_active":"2024-05-08T23:31:37.534671+0000","last_became_peered":"2024-05-08T23:31:37.534671+0000","last_unstale":"2024-05-08T23:31:38.905929+0000","last_undegraded":"2024-05-08T23:31:38.905929+0000","last_fullsized":"2024-05-08T23:31:38.905929+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:54:55.864126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971681+0000","last_change":"2024-05-08T23:31:37.925434+0000","last_active":"2024-05-08T23:31:38.971681+0000","last_peered":"2024-05-08T23:31:38.971681+0000","last_clean":"2024-05-08T23:31:38.971681+0000","last_became_active":"2024-05-08T23:31:37.924926+0000","last_became_peered":"2024-05-08T23:31:37.924926+0000","last_unstale":"2024-05-08T23:31:38.971681+0000","last_undegraded":"2024-05-08T23:31:38.971681+0000","last_fullsized":"2024-05-08T23:31:38.971681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:00.202616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970515+0000","last_change":"2024-05-08T23:31:37.535742+0000","last_active":"2024-05-08T23:31:38.970515+0000","last_peered":"2024-05-08T23:31:38.970515+0000","last_clean":"2024-05-08T23:31:38.970515+0000","last_became_active":"2024-05-08T23:31:37.535458+0000","last_became_peered":"2024-05-08T23:31:37.535458+0000","last_unstale":"2024-05-08T23:31:38.970515+0000","last_undegraded":"2024-05-08T23:31:38.970515+0000","last_fullsized":"2024-05-08T23:31:38.970515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:36.487313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968792+0000","last_change":"2024-05-08T23:31:37.931191+0000","last_active":"2024-05-08T23:31:38.968792+0000","last_peered":"2024-05-08T23:31:38.968792+0000","last_clean":"2024-05-08T23:31:38.968792+0000","last_became_active":"2024-05-08T23:31:37.930894+0000","last_became_peered":"2024-05-08T23:31:37.930894+0000","last_unstale":"2024-05-08T23:31:38.968792+0000","last_undegraded":"2024-05-08T23:31:38.968792+0000","last_fullsized":"2024-05-08T23:31:38.968792+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:03.410673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905914+0000","last_change":"2024-05-08T23:31:37.534246+0000","last_active":"2024-05-08T23:31:38.905914+0000","last_peered":"2024-05-08T23:31:38.905914+0000","last_clean":"2024-05-08T23:31:38.905914+0000","last_became_active":"2024-05-08T23:31:37.533972+0000","last_became_peered":"2024-05-08T23:31:37.533972+0000","last_unstale":"2024-05-08T23:31:38.905914+0000","last_undegraded":"2024-05-08T23:31:38.905914+0000","last_fullsized":"2024-05-08T23:31:38.905914+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:52.452931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906531+0000","last_change":"2024-05-08T23:31:37.921572+0000","last_active":"2024-05-08T23:31:38.906531+0000","last_peered":"2024-05-08T23:31:38.906531+0000","last_clean":"2024-05-08T23:31:38.906531+0000","last_became_active":"2024-05-08T23:31:37.921278+0000","last_became_peered":"2024-05-08T23:31:37.921278+0000","last_unstale":"2024-05-08T23:31:38.906531+0000","last_undegraded":"2024-05-08T23:31:38.906531+0000","last_fullsized":"2024-05-08T23:31:38.906531+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:08.519579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970464+0000","last_change":"2024-05-08T23:31:37.535485+0000","last_active":"2024-05-08T23:31:38.970464+0000","last_peered":"2024-05-08T23:31:38.970464+0000","last_clean":"2024-05-08T23:31:38.970464+0000","last_became_active":"2024-05-08T23:31:37.535085+0000","last_became_peered":"2024-05-08T23:31:37.535085+0000","last_unstale":"2024-05-08T23:31:38.970464+0000","last_undegraded":"2024-05-08T23:31:38.970464+0000","last_fullsized":"2024-05-08T23:31:38.970464+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:59.508639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968976+0000","last_change":"2024-05-08T23:31:37.919545+0000","last_active":"2024-05-08T23:31:38.968976+0000","last_peered":"2024-05-08T23:31:38.968976+0000","last_clean":"2024-05-08T23:31:38.968976+0000","last_became_active":"2024-05-08T23:31:37.919192+0000","last_became_peered":"2024-05-08T23:31:37.919192+0000","last_unstale":"2024-05-08T23:31:38.968976+0000","last_undegraded":"2024-05-08T23:31:38.968976+0000","last_fullsized":"2024-05-08T23:31:38.968976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:06.725435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969826+0000","last_change":"2024-05-08T23:31:37.533465+0000","last_active":"2024-05-08T23:31:38.969826+0000","last_peered":"2024-05-08T23:31:38.969826+0000","last_clean":"2024-05-08T23:31:38.969826+0000","last_became_active":"2024-05-08T23:31:37.533044+0000","last_became_peered":"2024-05-08T23:31:37.533044+0000","last_unstale":"2024-05-08T23:31:38.969826+0000","last_undegraded":"2024-05-08T23:31:38.969826+0000","last_fullsized":"2024-05-08T23:31:38.969826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:52.452931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969619+0000","last_change":"2024-05-08T23:31:37.925578+0000","last_active":"2024-05-08T23:31:38.969619+0000","last_peered":"2024-05-08T23:31:38.969619+0000","last_clean":"2024-05-08T23:31:38.969619+0000","last_became_active":"2024-05-08T23:31:37.922849+0000","last_became_peered":"2024-05-08T23:31:37.922849+0000","last_unstale":"2024-05-08T23:31:38.969619+0000","last_undegraded":"2024-05-08T23:31:38.969619+0000","last_fullsized":"2024-05-08T23:31:38.969619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:37.653737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969968+0000","last_change":"2024-05-08T23:31:37.531809+0000","last_active":"2024-05-08T23:31:38.969968+0000","last_peered":"2024-05-08T23:31:38.969968+0000","last_clean":"2024-05-08T23:31:38.969968+0000","last_became_active":"2024-05-08T23:31:37.531270+0000","last_became_peered":"2024-05-08T23:31:37.531270+0000","last_unstale":"2024-05-08T23:31:38.969968+0000","last_undegraded":"2024-05-08T23:31:38.969968+0000","last_fullsized":"2024-05-08T23:31:38.969968+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:36.640918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.967465+0000","last_change":"2024-05-08T23:31:37.919828+0000","last_active":"2024-05-08T23:31:38.967465+0000","last_peered":"2024-05-08T23:31:38.967465+0000","last_clean":"2024-05-08T23:31:38.967465+0000","last_became_active":"2024-05-08T23:31:37.919231+0000","last_became_peered":"2024-05-08T23:31:37.919231+0000","last_unstale":"2024-05-08T23:31:38.967465+0000","last_undegraded":"2024-05-08T23:31:38.967465+0000","last_fullsized":"2024-05-08T23:31:38.967465+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:42.138710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969819+0000","last_change":"2024-05-08T23:31:37.529809+0000","last_active":"2024-05-08T23:31:38.969819+0000","last_peered":"2024-05-08T23:31:38.969819+0000","last_clean":"2024-05-08T23:31:38.969819+0000","last_became_active":"2024-05-08T23:31:37.529488+0000","last_became_peered":"2024-05-08T23:31:37.529488+0000","last_unstale":"2024-05-08T23:31:38.969819+0000","last_undegraded":"2024-05-08T23:31:38.969819+0000","last_fullsized":"2024-05-08T23:31:38.969819+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:51.911491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971915+0000","last_change":"2024-05-08T23:31:37.920946+0000","last_active":"2024-05-08T23:31:38.971915+0000","last_peered":"2024-05-08T23:31:38.971915+0000","last_clean":"2024-05-08T23:31:38.971915+0000","last_became_active":"2024-05-08T23:31:37.920695+0000","last_became_peered":"2024-05-08T23:31:37.920695+0000","last_unstale":"2024-05-08T23:31:38.971915+0000","last_undegraded":"2024-05-08T23:31:38.971915+0000","last_fullsized":"2024-05-08T23:31:38.971915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:41:34.627276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971040+0000","last_change":"2024-05-08T23:31:37.536970+0000","last_active":"2024-05-08T23:31:38.971040+0000","last_peered":"2024-05-08T23:31:38.971040+0000","last_clean":"2024-05-08T23:31:38.971040+0000","last_became_active":"2024-05-08T23:31:37.536282+0000","last_became_peered":"2024-05-08T23:31:37.536282+0000","last_unstale":"2024-05-08T23:31:38.971040+0000","last_undegraded":"2024-05-08T23:31:38.971040+0000","last_fullsized":"2024-05-08T23:31:38.971040+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:51.213330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970659+0000","last_change":"2024-05-08T23:31:37.923712+0000","last_active":"2024-05-08T23:31:38.970659+0000","last_peered":"2024-05-08T23:31:38.970659+0000","last_clean":"2024-05-08T23:31:38.970659+0000","last_became_active":"2024-05-08T23:31:37.923414+0000","last_became_peered":"2024-05-08T23:31:37.923414+0000","last_unstale":"2024-05-08T23:31:38.970659+0000","last_undegraded":"2024-05-08T23:31:38.970659+0000","last_fullsized":"2024-05-08T23:31:38.970659+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:00.197636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906020+0000","last_change":"2024-05-08T23:31:37.535444+0000","last_active":"2024-05-08T23:31:38.906020+0000","last_peered":"2024-05-08T23:31:38.906020+0000","last_clean":"2024-05-08T23:31:38.906020+0000","last_became_active":"2024-05-08T23:31:37.534891+0000","last_became_peered":"2024-05-08T23:31:37.534891+0000","last_unstale":"2024-05-08T23:31:38.906020+0000","last_undegraded":"2024-05-08T23:31:38.906020+0000","last_fullsized":"2024-05-08T23:31:38.906020+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:50.257330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970618+0000","last_change":"2024-05-08T23:31:37.930846+0000","last_active":"2024-05-08T23:31:38.970618+0000","last_peered":"2024-05-08T23:31:38.970618+0000","last_clean":"2024-05-08T23:31:38.970618+0000","last_became_active":"2024-05-08T23:31:37.930592+0000","last_became_peered":"2024-05-08T23:31:37.930592+0000","last_unstale":"2024-05-08T23:31:38.970618+0000","last_undegraded":"2024-05-08T23:31:38.970618+0000","last_fullsized":"2024-05-08T23:31:38.970618+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:04.278422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971355+0000","last_change":"2024-05-08T23:31:37.535224+0000","last_active":"2024-05-08T23:31:38.971355+0000","last_peered":"2024-05-08T23:31:38.971355+0000","last_clean":"2024-05-08T23:31:38.971355+0000","last_became_active":"2024-05-08T23:31:37.534954+0000","last_became_peered":"2024-05-08T23:31:37.534954+0000","last_unstale":"2024-05-08T23:31:38.971355+0000","last_undegraded":"2024-05-08T23:31:38.971355+0000","last_fullsized":"2024-05-08T23:31:38.971355+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:32.645713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968962+0000","last_change":"2024-05-08T23:31:37.932300+0000","last_active":"2024-05-08T23:31:38.968962+0000","last_peered":"2024-05-08T23:31:38.968962+0000","last_clean":"2024-05-08T23:31:38.968962+0000","last_became_active":"2024-05-08T23:31:37.931805+0000","last_became_peered":"2024-05-08T23:31:37.931805+0000","last_unstale":"2024-05-08T23:31:38.968962+0000","last_undegraded":"2024-05-08T23:31:38.968962+0000","last_fullsized":"2024-05-08T23:31:38.968962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:32.524305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970639+0000","last_change":"2024-05-08T23:31:37.534967+0000","last_active":"2024-05-08T23:31:38.970639+0000","last_peered":"2024-05-08T23:31:38.970639+0000","last_clean":"2024-05-08T23:31:38.970639+0000","last_became_active":"2024-05-08T23:31:37.534713+0000","last_became_peered":"2024-05-08T23:31:37.534713+0000","last_unstale":"2024-05-08T23:31:38.970639+0000","last_undegraded":"2024-05-08T23:31:38.970639+0000","last_fullsized":"2024-05-08T23:31:38.970639+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:24.421439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.967617+0000","last_change":"2024-05-08T23:31:37.918810+0000","last_active":"2024-05-08T23:31:38.967617+0000","last_peered":"2024-05-08T23:31:38.967617+0000","last_clean":"2024-05-08T23:31:38.967617+0000","last_became_active":"2024-05-08T23:31:37.918530+0000","last_became_peered":"2024-05-08T23:31:37.918530+0000","last_unstale":"2024-05-08T23:31:38.967617+0000","last_undegraded":"2024-05-08T23:31:38.967617+0000","last_fullsized":"2024-05-08T23:31:38.967617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:40.390338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970663+0000","last_change":"2024-05-08T23:31:37.532553+0000","last_active":"2024-05-08T23:31:38.970663+0000","last_peered":"2024-05-08T23:31:38.970663+0000","last_clean":"2024-05-08T23:31:38.970663+0000","last_became_active":"2024-05-08T23:31:37.532258+0000","last_became_peered":"2024-05-08T23:31:37.532258+0000","last_unstale":"2024-05-08T23:31:38.970663+0000","last_undegraded":"2024-05-08T23:31:38.970663+0000","last_fullsized":"2024-05-08T23:31:38.970663+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:47:36.522089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970357+0000","last_change":"2024-05-08T23:31:37.925005+0000","last_active":"2024-05-08T23:31:38.970357+0000","last_peered":"2024-05-08T23:31:38.970357+0000","last_clean":"2024-05-08T23:31:38.970357+0000","last_became_active":"2024-05-08T23:31:37.924719+0000","last_became_peered":"2024-05-08T23:31:37.924719+0000","last_unstale":"2024-05-08T23:31:38.970357+0000","last_undegraded":"2024-05-08T23:31:38.970357+0000","last_fullsized":"2024-05-08T23:31:38.970357+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:00.231440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.967552+0000","last_change":"2024-05-08T23:31:37.530877+0000","last_active":"2024-05-08T23:31:38.967552+0000","last_peered":"2024-05-08T23:31:38.967552+0000","last_clean":"2024-05-08T23:31:38.967552+0000","last_became_active":"2024-05-08T23:31:37.530610+0000","last_became_peered":"2024-05-08T23:31:37.530610+0000","last_unstale":"2024-05-08T23:31:38.967552+0000","last_undegraded":"2024-05-08T23:31:38.967552+0000","last_fullsized":"2024-05-08T23:31:38.967552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:44.207694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906136+0000","last_change":"2024-05-08T23:31:37.922173+0000","last_active":"2024-05-08T23:31:38.906136+0000","last_peered":"2024-05-08T23:31:38.906136+0000","last_clean":"2024-05-08T23:31:38.906136+0000","last_became_active":"2024-05-08T23:31:37.921861+0000","last_became_peered":"2024-05-08T23:31:37.921861+0000","last_unstale":"2024-05-08T23:31:38.906136+0000","last_undegraded":"2024-05-08T23:31:38.906136+0000","last_fullsized":"2024-05-08T23:31:38.906136+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:06.391226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969796+0000","last_change":"2024-05-08T23:31:37.536371+0000","last_active":"2024-05-08T23:31:38.969796+0000","last_peered":"2024-05-08T23:31:38.969796+0000","last_clean":"2024-05-08T23:31:38.969796+0000","last_became_active":"2024-05-08T23:31:37.536116+0000","last_became_peered":"2024-05-08T23:31:37.536116+0000","last_unstale":"2024-05-08T23:31:38.969796+0000","last_undegraded":"2024-05-08T23:31:38.969796+0000","last_fullsized":"2024-05-08T23:31:38.969796+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:34.004562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970306+0000","last_change":"2024-05-08T23:31:37.926647+0000","last_active":"2024-05-08T23:31:38.970306+0000","last_peered":"2024-05-08T23:31:38.970306+0000","last_clean":"2024-05-08T23:31:38.970306+0000","last_became_active":"2024-05-08T23:31:37.926370+0000","last_became_peered":"2024-05-08T23:31:37.926370+0000","last_unstale":"2024-05-08T23:31:38.970306+0000","last_undegraded":"2024-05-08T23:31:38.970306+0000","last_fullsized":"2024-05-08T23:31:38.970306+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:25.434258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970011+0000","last_change":"2024-05-08T23:31:37.535568+0000","last_active":"2024-05-08T23:31:38.970011+0000","last_peered":"2024-05-08T23:31:38.970011+0000","last_clean":"2024-05-08T23:31:38.970011+0000","last_became_active":"2024-05-08T23:31:37.535188+0000","last_became_peered":"2024-05-08T23:31:37.535188+0000","last_unstale":"2024-05-08T23:31:38.970011+0000","last_undegraded":"2024-05-08T23:31:38.970011+0000","last_fullsized":"2024-05-08T23:31:38.970011+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:38.591255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971827+0000","last_change":"2024-05-08T23:31:37.917241+0000","last_active":"2024-05-08T23:31:38.971827+0000","last_peered":"2024-05-08T23:31:38.971827+0000","last_clean":"2024-05-08T23:31:38.971827+0000","last_became_active":"2024-05-08T23:31:37.916973+0000","last_became_peered":"2024-05-08T23:31:37.916973+0000","last_unstale":"2024-05-08T23:31:38.971827+0000","last_undegraded":"2024-05-08T23:31:38.971827+0000","last_fullsized":"2024-05-08T23:31:38.971827+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:35.017381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968425+0000","last_change":"2024-05-08T23:31:37.532481+0000","last_active":"2024-05-08T23:31:38.968425+0000","last_peered":"2024-05-08T23:31:38.968425+0000","last_clean":"2024-05-08T23:31:38.968425+0000","last_became_active":"2024-05-08T23:31:37.532070+0000","last_became_peered":"2024-05-08T23:31:37.532070+0000","last_unstale":"2024-05-08T23:31:38.968425+0000","last_undegraded":"2024-05-08T23:31:38.968425+0000","last_fullsized":"2024-05-08T23:31:38.968425+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:24.421439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.972223+0000","last_change":"2024-05-08T23:31:37.920666+0000","last_active":"2024-05-08T23:31:38.972223+0000","last_peered":"2024-05-08T23:31:38.972223+0000","last_clean":"2024-05-08T23:31:38.972223+0000","last_became_active":"2024-05-08T23:31:37.920382+0000","last_became_peered":"2024-05-08T23:31:37.920382+0000","last_unstale":"2024-05-08T23:31:38.972223+0000","last_undegraded":"2024-05-08T23:31:38.972223+0000","last_fullsized":"2024-05-08T23:31:38.972223+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:11.997851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971932+0000","last_change":"2024-05-08T23:31:37.529281+0000","last_active":"2024-05-08T23:31:38.971932+0000","last_peered":"2024-05-08T23:31:38.971932+0000","last_clean":"2024-05-08T23:31:38.971932+0000","last_became_active":"2024-05-08T23:31:37.529015+0000","last_became_peered":"2024-05-08T23:31:37.529015+0000","last_unstale":"2024-05-08T23:31:38.971932+0000","last_undegraded":"2024-05-08T23:31:38.971932+0000","last_fullsized":"2024-05-08T23:31:38.971932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:44.074366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968065+0000","last_change":"2024-05-08T23:31:37.922947+0000","last_active":"2024-05-08T23:31:38.968065+0000","last_peered":"2024-05-08T23:31:38.968065+0000","last_clean":"2024-05-08T23:31:38.968065+0000","last_became_active":"2024-05-08T23:31:37.921506+0000","last_became_peered":"2024-05-08T23:31:37.921506+0000","last_unstale":"2024-05-08T23:31:38.968065+0000","last_undegraded":"2024-05-08T23:31:38.968065+0000","last_fullsized":"2024-05-08T23:31:38.968065+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:54.854483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971998+0000","last_change":"2024-05-08T23:31:37.533680+0000","last_active":"2024-05-08T23:31:38.971998+0000","last_peered":"2024-05-08T23:31:38.971998+0000","last_clean":"2024-05-08T23:31:38.971998+0000","last_became_active":"2024-05-08T23:31:37.532483+0000","last_became_peered":"2024-05-08T23:31:37.532483+0000","last_unstale":"2024-05-08T23:31:38.971998+0000","last_undegraded":"2024-05-08T23:31:38.971998+0000","last_fullsized":"2024-05-08T23:31:38.971998+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:52.114816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.904901+0000","last_change":"2024-05-08T23:31:37.931134+0000","last_active":"2024-05-08T23:31:38.904901+0000","last_peered":"2024-05-08T23:31:38.904901+0000","last_clean":"2024-05-08T23:31:38.904901+0000","last_became_active":"2024-05-08T23:31:37.930888+0000","last_became_peered":"2024-05-08T23:31:37.930888+0000","last_unstale":"2024-05-08T23:31:38.904901+0000","last_undegraded":"2024-05-08T23:31:38.904901+0000","last_fullsized":"2024-05-08T23:31:38.904901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:37.500132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971928+0000","last_change":"2024-05-08T23:31:37.533142+0000","last_active":"2024-05-08T23:31:38.971928+0000","last_peered":"2024-05-08T23:31:38.971928+0000","last_clean":"2024-05-08T23:31:38.971928+0000","last_became_active":"2024-05-08T23:31:37.532854+0000","last_became_peered":"2024-05-08T23:31:37.532854+0000","last_unstale":"2024-05-08T23:31:38.971928+0000","last_undegraded":"2024-05-08T23:31:38.971928+0000","last_fullsized":"2024-05-08T23:31:38.971928+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:26.190141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905089+0000","last_change":"2024-05-08T23:31:37.926706+0000","last_active":"2024-05-08T23:31:38.905089+0000","last_peered":"2024-05-08T23:31:38.905089+0000","last_clean":"2024-05-08T23:31:38.905089+0000","last_became_active":"2024-05-08T23:31:37.926416+0000","last_became_peered":"2024-05-08T23:31:37.926416+0000","last_unstale":"2024-05-08T23:31:38.905089+0000","last_undegraded":"2024-05-08T23:31:38.905089+0000","last_fullsized":"2024-05-08T23:31:38.905089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:45.220513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905891+0000","last_change":"2024-05-08T23:31:37.526613+0000","last_active":"2024-05-08T23:31:38.905891+0000","last_peered":"2024-05-08T23:31:38.905891+0000","last_clean":"2024-05-08T23:31:38.905891+0000","last_became_active":"2024-05-08T23:31:37.526325+0000","last_became_peered":"2024-05-08T23:31:37.526325+0000","last_unstale":"2024-05-08T23:31:38.905891+0000","last_undegraded":"2024-05-08T23:31:38.905891+0000","last_fullsized":"2024-05-08T23:31:38.905891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:49.435279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905204+0000","last_change":"2024-05-08T23:31:37.925330+0000","last_active":"2024-05-08T23:31:38.905204+0000","last_peered":"2024-05-08T23:31:38.905204+0000","last_clean":"2024-05-08T23:31:38.905204+0000","last_became_active":"2024-05-08T23:31:37.925073+0000","last_became_peered":"2024-05-08T23:31:37.925073+0000","last_unstale":"2024-05-08T23:31:38.905204+0000","last_undegraded":"2024-05-08T23:31:38.905204+0000","last_fullsized":"2024-05-08T23:31:38.905204+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:35.054111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970757+0000","last_change":"2024-05-08T23:31:37.534454+0000","last_active":"2024-05-08T23:31:38.970757+0000","last_peered":"2024-05-08T23:31:38.970757+0000","last_clean":"2024-05-08T23:31:38.970757+0000","last_became_active":"2024-05-08T23:31:37.534174+0000","last_became_peered":"2024-05-08T23:31:37.534174+0000","last_unstale":"2024-05-08T23:31:38.970757+0000","last_undegraded":"2024-05-08T23:31:38.970757+0000","last_fullsized":"2024-05-08T23:31:38.970757+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:42.982299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.904937+0000","last_change":"2024-05-08T23:31:37.925161+0000","last_active":"2024-05-08T23:31:38.904937+0000","last_peered":"2024-05-08T23:31:38.904937+0000","last_clean":"2024-05-08T23:31:38.904937+0000","last_became_active":"2024-05-08T23:31:37.924905+0000","last_became_peered":"2024-05-08T23:31:37.924905+0000","last_unstale":"2024-05-08T23:31:38.904937+0000","last_undegraded":"2024-05-08T23:31:38.904937+0000","last_fullsized":"2024-05-08T23:31:38.904937+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:29.553929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970412+0000","last_change":"2024-05-08T23:31:37.532697+0000","last_active":"2024-05-08T23:31:38.970412+0000","last_peered":"2024-05-08T23:31:38.970412+0000","last_clean":"2024-05-08T23:31:38.970412+0000","last_became_active":"2024-05-08T23:31:37.532428+0000","last_became_peered":"2024-05-08T23:31:37.532428+0000","last_unstale":"2024-05-08T23:31:38.970412+0000","last_undegraded":"2024-05-08T23:31:38.970412+0000","last_fullsized":"2024-05-08T23:31:38.970412+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:41.831458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906206+0000","last_change":"2024-05-08T23:31:37.924622+0000","last_active":"2024-05-08T23:31:38.906206+0000","last_peered":"2024-05-08T23:31:38.906206+0000","last_clean":"2024-05-08T23:31:38.906206+0000","last_became_active":"2024-05-08T23:31:37.924377+0000","last_became_peered":"2024-05-08T23:31:37.924377+0000","last_unstale":"2024-05-08T23:31:38.906206+0000","last_undegraded":"2024-05-08T23:31:38.906206+0000","last_fullsized":"2024-05-08T23:31:38.906206+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:13.549112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969066+0000","last_change":"2024-05-08T23:31:37.537714+0000","last_active":"2024-05-08T23:31:38.969066+0000","last_peered":"2024-05-08T23:31:38.969066+0000","last_clean":"2024-05-08T23:31:38.969066+0000","last_became_active":"2024-05-08T23:31:37.537359+0000","last_became_peered":"2024-05-08T23:31:37.537359+0000","last_unstale":"2024-05-08T23:31:38.969066+0000","last_undegraded":"2024-05-08T23:31:38.969066+0000","last_fullsized":"2024-05-08T23:31:38.969066+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:48.852286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905146+0000","last_change":"2024-05-08T23:31:37.924407+0000","last_active":"2024-05-08T23:31:38.905146+0000","last_peered":"2024-05-08T23:31:38.905146+0000","last_clean":"2024-05-08T23:31:38.905146+0000","last_became_active":"2024-05-08T23:31:37.924132+0000","last_became_peered":"2024-05-08T23:31:37.924132+0000","last_unstale":"2024-05-08T23:31:38.905146+0000","last_undegraded":"2024-05-08T23:31:38.905146+0000","last_fullsized":"2024-05-08T23:31:38.905146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:35:12.810159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968530+0000","last_change":"2024-05-08T23:31:37.530961+0000","last_active":"2024-05-08T23:31:38.968530+0000","last_peered":"2024-05-08T23:31:38.968530+0000","last_clean":"2024-05-08T23:31:38.968530+0000","last_became_active":"2024-05-08T23:31:37.530656+0000","last_became_peered":"2024-05-08T23:31:37.530656+0000","last_unstale":"2024-05-08T23:31:38.968530+0000","last_undegraded":"2024-05-08T23:31:38.968530+0000","last_fullsized":"2024-05-08T23:31:38.968530+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:25.464997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.972342+0000","last_change":"2024-05-08T23:31:37.921784+0000","last_active":"2024-05-08T23:31:38.972342+0000","last_peered":"2024-05-08T23:31:38.972342+0000","last_clean":"2024-05-08T23:31:38.972342+0000","last_became_active":"2024-05-08T23:31:37.921385+0000","last_became_peered":"2024-05-08T23:31:37.921385+0000","last_unstale":"2024-05-08T23:31:38.972342+0000","last_undegraded":"2024-05-08T23:31:38.972342+0000","last_fullsized":"2024-05-08T23:31:38.972342+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:57.215271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906071+0000","last_change":"2024-05-08T23:31:37.529247+0000","last_active":"2024-05-08T23:31:38.906071+0000","last_peered":"2024-05-08T23:31:38.906071+0000","last_clean":"2024-05-08T23:31:38.906071+0000","last_became_active":"2024-05-08T23:31:37.528917+0000","last_became_peered":"2024-05-08T23:31:37.528917+0000","last_unstale":"2024-05-08T23:31:38.906071+0000","last_undegraded":"2024-05-08T23:31:38.906071+0000","last_fullsized":"2024-05-08T23:31:38.906071+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:30.372846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905833+0000","last_change":"2024-05-08T23:31:37.922597+0000","last_active":"2024-05-08T23:31:38.905833+0000","last_peered":"2024-05-08T23:31:38.905833+0000","last_clean":"2024-05-08T23:31:38.905833+0000","last_became_active":"2024-05-08T23:31:37.922316+0000","last_became_peered":"2024-05-08T23:31:37.922316+0000","last_unstale":"2024-05-08T23:31:38.905833+0000","last_undegraded":"2024-05-08T23:31:38.905833+0000","last_fullsized":"2024-05-08T23:31:38.905833+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:58.676646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968237+0000","last_change":"2024-05-08T23:31:37.532231+0000","last_active":"2024-05-08T23:31:38.968237+0000","last_peered":"2024-05-08T23:31:38.968237+0000","last_clean":"2024-05-08T23:31:38.968237+0000","last_became_active":"2024-05-08T23:31:37.531950+0000","last_became_peered":"2024-05-08T23:31:37.531950+0000","last_unstale":"2024-05-08T23:31:38.968237+0000","last_undegraded":"2024-05-08T23:31:38.968237+0000","last_fullsized":"2024-05-08T23:31:38.968237+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:36.487313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970664+0000","last_change":"2024-05-08T23:31:37.925173+0000","last_active":"2024-05-08T23:31:38.970664+0000","last_peered":"2024-05-08T23:31:38.970664+0000","last_clean":"2024-05-08T23:31:38.970664+0000","last_became_active":"2024-05-08T23:31:37.924920+0000","last_became_peered":"2024-05-08T23:31:37.924920+0000","last_unstale":"2024-05-08T23:31:38.970664+0000","last_undegraded":"2024-05-08T23:31:38.970664+0000","last_fullsized":"2024-05-08T23:31:38.970664+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:35.017381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905954+0000","last_change":"2024-05-08T23:31:37.527972+0000","last_active":"2024-05-08T23:31:38.905954+0000","last_peered":"2024-05-08T23:31:38.905954+0000","last_clean":"2024-05-08T23:31:38.905954+0000","last_became_active":"2024-05-08T23:31:37.527641+0000","last_became_peered":"2024-05-08T23:31:37.527641+0000","last_unstale":"2024-05-08T23:31:38.905954+0000","last_undegraded":"2024-05-08T23:31:38.905954+0000","last_fullsized":"2024-05-08T23:31:38.905954+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:24.719550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905821+0000","last_change":"2024-05-08T23:31:37.915918+0000","last_active":"2024-05-08T23:31:38.905821+0000","last_peered":"2024-05-08T23:31:38.905821+0000","last_clean":"2024-05-08T23:31:38.905821+0000","last_became_active":"2024-05-08T23:31:37.915687+0000","last_became_peered":"2024-05-08T23:31:37.915687+0000","last_unstale":"2024-05-08T23:31:38.905821+0000","last_undegraded":"2024-05-08T23:31:38.905821+0000","last_fullsized":"2024-05-08T23:31:38.905821+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:28.759487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.967722+0000","last_change":"2024-05-08T23:31:37.529483+0000","last_active":"2024-05-08T23:31:38.967722+0000","last_peered":"2024-05-08T23:31:38.967722+0000","last_clean":"2024-05-08T23:31:38.967722+0000","last_became_active":"2024-05-08T23:31:37.528480+0000","last_became_peered":"2024-05-08T23:31:37.528480+0000","last_unstale":"2024-05-08T23:31:38.967722+0000","last_undegraded":"2024-05-08T23:31:38.967722+0000","last_fullsized":"2024-05-08T23:31:38.967722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:38.591255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969726+0000","last_change":"2024-05-08T23:31:37.921085+0000","last_active":"2024-05-08T23:31:38.969726+0000","last_peered":"2024-05-08T23:31:38.969726+0000","last_clean":"2024-05-08T23:31:38.969726+0000","last_became_active":"2024-05-08T23:31:37.920833+0000","last_became_peered":"2024-05-08T23:31:37.920833+0000","last_unstale":"2024-05-08T23:31:38.969726+0000","last_undegraded":"2024-05-08T23:31:38.969726+0000","last_fullsized":"2024-05-08T23:31:38.969726+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:19.037204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970580+0000","last_change":"2024-05-08T23:31:37.534032+0000","last_active":"2024-05-08T23:31:38.970580+0000","last_peered":"2024-05-08T23:31:38.970580+0000","last_clean":"2024-05-08T23:31:38.970580+0000","last_became_active":"2024-05-08T23:31:37.532189+0000","last_became_peered":"2024-05-08T23:31:37.532189+0000","last_unstale":"2024-05-08T23:31:38.970580+0000","last_undegraded":"2024-05-08T23:31:38.970580+0000","last_fullsized":"2024-05-08T23:31:38.970580+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:26.197274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970223+0000","last_change":"2024-05-08T23:31:37.924638+0000","last_active":"2024-05-08T23:31:38.970223+0000","last_peered":"2024-05-08T23:31:38.970223+0000","last_clean":"2024-05-08T23:31:38.970223+0000","last_became_active":"2024-05-08T23:31:37.922881+0000","last_became_peered":"2024-05-08T23:31:37.922881+0000","last_unstale":"2024-05-08T23:31:38.970223+0000","last_undegraded":"2024-05-08T23:31:38.970223+0000","last_fullsized":"2024-05-08T23:31:38.970223+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:33.658532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969845+0000","last_change":"2024-05-08T23:31:37.537638+0000","last_active":"2024-05-08T23:31:38.969845+0000","last_peered":"2024-05-08T23:31:38.969845+0000","last_clean":"2024-05-08T23:31:38.969845+0000","last_became_active":"2024-05-08T23:31:37.537360+0000","last_became_peered":"2024-05-08T23:31:37.537360+0000","last_unstale":"2024-05-08T23:31:38.969845+0000","last_undegraded":"2024-05-08T23:31:38.969845+0000","last_fullsized":"2024-05-08T23:31:38.969845+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:25.464997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906666+0000","last_change":"2024-05-08T23:31:37.917985+0000","last_active":"2024-05-08T23:31:38.906666+0000","last_peered":"2024-05-08T23:31:38.906666+0000","last_clean":"2024-05-08T23:31:38.906666+0000","last_became_active":"2024-05-08T23:31:37.917678+0000","last_became_peered":"2024-05-08T23:31:37.917678+0000","last_unstale":"2024-05-08T23:31:38.906666+0000","last_undegraded":"2024-05-08T23:31:38.906666+0000","last_fullsized":"2024-05-08T23:31:38.906666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:14.429360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970885+0000","last_change":"2024-05-08T23:31:37.535656+0000","last_active":"2024-05-08T23:31:38.970885+0000","last_peered":"2024-05-08T23:31:38.970885+0000","last_clean":"2024-05-08T23:31:38.970885+0000","last_became_active":"2024-05-08T23:31:37.535231+0000","last_became_peered":"2024-05-08T23:31:37.535231+0000","last_unstale":"2024-05-08T23:31:38.970885+0000","last_undegraded":"2024-05-08T23:31:38.970885+0000","last_fullsized":"2024-05-08T23:31:38.970885+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:36.141659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":175,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-08T23:31:37.902873+0000","last_change":"2024-05-08T23:30:27.226484+0000","last_active":"2024-05-08T23:31:37.902873+0000","last_peered":"2024-05-08T23:31:37.902873+0000","last_clean":"2024-05-08T23:31:37.902873+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:31:37.902873+0000","last_undegraded":"2024-05-08T23:31:37.902873+0000","last_fullsized":"2024-05-08T23:31:37.902873+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_clean_scrub_stamp":"2024-05-08T23:30:26.204118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:14.267287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":158427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.84599999999999997}]},{"osd":1,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.998}]},{"osd":2,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.1479999999999999}]},{"osd":3,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.79500000000000004}]},{"osd":4,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.94699999999999995}]},{"osd":5,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.105},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.89200000000000002}]},{"osd":7,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":1.048},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.018}]}]},{"osd":5,"up_from":15,"seq":64424509460,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":158483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.056},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.5449999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.90300000000000002}]},{"osd":1,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.752}]},{"osd":2,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.242}]},{"osd":3,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.03},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.1659999999999999}]},{"osd":4,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.464,"5min":2.464,"15min":2.464},"last":1.099}]},{"osd":6,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.1419999999999999}]},{"osd":7,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":1.4359999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":157915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:31:25 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.54200000000000004}]},{"osd":1,"last update":"Wed May 8 23:31:25 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.51300000000000001}]},{"osd":2,"last update":"Wed May 8 23:31:25 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed May 8 23:31:25 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.70999999999999996}]},{"osd":5,"last update":"Wed May 8 23:31:25 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.997}]},{"osd":6,"last update":"Wed May 8 23:31:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.77800000000000002}]},{"osd":7,"last update":"Wed May 8 23:31:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.036},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.95399999999999996}]}]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":159397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.48199999999999998}]},{"osd":1,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.56000000000000005}]},{"osd":2,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.99399999999999999}]},{"osd":4,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.86599999999999999}]},{"osd":5,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.0580000000000001}]},{"osd":6,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.90300000000000002}]},{"osd":7,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.92600000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":157881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.747},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.442}]},{"osd":1,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.621}]},{"osd":3,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":1.3420000000000001}]},{"osd":5,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.875}]},{"osd":6,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.0920000000000001}]},{"osd":7,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.056},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.492}]}]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":159397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.73399999999999999}]},{"osd":3,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.4740000000000002,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.85899999999999999}]},{"osd":5,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.03},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.5830000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.95899999999999996}]},{"osd":6,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.88}]},{"osd":7,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.91400000000000003}]}]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":741969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.73599999999999999}]},{"osd":1,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":1.016},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.871}]},{"osd":2,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.89700000000000002}]},{"osd":3,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":1.137},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.82099999999999995}]},{"osd":4,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":1.056},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.92900000000000005}]},{"osd":5,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.76900000000000002}]},{"osd":6,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.84099999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":741879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.69199999999999995}]},{"osd":2,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.012},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.58599999999999997}]},{"osd":3,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.91200000000000003}]},{"osd":4,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.87}]},{"osd":5,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.80700000000000005}]},{"osd":6,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":1.171},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.89000000000000001}]},{"osd":7,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.0860000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:31:41.935 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:31:42.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.044+0000 7ff88bccc700 1 Processor -- start 2024-05-08T23:31:42.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.044+0000 7ff88bccc700 1 -- start start 2024-05-08T23:31:42.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff88bccc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8841408f0 0x7ff884140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff88bccc700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8840cdeb0 con 0x7ff8840cec30 2024-05-08T23:31:42.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff88bccc700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff884141210 con 0x7ff88406aa70 2024-05-08T23:31:42.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff88bccc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840cc7a0 con 0x7ff8841408f0 2024-05-08T23:31:42.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff889267700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff8840cec30 0x7ff88406a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58232/0 (socket says 172.21.15.81:58232) 2024-05-08T23:31:42.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff889267700 1 -- 172.21.15.81:0/1012318269 learned_addr learned my addr 172.21.15.81:0/1012318269 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:42.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff889a68700 1 --2- 172.21.15.81:0/1012318269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8841408f0 0x7ff884140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff888a66700 1 -- 172.21.15.81:0/1012318269 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639918458 0 0) 0x7ff8840cdeb0 con 0x7ff8840cec30 2024-05-08T23:31:42.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff888a66700 1 -- 172.21.15.81:0/1012318269 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8700035d0 con 0x7ff8840cec30 2024-05-08T23:31:42.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff888a66700 1 -- 172.21.15.81:0/1012318269 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278592620 0 0) 0x7ff884141210 con 0x7ff88406aa70 2024-05-08T23:31:42.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff889a68700 1 -- 172.21.15.81:0/1012318269 >> v1:172.21.15.117:6789/0 conn(0x7ff88406aa70 legacy=0x7ff8840cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff889a68700 1 -- 172.21.15.81:0/1012318269 >> v1:172.21.15.81:6789/0 conn(0x7ff8840cec30 legacy=0x7ff88406a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff889a68700 1 -- 172.21.15.81:0/1012318269 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8840ce150 con 0x7ff8841408f0 2024-05-08T23:31:42.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff889a68700 1 --2- 172.21.15.81:0/1012318269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8841408f0 0x7ff884140cd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff880009880 tx=0x7ff880009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=520130365004ec6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:42.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff888a66700 1 -- 172.21.15.81:0/1012318269 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880011830 con 0x7ff8841408f0 2024-05-08T23:31:42.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff888a66700 1 -- 172.21.15.81:0/1012318269 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff880011990 con 0x7ff8841408f0 2024-05-08T23:31:42.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff888a66700 1 -- 172.21.15.81:0/1012318269 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880011d20 con 0x7ff8841408f0 2024-05-08T23:31:42.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff88bccc700 1 -- 172.21.15.81:0/1012318269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8841408f0 msgr2=0x7ff884140cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff88bccc700 1 --2- 172.21.15.81:0/1012318269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8841408f0 0x7ff884140cd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff880009880 tx=0x7ff880009b90 comp rx=0 tx=0).stop 2024-05-08T23:31:42.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff88bccc700 1 -- 172.21.15.81:0/1012318269 shutdown_connections 2024-05-08T23:31:42.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff88bccc700 1 --2- 172.21.15.81:0/1012318269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8841408f0 0x7ff884140cd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff88bccc700 1 -- 172.21.15.81:0/1012318269 >> 172.21.15.81:0/1012318269 conn(0x7ff8840cf650 msgr2=0x7ff8840cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:42.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff88bccc700 1 -- 172.21.15.81:0/1012318269 shutdown_connections 2024-05-08T23:31:42.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.048+0000 7ff88bccc700 1 -- 172.21.15.81:0/1012318269 wait complete. 2024-05-08T23:31:42.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff88bccc700 1 Processor -- start 2024-05-08T23:31:42.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff88bccc700 1 -- start start 2024-05-08T23:31:42.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff88bccc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8840cec30 0x7ff884158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff88bccc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8841408f0 0x7ff88406ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff88bccc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff88406ddb0 0x7ff884143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff88bccc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8841424b0 con 0x7ff8840cec30 2024-05-08T23:31:42.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff88bccc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8841418c0 con 0x7ff88406ddb0 2024-05-08T23:31:42.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff88bccc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff884141b10 con 0x7ff8841408f0 2024-05-08T23:31:42.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff889a68700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8840cec30 0x7ff884158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff889a68700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8840cec30 0x7ff884158850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60570/0 (socket says 172.21.15.81:60570) 2024-05-08T23:31:42.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff889a68700 1 -- 172.21.15.81:0/2696402152 learned_addr learned my addr 172.21.15.81:0/2696402152 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:42.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff88a269700 1 --2- 172.21.15.81:0/2696402152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff88406ddb0 0x7ff884143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff889267700 1 --2- 172.21.15.81:0/2696402152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8841408f0 0x7ff88406ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff889a68700 1 -- 172.21.15.81:0/2696402152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8841408f0 msgr2=0x7ff88406ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff889a68700 1 --2- 172.21.15.81:0/2696402152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8841408f0 0x7ff88406ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff889a68700 1 -- 172.21.15.81:0/2696402152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff88406ddb0 msgr2=0x7ff884143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff889a68700 1 --2- 172.21.15.81:0/2696402152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff88406ddb0 0x7ff884143eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.052+0000 7ff889a68700 1 -- 172.21.15.81:0/2696402152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8800094b0 con 0x7ff8840cec30 2024-05-08T23:31:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.056+0000 7ff889a68700 1 --2- 172.21.15.81:0/2696402152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8840cec30 0x7ff884158850 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff880013040 tx=0x7ff880004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.056+0000 7ff87a7fc700 1 -- 172.21.15.81:0/2696402152 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880019950 con 0x7ff8840cec30 2024-05-08T23:31:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.056+0000 7ff87a7fc700 1 -- 172.21.15.81:0/2696402152 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88001e070 con 0x7ff8840cec30 2024-05-08T23:31:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.056+0000 7ff87a7fc700 1 -- 172.21.15.81:0/2696402152 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880019cf0 con 0x7ff8840cec30 2024-05-08T23:31:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.056+0000 7ff88bccc700 1 -- 172.21.15.81:0/2696402152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff884141df0 con 0x7ff8840cec30 2024-05-08T23:31:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.056+0000 7ff88bccc700 1 -- 172.21.15.81:0/2696402152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff884158d90 con 0x7ff8840cec30 2024-05-08T23:31:42.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.056+0000 7ff88bccc700 1 -- 172.21.15.81:0/2696402152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8840672a0 con 0x7ff8840cec30 2024-05-08T23:31:42.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.056+0000 7ff87a7fc700 1 -- 172.21.15.81:0/2696402152 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff88002d450 con 0x7ff8840cec30 2024-05-08T23:31:42.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.056+0000 7ff87a7fc700 1 --2- 172.21.15.81:0/2696402152 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff868066730 0x7ff868068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.056+0000 7ff87a7fc700 1 -- 172.21.15.81:0/2696402152 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff880084fc0 con 0x7ff8840cec30 2024-05-08T23:31:42.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.056+0000 7ff889267700 1 --2- 172.21.15.81:0/2696402152 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff868066730 0x7ff868068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.060+0000 7ff889267700 1 --2- 172.21.15.81:0/2696402152 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff868066730 0x7ff868068bf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff8840cc0a0 tx=0x7ff874009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:42.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.064+0000 7ff87a7fc700 1 -- 172.21.15.81:0/2696402152 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff8800b6cf0 con 0x7ff8840cec30 2024-05-08T23:31:42.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.232+0000 7ff88bccc700 1 -- 172.21.15.81:0/2696402152 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff884141c80 con 0x7ff868066730 2024-05-08T23:31:42.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.248+0000 7ff87a7fc700 1 -- 172.21.15.81:0/2696402152 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288005 (secure 0 0 0) 0x7ff884141c80 con 0x7ff868066730 2024-05-08T23:31:42.254 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:42.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.252+0000 7ff88bccc700 1 -- 172.21.15.81:0/2696402152 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff868066730 msgr2=0x7ff868068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.252+0000 7ff88bccc700 1 --2- 172.21.15.81:0/2696402152 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff868066730 0x7ff868068bf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff8840cc0a0 tx=0x7ff874009040 comp rx=0 tx=0).stop 2024-05-08T23:31:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.252+0000 7ff88bccc700 1 -- 172.21.15.81:0/2696402152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8840cec30 msgr2=0x7ff884158850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.252+0000 7ff88bccc700 1 --2- 172.21.15.81:0/2696402152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8840cec30 0x7ff884158850 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff880013040 tx=0x7ff880004770 comp rx=0 tx=0).stop 2024-05-08T23:31:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.252+0000 7ff88bccc700 1 -- 172.21.15.81:0/2696402152 shutdown_connections 2024-05-08T23:31:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.252+0000 7ff88bccc700 1 --2- 172.21.15.81:0/2696402152 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff868066730 0x7ff868068bf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.252+0000 7ff88bccc700 1 --2- 172.21.15.81:0/2696402152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff88406ddb0 0x7ff884143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.252+0000 7ff88bccc700 1 --2- 172.21.15.81:0/2696402152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8841408f0 0x7ff88406ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.252+0000 7ff88bccc700 1 --2- 172.21.15.81:0/2696402152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8840cec30 0x7ff884158850 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.252+0000 7ff88bccc700 1 -- 172.21.15.81:0/2696402152 >> 172.21.15.81:0/2696402152 conn(0x7ff8840cf650 msgr2=0x7ff8840b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.252+0000 7ff88bccc700 1 -- 172.21.15.81:0/2696402152 shutdown_connections 2024-05-08T23:31:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.252+0000 7ff88bccc700 1 -- 172.21.15.81:0/2696402152 wait complete. 2024-05-08T23:31:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:31:42.268 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-05-08T23:31:41.345079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220512,"kb_used_data":4336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754208,"statfs":{"total":773094113280,"available":772868308992,"internally_reserved":0,"allocated":4440064,"data_stored":2435348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.841901"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971251+0000","last_change":"2024-05-08T23:31:37.917280+0000","last_active":"2024-05-08T23:31:38.971251+0000","last_peered":"2024-05-08T23:31:38.971251+0000","last_clean":"2024-05-08T23:31:38.971251+0000","last_became_active":"2024-05-08T23:31:37.917030+0000","last_became_peered":"2024-05-08T23:31:37.917030+0000","last_unstale":"2024-05-08T23:31:38.971251+0000","last_undegraded":"2024-05-08T23:31:38.971251+0000","last_fullsized":"2024-05-08T23:31:38.971251+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:52.226149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970776+0000","last_change":"2024-05-08T23:31:37.924768+0000","last_active":"2024-05-08T23:31:38.970776+0000","last_peered":"2024-05-08T23:31:38.970776+0000","last_clean":"2024-05-08T23:31:38.970776+0000","last_became_active":"2024-05-08T23:31:37.924526+0000","last_became_peered":"2024-05-08T23:31:37.924526+0000","last_unstale":"2024-05-08T23:31:38.970776+0000","last_undegraded":"2024-05-08T23:31:38.970776+0000","last_fullsized":"2024-05-08T23:31:38.970776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:57.215271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969190+0000","last_change":"2024-05-08T23:31:37.932086+0000","last_active":"2024-05-08T23:31:38.969190+0000","last_peered":"2024-05-08T23:31:38.969190+0000","last_clean":"2024-05-08T23:31:38.969190+0000","last_became_active":"2024-05-08T23:31:37.931426+0000","last_became_peered":"2024-05-08T23:31:37.931426+0000","last_unstale":"2024-05-08T23:31:38.969190+0000","last_undegraded":"2024-05-08T23:31:38.969190+0000","last_fullsized":"2024-05-08T23:31:38.969190+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:18.676110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968039+0000","last_change":"2024-05-08T23:31:37.923671+0000","last_active":"2024-05-08T23:31:38.968039+0000","last_peered":"2024-05-08T23:31:38.968039+0000","last_clean":"2024-05-08T23:31:38.968039+0000","last_became_active":"2024-05-08T23:31:37.922905+0000","last_became_peered":"2024-05-08T23:31:37.922905+0000","last_unstale":"2024-05-08T23:31:38.968039+0000","last_undegraded":"2024-05-08T23:31:38.968039+0000","last_fullsized":"2024-05-08T23:31:38.968039+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:30:52.429721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.904935+0000","last_change":"2024-05-08T23:31:37.922675+0000","last_active":"2024-05-08T23:31:38.904935+0000","last_peered":"2024-05-08T23:31:38.904935+0000","last_clean":"2024-05-08T23:31:38.904935+0000","last_became_active":"2024-05-08T23:31:37.922431+0000","last_became_peered":"2024-05-08T23:31:37.922431+0000","last_unstale":"2024-05-08T23:31:38.904935+0000","last_undegraded":"2024-05-08T23:31:38.904935+0000","last_fullsized":"2024-05-08T23:31:38.904935+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:25.434258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969900+0000","last_change":"2024-05-08T23:31:37.917619+0000","last_active":"2024-05-08T23:31:38.969900+0000","last_peered":"2024-05-08T23:31:38.969900+0000","last_clean":"2024-05-08T23:31:38.969900+0000","last_became_active":"2024-05-08T23:31:37.917303+0000","last_became_peered":"2024-05-08T23:31:37.917303+0000","last_unstale":"2024-05-08T23:31:38.969900+0000","last_undegraded":"2024-05-08T23:31:38.969900+0000","last_fullsized":"2024-05-08T23:31:38.969900+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:06.725435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906357+0000","last_change":"2024-05-08T23:31:37.924580+0000","last_active":"2024-05-08T23:31:38.906357+0000","last_peered":"2024-05-08T23:31:38.906357+0000","last_clean":"2024-05-08T23:31:38.906357+0000","last_became_active":"2024-05-08T23:31:37.924326+0000","last_became_peered":"2024-05-08T23:31:37.924326+0000","last_unstale":"2024-05-08T23:31:38.906357+0000","last_undegraded":"2024-05-08T23:31:38.906357+0000","last_fullsized":"2024-05-08T23:31:38.906357+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:14.919240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906169+0000","last_change":"2024-05-08T23:31:37.919200+0000","last_active":"2024-05-08T23:31:38.906169+0000","last_peered":"2024-05-08T23:31:38.906169+0000","last_clean":"2024-05-08T23:31:38.906169+0000","last_became_active":"2024-05-08T23:31:37.918871+0000","last_became_peered":"2024-05-08T23:31:37.918871+0000","last_unstale":"2024-05-08T23:31:38.906169+0000","last_undegraded":"2024-05-08T23:31:38.906169+0000","last_fullsized":"2024-05-08T23:31:38.906169+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:31:53.198457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.972230+0000","last_change":"2024-05-08T23:31:37.925863+0000","last_active":"2024-05-08T23:31:38.972230+0000","last_peered":"2024-05-08T23:31:38.972230+0000","last_clean":"2024-05-08T23:31:38.972230+0000","last_became_active":"2024-05-08T23:31:37.925297+0000","last_became_peered":"2024-05-08T23:31:37.925297+0000","last_unstale":"2024-05-08T23:31:38.972230+0000","last_undegraded":"2024-05-08T23:31:38.972230+0000","last_fullsized":"2024-05-08T23:31:38.972230+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:26.477816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968176+0000","last_change":"2024-05-08T23:31:37.923544+0000","last_active":"2024-05-08T23:31:38.968176+0000","last_peered":"2024-05-08T23:31:38.968176+0000","last_clean":"2024-05-08T23:31:38.968176+0000","last_became_active":"2024-05-08T23:31:37.923174+0000","last_became_peered":"2024-05-08T23:31:37.923174+0000","last_unstale":"2024-05-08T23:31:38.968176+0000","last_undegraded":"2024-05-08T23:31:38.968176+0000","last_fullsized":"2024-05-08T23:31:38.968176+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:00.231440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906830+0000","last_change":"2024-05-08T23:31:37.923737+0000","last_active":"2024-05-08T23:31:38.906830+0000","last_peered":"2024-05-08T23:31:38.906830+0000","last_clean":"2024-05-08T23:31:38.906830+0000","last_became_active":"2024-05-08T23:31:37.923445+0000","last_became_peered":"2024-05-08T23:31:37.923445+0000","last_unstale":"2024-05-08T23:31:38.906830+0000","last_undegraded":"2024-05-08T23:31:38.906830+0000","last_fullsized":"2024-05-08T23:31:38.906830+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:01.173647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971070+0000","last_change":"2024-05-08T23:31:37.922985+0000","last_active":"2024-05-08T23:31:38.971070+0000","last_peered":"2024-05-08T23:31:38.971070+0000","last_clean":"2024-05-08T23:31:38.971070+0000","last_became_active":"2024-05-08T23:31:37.922703+0000","last_became_peered":"2024-05-08T23:31:37.922703+0000","last_unstale":"2024-05-08T23:31:38.971070+0000","last_undegraded":"2024-05-08T23:31:38.971070+0000","last_fullsized":"2024-05-08T23:31:38.971070+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:20:43.227330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905064+0000","last_change":"2024-05-08T23:31:37.931800+0000","last_active":"2024-05-08T23:31:38.905064+0000","last_peered":"2024-05-08T23:31:38.905064+0000","last_clean":"2024-05-08T23:31:38.905064+0000","last_became_active":"2024-05-08T23:31:37.931449+0000","last_became_peered":"2024-05-08T23:31:37.931449+0000","last_unstale":"2024-05-08T23:31:38.905064+0000","last_undegraded":"2024-05-08T23:31:38.905064+0000","last_fullsized":"2024-05-08T23:31:38.905064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:02.052324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970055+0000","last_change":"2024-05-08T23:31:37.917578+0000","last_active":"2024-05-08T23:31:38.970055+0000","last_peered":"2024-05-08T23:31:38.970055+0000","last_clean":"2024-05-08T23:31:38.970055+0000","last_became_active":"2024-05-08T23:31:37.917301+0000","last_became_peered":"2024-05-08T23:31:37.917301+0000","last_unstale":"2024-05-08T23:31:38.970055+0000","last_undegraded":"2024-05-08T23:31:38.970055+0000","last_fullsized":"2024-05-08T23:31:38.970055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:26.623286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968944+0000","last_change":"2024-05-08T23:31:37.922951+0000","last_active":"2024-05-08T23:31:38.968944+0000","last_peered":"2024-05-08T23:31:38.968944+0000","last_clean":"2024-05-08T23:31:38.968944+0000","last_became_active":"2024-05-08T23:31:37.922681+0000","last_became_peered":"2024-05-08T23:31:37.922681+0000","last_unstale":"2024-05-08T23:31:38.968944+0000","last_undegraded":"2024-05-08T23:31:38.968944+0000","last_fullsized":"2024-05-08T23:31:38.968944+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:26.623286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906218+0000","last_change":"2024-05-08T23:31:37.923756+0000","last_active":"2024-05-08T23:31:38.906218+0000","last_peered":"2024-05-08T23:31:38.906218+0000","last_clean":"2024-05-08T23:31:38.906218+0000","last_became_active":"2024-05-08T23:31:37.923586+0000","last_became_peered":"2024-05-08T23:31:37.923586+0000","last_unstale":"2024-05-08T23:31:38.906218+0000","last_undegraded":"2024-05-08T23:31:38.906218+0000","last_fullsized":"2024-05-08T23:31:38.906218+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:05.927456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.907246+0000","last_change":"2024-05-08T23:31:37.919928+0000","last_active":"2024-05-08T23:31:38.907246+0000","last_peered":"2024-05-08T23:31:38.907246+0000","last_clean":"2024-05-08T23:31:38.907246+0000","last_became_active":"2024-05-08T23:31:37.919638+0000","last_became_peered":"2024-05-08T23:31:37.919638+0000","last_unstale":"2024-05-08T23:31:38.907246+0000","last_undegraded":"2024-05-08T23:31:38.907246+0000","last_fullsized":"2024-05-08T23:31:38.907246+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:35.727462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970842+0000","last_change":"2024-05-08T23:31:37.920467+0000","last_active":"2024-05-08T23:31:38.970842+0000","last_peered":"2024-05-08T23:31:38.970842+0000","last_clean":"2024-05-08T23:31:38.970842+0000","last_became_active":"2024-05-08T23:31:37.920270+0000","last_became_peered":"2024-05-08T23:31:37.920270+0000","last_unstale":"2024-05-08T23:31:38.970842+0000","last_undegraded":"2024-05-08T23:31:38.970842+0000","last_fullsized":"2024-05-08T23:31:38.970842+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:57.561916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970454+0000","last_change":"2024-05-08T23:31:37.923846+0000","last_active":"2024-05-08T23:31:38.970454+0000","last_peered":"2024-05-08T23:31:38.970454+0000","last_clean":"2024-05-08T23:31:38.970454+0000","last_became_active":"2024-05-08T23:31:37.923555+0000","last_became_peered":"2024-05-08T23:31:37.923555+0000","last_unstale":"2024-05-08T23:31:38.970454+0000","last_undegraded":"2024-05-08T23:31:38.970454+0000","last_fullsized":"2024-05-08T23:31:38.970454+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:42.138710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905598+0000","last_change":"2024-05-08T23:31:37.917032+0000","last_active":"2024-05-08T23:31:38.905598+0000","last_peered":"2024-05-08T23:31:38.905598+0000","last_clean":"2024-05-08T23:31:38.905598+0000","last_became_active":"2024-05-08T23:31:37.916751+0000","last_became_peered":"2024-05-08T23:31:37.916751+0000","last_unstale":"2024-05-08T23:31:38.905598+0000","last_undegraded":"2024-05-08T23:31:38.905598+0000","last_fullsized":"2024-05-08T23:31:38.905598+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:39.604074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970071+0000","last_change":"2024-05-08T23:31:37.928252+0000","last_active":"2024-05-08T23:31:38.970071+0000","last_peered":"2024-05-08T23:31:38.970071+0000","last_clean":"2024-05-08T23:31:38.970071+0000","last_became_active":"2024-05-08T23:31:37.925818+0000","last_became_peered":"2024-05-08T23:31:37.925818+0000","last_unstale":"2024-05-08T23:31:38.970071+0000","last_undegraded":"2024-05-08T23:31:38.970071+0000","last_fullsized":"2024-05-08T23:31:38.970071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:52.226149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969619+0000","last_change":"2024-05-08T23:31:37.924930+0000","last_active":"2024-05-08T23:31:38.969619+0000","last_peered":"2024-05-08T23:31:38.969619+0000","last_clean":"2024-05-08T23:31:38.969619+0000","last_became_active":"2024-05-08T23:31:37.924672+0000","last_became_peered":"2024-05-08T23:31:37.924672+0000","last_unstale":"2024-05-08T23:31:38.969619+0000","last_undegraded":"2024-05-08T23:31:38.969619+0000","last_fullsized":"2024-05-08T23:31:38.969619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:49.865105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905058+0000","last_change":"2024-05-08T23:31:37.922277+0000","last_active":"2024-05-08T23:31:38.905058+0000","last_peered":"2024-05-08T23:31:38.905058+0000","last_clean":"2024-05-08T23:31:38.905058+0000","last_became_active":"2024-05-08T23:31:37.921985+0000","last_became_peered":"2024-05-08T23:31:37.921985+0000","last_unstale":"2024-05-08T23:31:38.905058+0000","last_undegraded":"2024-05-08T23:31:38.905058+0000","last_fullsized":"2024-05-08T23:31:38.905058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:32.867982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968841+0000","last_change":"2024-05-08T23:31:37.920044+0000","last_active":"2024-05-08T23:31:38.968841+0000","last_peered":"2024-05-08T23:31:38.968841+0000","last_clean":"2024-05-08T23:31:38.968841+0000","last_became_active":"2024-05-08T23:31:37.919598+0000","last_became_peered":"2024-05-08T23:31:37.919598+0000","last_unstale":"2024-05-08T23:31:38.968841+0000","last_undegraded":"2024-05-08T23:31:38.968841+0000","last_fullsized":"2024-05-08T23:31:38.968841+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:04.278422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968270+0000","last_change":"2024-05-08T23:31:37.931317+0000","last_active":"2024-05-08T23:31:38.968270+0000","last_peered":"2024-05-08T23:31:38.968270+0000","last_clean":"2024-05-08T23:31:38.968270+0000","last_became_active":"2024-05-08T23:31:37.931059+0000","last_became_peered":"2024-05-08T23:31:37.931059+0000","last_unstale":"2024-05-08T23:31:38.968270+0000","last_undegraded":"2024-05-08T23:31:38.968270+0000","last_fullsized":"2024-05-08T23:31:38.968270+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:19.037204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970868+0000","last_change":"2024-05-08T23:31:37.930968+0000","last_active":"2024-05-08T23:31:38.970868+0000","last_peered":"2024-05-08T23:31:38.970868+0000","last_clean":"2024-05-08T23:31:38.970868+0000","last_became_active":"2024-05-08T23:31:37.930671+0000","last_became_peered":"2024-05-08T23:31:37.930671+0000","last_unstale":"2024-05-08T23:31:38.970868+0000","last_undegraded":"2024-05-08T23:31:38.970868+0000","last_fullsized":"2024-05-08T23:31:38.970868+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:31:18.676110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968760+0000","last_change":"2024-05-08T23:31:37.930499+0000","last_active":"2024-05-08T23:31:38.968760+0000","last_peered":"2024-05-08T23:31:38.968760+0000","last_clean":"2024-05-08T23:31:38.968760+0000","last_became_active":"2024-05-08T23:31:37.930212+0000","last_became_peered":"2024-05-08T23:31:37.930212+0000","last_unstale":"2024-05-08T23:31:38.968760+0000","last_undegraded":"2024-05-08T23:31:38.968760+0000","last_fullsized":"2024-05-08T23:31:38.968760+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:00.197636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971198+0000","last_change":"2024-05-08T23:31:37.918834+0000","last_active":"2024-05-08T23:31:38.971198+0000","last_peered":"2024-05-08T23:31:38.971198+0000","last_clean":"2024-05-08T23:31:38.971198+0000","last_became_active":"2024-05-08T23:31:37.918578+0000","last_became_peered":"2024-05-08T23:31:37.918578+0000","last_unstale":"2024-05-08T23:31:38.971198+0000","last_undegraded":"2024-05-08T23:31:38.971198+0000","last_fullsized":"2024-05-08T23:31:38.971198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:19.037204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970273+0000","last_change":"2024-05-08T23:31:37.925892+0000","last_active":"2024-05-08T23:31:38.970273+0000","last_peered":"2024-05-08T23:31:38.970273+0000","last_clean":"2024-05-08T23:31:38.970273+0000","last_became_active":"2024-05-08T23:31:37.925620+0000","last_became_peered":"2024-05-08T23:31:37.925620+0000","last_unstale":"2024-05-08T23:31:38.970273+0000","last_undegraded":"2024-05-08T23:31:38.970273+0000","last_fullsized":"2024-05-08T23:31:38.970273+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:16.487914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906325+0000","last_change":"2024-05-08T23:31:37.919954+0000","last_active":"2024-05-08T23:31:38.906325+0000","last_peered":"2024-05-08T23:31:38.906325+0000","last_clean":"2024-05-08T23:31:38.906325+0000","last_became_active":"2024-05-08T23:31:37.919693+0000","last_became_peered":"2024-05-08T23:31:37.919693+0000","last_unstale":"2024-05-08T23:31:38.906325+0000","last_undegraded":"2024-05-08T23:31:38.906325+0000","last_fullsized":"2024-05-08T23:31:38.906325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:17.662962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970054+0000","last_change":"2024-05-08T23:31:37.918774+0000","last_active":"2024-05-08T23:31:38.970054+0000","last_peered":"2024-05-08T23:31:38.970054+0000","last_clean":"2024-05-08T23:31:38.970054+0000","last_became_active":"2024-05-08T23:31:37.918490+0000","last_became_peered":"2024-05-08T23:31:37.918490+0000","last_unstale":"2024-05-08T23:31:38.970054+0000","last_undegraded":"2024-05-08T23:31:38.970054+0000","last_fullsized":"2024-05-08T23:31:38.970054+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:37.500132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906044+0000","last_change":"2024-05-08T23:31:37.922333+0000","last_active":"2024-05-08T23:31:38.906044+0000","last_peered":"2024-05-08T23:31:38.906044+0000","last_clean":"2024-05-08T23:31:38.906044+0000","last_became_active":"2024-05-08T23:31:37.922073+0000","last_became_peered":"2024-05-08T23:31:37.922073+0000","last_unstale":"2024-05-08T23:31:38.906044+0000","last_undegraded":"2024-05-08T23:31:38.906044+0000","last_fullsized":"2024-05-08T23:31:38.906044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:04.613680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905981+0000","last_change":"2024-05-08T23:31:37.911527+0000","last_active":"2024-05-08T23:31:38.905981+0000","last_peered":"2024-05-08T23:31:38.905981+0000","last_clean":"2024-05-08T23:31:38.905981+0000","last_became_active":"2024-05-08T23:31:37.910910+0000","last_became_peered":"2024-05-08T23:31:37.910910+0000","last_unstale":"2024-05-08T23:31:38.905981+0000","last_undegraded":"2024-05-08T23:31:38.905981+0000","last_fullsized":"2024-05-08T23:31:38.905981+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:04:01.173647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971503+0000","last_change":"2024-05-08T23:31:37.531421+0000","last_active":"2024-05-08T23:31:38.971503+0000","last_peered":"2024-05-08T23:31:38.971503+0000","last_clean":"2024-05-08T23:31:38.971503+0000","last_became_active":"2024-05-08T23:31:37.530860+0000","last_became_peered":"2024-05-08T23:31:37.530860+0000","last_unstale":"2024-05-08T23:31:38.971503+0000","last_undegraded":"2024-05-08T23:31:38.971503+0000","last_fullsized":"2024-05-08T23:31:38.971503+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:44.074366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971317+0000","last_change":"2024-05-08T23:31:37.922943+0000","last_active":"2024-05-08T23:31:38.971317+0000","last_peered":"2024-05-08T23:31:38.971317+0000","last_clean":"2024-05-08T23:31:38.971317+0000","last_became_active":"2024-05-08T23:31:37.922709+0000","last_became_peered":"2024-05-08T23:31:37.922709+0000","last_unstale":"2024-05-08T23:31:38.971317+0000","last_undegraded":"2024-05-08T23:31:38.971317+0000","last_fullsized":"2024-05-08T23:31:38.971317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:26.477816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905929+0000","last_change":"2024-05-08T23:31:37.534937+0000","last_active":"2024-05-08T23:31:38.905929+0000","last_peered":"2024-05-08T23:31:38.905929+0000","last_clean":"2024-05-08T23:31:38.905929+0000","last_became_active":"2024-05-08T23:31:37.534671+0000","last_became_peered":"2024-05-08T23:31:37.534671+0000","last_unstale":"2024-05-08T23:31:38.905929+0000","last_undegraded":"2024-05-08T23:31:38.905929+0000","last_fullsized":"2024-05-08T23:31:38.905929+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:54:55.864126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971681+0000","last_change":"2024-05-08T23:31:37.925434+0000","last_active":"2024-05-08T23:31:38.971681+0000","last_peered":"2024-05-08T23:31:38.971681+0000","last_clean":"2024-05-08T23:31:38.971681+0000","last_became_active":"2024-05-08T23:31:37.924926+0000","last_became_peered":"2024-05-08T23:31:37.924926+0000","last_unstale":"2024-05-08T23:31:38.971681+0000","last_undegraded":"2024-05-08T23:31:38.971681+0000","last_fullsized":"2024-05-08T23:31:38.971681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:00.202616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970515+0000","last_change":"2024-05-08T23:31:37.535742+0000","last_active":"2024-05-08T23:31:38.970515+0000","last_peered":"2024-05-08T23:31:38.970515+0000","last_clean":"2024-05-08T23:31:38.970515+0000","last_became_active":"2024-05-08T23:31:37.535458+0000","last_became_peered":"2024-05-08T23:31:37.535458+0000","last_unstale":"2024-05-08T23:31:38.970515+0000","last_undegraded":"2024-05-08T23:31:38.970515+0000","last_fullsized":"2024-05-08T23:31:38.970515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:36.487313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968792+0000","last_change":"2024-05-08T23:31:37.931191+0000","last_active":"2024-05-08T23:31:38.968792+0000","last_peered":"2024-05-08T23:31:38.968792+0000","last_clean":"2024-05-08T23:31:38.968792+0000","last_became_active":"2024-05-08T23:31:37.930894+0000","last_became_peered":"2024-05-08T23:31:37.930894+0000","last_unstale":"2024-05-08T23:31:38.968792+0000","last_undegraded":"2024-05-08T23:31:38.968792+0000","last_fullsized":"2024-05-08T23:31:38.968792+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:03.410673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905914+0000","last_change":"2024-05-08T23:31:37.534246+0000","last_active":"2024-05-08T23:31:38.905914+0000","last_peered":"2024-05-08T23:31:38.905914+0000","last_clean":"2024-05-08T23:31:38.905914+0000","last_became_active":"2024-05-08T23:31:37.533972+0000","last_became_peered":"2024-05-08T23:31:37.533972+0000","last_unstale":"2024-05-08T23:31:38.905914+0000","last_undegraded":"2024-05-08T23:31:38.905914+0000","last_fullsized":"2024-05-08T23:31:38.905914+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:52.452931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906531+0000","last_change":"2024-05-08T23:31:37.921572+0000","last_active":"2024-05-08T23:31:38.906531+0000","last_peered":"2024-05-08T23:31:38.906531+0000","last_clean":"2024-05-08T23:31:38.906531+0000","last_became_active":"2024-05-08T23:31:37.921278+0000","last_became_peered":"2024-05-08T23:31:37.921278+0000","last_unstale":"2024-05-08T23:31:38.906531+0000","last_undegraded":"2024-05-08T23:31:38.906531+0000","last_fullsized":"2024-05-08T23:31:38.906531+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:08.519579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970464+0000","last_change":"2024-05-08T23:31:37.535485+0000","last_active":"2024-05-08T23:31:38.970464+0000","last_peered":"2024-05-08T23:31:38.970464+0000","last_clean":"2024-05-08T23:31:38.970464+0000","last_became_active":"2024-05-08T23:31:37.535085+0000","last_became_peered":"2024-05-08T23:31:37.535085+0000","last_unstale":"2024-05-08T23:31:38.970464+0000","last_undegraded":"2024-05-08T23:31:38.970464+0000","last_fullsized":"2024-05-08T23:31:38.970464+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:18:59.508639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968976+0000","last_change":"2024-05-08T23:31:37.919545+0000","last_active":"2024-05-08T23:31:38.968976+0000","last_peered":"2024-05-08T23:31:38.968976+0000","last_clean":"2024-05-08T23:31:38.968976+0000","last_became_active":"2024-05-08T23:31:37.919192+0000","last_became_peered":"2024-05-08T23:31:37.919192+0000","last_unstale":"2024-05-08T23:31:38.968976+0000","last_undegraded":"2024-05-08T23:31:38.968976+0000","last_fullsized":"2024-05-08T23:31:38.968976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:45:06.725435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969826+0000","last_change":"2024-05-08T23:31:37.533465+0000","last_active":"2024-05-08T23:31:38.969826+0000","last_peered":"2024-05-08T23:31:38.969826+0000","last_clean":"2024-05-08T23:31:38.969826+0000","last_became_active":"2024-05-08T23:31:37.533044+0000","last_became_peered":"2024-05-08T23:31:37.533044+0000","last_unstale":"2024-05-08T23:31:38.969826+0000","last_undegraded":"2024-05-08T23:31:38.969826+0000","last_fullsized":"2024-05-08T23:31:38.969826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:55:52.452931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969619+0000","last_change":"2024-05-08T23:31:37.925578+0000","last_active":"2024-05-08T23:31:38.969619+0000","last_peered":"2024-05-08T23:31:38.969619+0000","last_clean":"2024-05-08T23:31:38.969619+0000","last_became_active":"2024-05-08T23:31:37.922849+0000","last_became_peered":"2024-05-08T23:31:37.922849+0000","last_unstale":"2024-05-08T23:31:38.969619+0000","last_undegraded":"2024-05-08T23:31:38.969619+0000","last_fullsized":"2024-05-08T23:31:38.969619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:37.653737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969968+0000","last_change":"2024-05-08T23:31:37.531809+0000","last_active":"2024-05-08T23:31:38.969968+0000","last_peered":"2024-05-08T23:31:38.969968+0000","last_clean":"2024-05-08T23:31:38.969968+0000","last_became_active":"2024-05-08T23:31:37.531270+0000","last_became_peered":"2024-05-08T23:31:37.531270+0000","last_unstale":"2024-05-08T23:31:38.969968+0000","last_undegraded":"2024-05-08T23:31:38.969968+0000","last_fullsized":"2024-05-08T23:31:38.969968+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:36.640918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.967465+0000","last_change":"2024-05-08T23:31:37.919828+0000","last_active":"2024-05-08T23:31:38.967465+0000","last_peered":"2024-05-08T23:31:38.967465+0000","last_clean":"2024-05-08T23:31:38.967465+0000","last_became_active":"2024-05-08T23:31:37.919231+0000","last_became_peered":"2024-05-08T23:31:37.919231+0000","last_unstale":"2024-05-08T23:31:38.967465+0000","last_undegraded":"2024-05-08T23:31:38.967465+0000","last_fullsized":"2024-05-08T23:31:38.967465+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:42.138710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969819+0000","last_change":"2024-05-08T23:31:37.529809+0000","last_active":"2024-05-08T23:31:38.969819+0000","last_peered":"2024-05-08T23:31:38.969819+0000","last_clean":"2024-05-08T23:31:38.969819+0000","last_became_active":"2024-05-08T23:31:37.529488+0000","last_became_peered":"2024-05-08T23:31:37.529488+0000","last_unstale":"2024-05-08T23:31:38.969819+0000","last_undegraded":"2024-05-08T23:31:38.969819+0000","last_fullsized":"2024-05-08T23:31:38.969819+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:51.911491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971915+0000","last_change":"2024-05-08T23:31:37.920946+0000","last_active":"2024-05-08T23:31:38.971915+0000","last_peered":"2024-05-08T23:31:38.971915+0000","last_clean":"2024-05-08T23:31:38.971915+0000","last_became_active":"2024-05-08T23:31:37.920695+0000","last_became_peered":"2024-05-08T23:31:37.920695+0000","last_unstale":"2024-05-08T23:31:38.971915+0000","last_undegraded":"2024-05-08T23:31:38.971915+0000","last_fullsized":"2024-05-08T23:31:38.971915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:41:34.627276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971040+0000","last_change":"2024-05-08T23:31:37.536970+0000","last_active":"2024-05-08T23:31:38.971040+0000","last_peered":"2024-05-08T23:31:38.971040+0000","last_clean":"2024-05-08T23:31:38.971040+0000","last_became_active":"2024-05-08T23:31:37.536282+0000","last_became_peered":"2024-05-08T23:31:37.536282+0000","last_unstale":"2024-05-08T23:31:38.971040+0000","last_undegraded":"2024-05-08T23:31:38.971040+0000","last_fullsized":"2024-05-08T23:31:38.971040+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:19:51.213330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970659+0000","last_change":"2024-05-08T23:31:37.923712+0000","last_active":"2024-05-08T23:31:38.970659+0000","last_peered":"2024-05-08T23:31:38.970659+0000","last_clean":"2024-05-08T23:31:38.970659+0000","last_became_active":"2024-05-08T23:31:37.923414+0000","last_became_peered":"2024-05-08T23:31:37.923414+0000","last_unstale":"2024-05-08T23:31:38.970659+0000","last_undegraded":"2024-05-08T23:31:38.970659+0000","last_fullsized":"2024-05-08T23:31:38.970659+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:00.197636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906020+0000","last_change":"2024-05-08T23:31:37.535444+0000","last_active":"2024-05-08T23:31:38.906020+0000","last_peered":"2024-05-08T23:31:38.906020+0000","last_clean":"2024-05-08T23:31:38.906020+0000","last_became_active":"2024-05-08T23:31:37.534891+0000","last_became_peered":"2024-05-08T23:31:37.534891+0000","last_unstale":"2024-05-08T23:31:38.906020+0000","last_undegraded":"2024-05-08T23:31:38.906020+0000","last_fullsized":"2024-05-08T23:31:38.906020+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:50.257330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970618+0000","last_change":"2024-05-08T23:31:37.930846+0000","last_active":"2024-05-08T23:31:38.970618+0000","last_peered":"2024-05-08T23:31:38.970618+0000","last_clean":"2024-05-08T23:31:38.970618+0000","last_became_active":"2024-05-08T23:31:37.930592+0000","last_became_peered":"2024-05-08T23:31:37.930592+0000","last_unstale":"2024-05-08T23:31:38.970618+0000","last_undegraded":"2024-05-08T23:31:38.970618+0000","last_fullsized":"2024-05-08T23:31:38.970618+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:04.278422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971355+0000","last_change":"2024-05-08T23:31:37.535224+0000","last_active":"2024-05-08T23:31:38.971355+0000","last_peered":"2024-05-08T23:31:38.971355+0000","last_clean":"2024-05-08T23:31:38.971355+0000","last_became_active":"2024-05-08T23:31:37.534954+0000","last_became_peered":"2024-05-08T23:31:37.534954+0000","last_unstale":"2024-05-08T23:31:38.971355+0000","last_undegraded":"2024-05-08T23:31:38.971355+0000","last_fullsized":"2024-05-08T23:31:38.971355+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:32.645713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968962+0000","last_change":"2024-05-08T23:31:37.932300+0000","last_active":"2024-05-08T23:31:38.968962+0000","last_peered":"2024-05-08T23:31:38.968962+0000","last_clean":"2024-05-08T23:31:38.968962+0000","last_became_active":"2024-05-08T23:31:37.931805+0000","last_became_peered":"2024-05-08T23:31:37.931805+0000","last_unstale":"2024-05-08T23:31:38.968962+0000","last_undegraded":"2024-05-08T23:31:38.968962+0000","last_fullsized":"2024-05-08T23:31:38.968962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:32.524305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970639+0000","last_change":"2024-05-08T23:31:37.534967+0000","last_active":"2024-05-08T23:31:38.970639+0000","last_peered":"2024-05-08T23:31:38.970639+0000","last_clean":"2024-05-08T23:31:38.970639+0000","last_became_active":"2024-05-08T23:31:37.534713+0000","last_became_peered":"2024-05-08T23:31:37.534713+0000","last_unstale":"2024-05-08T23:31:38.970639+0000","last_undegraded":"2024-05-08T23:31:38.970639+0000","last_fullsized":"2024-05-08T23:31:38.970639+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:24.421439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.967617+0000","last_change":"2024-05-08T23:31:37.918810+0000","last_active":"2024-05-08T23:31:38.967617+0000","last_peered":"2024-05-08T23:31:38.967617+0000","last_clean":"2024-05-08T23:31:38.967617+0000","last_became_active":"2024-05-08T23:31:37.918530+0000","last_became_peered":"2024-05-08T23:31:37.918530+0000","last_unstale":"2024-05-08T23:31:38.967617+0000","last_undegraded":"2024-05-08T23:31:38.967617+0000","last_fullsized":"2024-05-08T23:31:38.967617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:40.390338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970663+0000","last_change":"2024-05-08T23:31:37.532553+0000","last_active":"2024-05-08T23:31:38.970663+0000","last_peered":"2024-05-08T23:31:38.970663+0000","last_clean":"2024-05-08T23:31:38.970663+0000","last_became_active":"2024-05-08T23:31:37.532258+0000","last_became_peered":"2024-05-08T23:31:37.532258+0000","last_unstale":"2024-05-08T23:31:38.970663+0000","last_undegraded":"2024-05-08T23:31:38.970663+0000","last_fullsized":"2024-05-08T23:31:38.970663+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:47:36.522089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970357+0000","last_change":"2024-05-08T23:31:37.925005+0000","last_active":"2024-05-08T23:31:38.970357+0000","last_peered":"2024-05-08T23:31:38.970357+0000","last_clean":"2024-05-08T23:31:38.970357+0000","last_became_active":"2024-05-08T23:31:37.924719+0000","last_became_peered":"2024-05-08T23:31:37.924719+0000","last_unstale":"2024-05-08T23:31:38.970357+0000","last_undegraded":"2024-05-08T23:31:38.970357+0000","last_fullsized":"2024-05-08T23:31:38.970357+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:00.231440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.967552+0000","last_change":"2024-05-08T23:31:37.530877+0000","last_active":"2024-05-08T23:31:38.967552+0000","last_peered":"2024-05-08T23:31:38.967552+0000","last_clean":"2024-05-08T23:31:38.967552+0000","last_became_active":"2024-05-08T23:31:37.530610+0000","last_became_peered":"2024-05-08T23:31:37.530610+0000","last_unstale":"2024-05-08T23:31:38.967552+0000","last_undegraded":"2024-05-08T23:31:38.967552+0000","last_fullsized":"2024-05-08T23:31:38.967552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:44.207694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906136+0000","last_change":"2024-05-08T23:31:37.922173+0000","last_active":"2024-05-08T23:31:38.906136+0000","last_peered":"2024-05-08T23:31:38.906136+0000","last_clean":"2024-05-08T23:31:38.906136+0000","last_became_active":"2024-05-08T23:31:37.921861+0000","last_became_peered":"2024-05-08T23:31:37.921861+0000","last_unstale":"2024-05-08T23:31:38.906136+0000","last_undegraded":"2024-05-08T23:31:38.906136+0000","last_fullsized":"2024-05-08T23:31:38.906136+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:06.391226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969796+0000","last_change":"2024-05-08T23:31:37.536371+0000","last_active":"2024-05-08T23:31:38.969796+0000","last_peered":"2024-05-08T23:31:38.969796+0000","last_clean":"2024-05-08T23:31:38.969796+0000","last_became_active":"2024-05-08T23:31:37.536116+0000","last_became_peered":"2024-05-08T23:31:37.536116+0000","last_unstale":"2024-05-08T23:31:38.969796+0000","last_undegraded":"2024-05-08T23:31:38.969796+0000","last_fullsized":"2024-05-08T23:31:38.969796+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:34.004562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970306+0000","last_change":"2024-05-08T23:31:37.926647+0000","last_active":"2024-05-08T23:31:38.970306+0000","last_peered":"2024-05-08T23:31:38.970306+0000","last_clean":"2024-05-08T23:31:38.970306+0000","last_became_active":"2024-05-08T23:31:37.926370+0000","last_became_peered":"2024-05-08T23:31:37.926370+0000","last_unstale":"2024-05-08T23:31:38.970306+0000","last_undegraded":"2024-05-08T23:31:38.970306+0000","last_fullsized":"2024-05-08T23:31:38.970306+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:25.434258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970011+0000","last_change":"2024-05-08T23:31:37.535568+0000","last_active":"2024-05-08T23:31:38.970011+0000","last_peered":"2024-05-08T23:31:38.970011+0000","last_clean":"2024-05-08T23:31:38.970011+0000","last_became_active":"2024-05-08T23:31:37.535188+0000","last_became_peered":"2024-05-08T23:31:37.535188+0000","last_unstale":"2024-05-08T23:31:38.970011+0000","last_undegraded":"2024-05-08T23:31:38.970011+0000","last_fullsized":"2024-05-08T23:31:38.970011+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:38.591255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971827+0000","last_change":"2024-05-08T23:31:37.917241+0000","last_active":"2024-05-08T23:31:38.971827+0000","last_peered":"2024-05-08T23:31:38.971827+0000","last_clean":"2024-05-08T23:31:38.971827+0000","last_became_active":"2024-05-08T23:31:37.916973+0000","last_became_peered":"2024-05-08T23:31:37.916973+0000","last_unstale":"2024-05-08T23:31:38.971827+0000","last_undegraded":"2024-05-08T23:31:38.971827+0000","last_fullsized":"2024-05-08T23:31:38.971827+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:35.017381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968425+0000","last_change":"2024-05-08T23:31:37.532481+0000","last_active":"2024-05-08T23:31:38.968425+0000","last_peered":"2024-05-08T23:31:38.968425+0000","last_clean":"2024-05-08T23:31:38.968425+0000","last_became_active":"2024-05-08T23:31:37.532070+0000","last_became_peered":"2024-05-08T23:31:37.532070+0000","last_unstale":"2024-05-08T23:31:38.968425+0000","last_undegraded":"2024-05-08T23:31:38.968425+0000","last_fullsized":"2024-05-08T23:31:38.968425+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:24.421439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.972223+0000","last_change":"2024-05-08T23:31:37.920666+0000","last_active":"2024-05-08T23:31:38.972223+0000","last_peered":"2024-05-08T23:31:38.972223+0000","last_clean":"2024-05-08T23:31:38.972223+0000","last_became_active":"2024-05-08T23:31:37.920382+0000","last_became_peered":"2024-05-08T23:31:37.920382+0000","last_unstale":"2024-05-08T23:31:38.972223+0000","last_undegraded":"2024-05-08T23:31:38.972223+0000","last_fullsized":"2024-05-08T23:31:38.972223+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:11.997851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971932+0000","last_change":"2024-05-08T23:31:37.529281+0000","last_active":"2024-05-08T23:31:38.971932+0000","last_peered":"2024-05-08T23:31:38.971932+0000","last_clean":"2024-05-08T23:31:38.971932+0000","last_became_active":"2024-05-08T23:31:37.529015+0000","last_became_peered":"2024-05-08T23:31:37.529015+0000","last_unstale":"2024-05-08T23:31:38.971932+0000","last_undegraded":"2024-05-08T23:31:38.971932+0000","last_fullsized":"2024-05-08T23:31:38.971932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:44.074366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968065+0000","last_change":"2024-05-08T23:31:37.922947+0000","last_active":"2024-05-08T23:31:38.968065+0000","last_peered":"2024-05-08T23:31:38.968065+0000","last_clean":"2024-05-08T23:31:38.968065+0000","last_became_active":"2024-05-08T23:31:37.921506+0000","last_became_peered":"2024-05-08T23:31:37.921506+0000","last_unstale":"2024-05-08T23:31:38.968065+0000","last_undegraded":"2024-05-08T23:31:38.968065+0000","last_fullsized":"2024-05-08T23:31:38.968065+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:54.854483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971998+0000","last_change":"2024-05-08T23:31:37.533680+0000","last_active":"2024-05-08T23:31:38.971998+0000","last_peered":"2024-05-08T23:31:38.971998+0000","last_clean":"2024-05-08T23:31:38.971998+0000","last_became_active":"2024-05-08T23:31:37.532483+0000","last_became_peered":"2024-05-08T23:31:37.532483+0000","last_unstale":"2024-05-08T23:31:38.971998+0000","last_undegraded":"2024-05-08T23:31:38.971998+0000","last_fullsized":"2024-05-08T23:31:38.971998+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:52.114816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.904901+0000","last_change":"2024-05-08T23:31:37.931134+0000","last_active":"2024-05-08T23:31:38.904901+0000","last_peered":"2024-05-08T23:31:38.904901+0000","last_clean":"2024-05-08T23:31:38.904901+0000","last_became_active":"2024-05-08T23:31:37.930888+0000","last_became_peered":"2024-05-08T23:31:37.930888+0000","last_unstale":"2024-05-08T23:31:38.904901+0000","last_undegraded":"2024-05-08T23:31:38.904901+0000","last_fullsized":"2024-05-08T23:31:38.904901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:37.500132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.971928+0000","last_change":"2024-05-08T23:31:37.533142+0000","last_active":"2024-05-08T23:31:38.971928+0000","last_peered":"2024-05-08T23:31:38.971928+0000","last_clean":"2024-05-08T23:31:38.971928+0000","last_became_active":"2024-05-08T23:31:37.532854+0000","last_became_peered":"2024-05-08T23:31:37.532854+0000","last_unstale":"2024-05-08T23:31:38.971928+0000","last_undegraded":"2024-05-08T23:31:38.971928+0000","last_fullsized":"2024-05-08T23:31:38.971928+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:26.190141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905089+0000","last_change":"2024-05-08T23:31:37.926706+0000","last_active":"2024-05-08T23:31:38.905089+0000","last_peered":"2024-05-08T23:31:38.905089+0000","last_clean":"2024-05-08T23:31:38.905089+0000","last_became_active":"2024-05-08T23:31:37.926416+0000","last_became_peered":"2024-05-08T23:31:37.926416+0000","last_unstale":"2024-05-08T23:31:38.905089+0000","last_undegraded":"2024-05-08T23:31:38.905089+0000","last_fullsized":"2024-05-08T23:31:38.905089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:45.220513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905891+0000","last_change":"2024-05-08T23:31:37.526613+0000","last_active":"2024-05-08T23:31:38.905891+0000","last_peered":"2024-05-08T23:31:38.905891+0000","last_clean":"2024-05-08T23:31:38.905891+0000","last_became_active":"2024-05-08T23:31:37.526325+0000","last_became_peered":"2024-05-08T23:31:37.526325+0000","last_unstale":"2024-05-08T23:31:38.905891+0000","last_undegraded":"2024-05-08T23:31:38.905891+0000","last_fullsized":"2024-05-08T23:31:38.905891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:49.435279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905204+0000","last_change":"2024-05-08T23:31:37.925330+0000","last_active":"2024-05-08T23:31:38.905204+0000","last_peered":"2024-05-08T23:31:38.905204+0000","last_clean":"2024-05-08T23:31:38.905204+0000","last_became_active":"2024-05-08T23:31:37.925073+0000","last_became_peered":"2024-05-08T23:31:37.925073+0000","last_unstale":"2024-05-08T23:31:38.905204+0000","last_undegraded":"2024-05-08T23:31:38.905204+0000","last_fullsized":"2024-05-08T23:31:38.905204+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:35.054111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970757+0000","last_change":"2024-05-08T23:31:37.534454+0000","last_active":"2024-05-08T23:31:38.970757+0000","last_peered":"2024-05-08T23:31:38.970757+0000","last_clean":"2024-05-08T23:31:38.970757+0000","last_became_active":"2024-05-08T23:31:37.534174+0000","last_became_peered":"2024-05-08T23:31:37.534174+0000","last_unstale":"2024-05-08T23:31:38.970757+0000","last_undegraded":"2024-05-08T23:31:38.970757+0000","last_fullsized":"2024-05-08T23:31:38.970757+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:42.982299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.904937+0000","last_change":"2024-05-08T23:31:37.925161+0000","last_active":"2024-05-08T23:31:38.904937+0000","last_peered":"2024-05-08T23:31:38.904937+0000","last_clean":"2024-05-08T23:31:38.904937+0000","last_became_active":"2024-05-08T23:31:37.924905+0000","last_became_peered":"2024-05-08T23:31:37.924905+0000","last_unstale":"2024-05-08T23:31:38.904937+0000","last_undegraded":"2024-05-08T23:31:38.904937+0000","last_fullsized":"2024-05-08T23:31:38.904937+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:29.553929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970412+0000","last_change":"2024-05-08T23:31:37.532697+0000","last_active":"2024-05-08T23:31:38.970412+0000","last_peered":"2024-05-08T23:31:38.970412+0000","last_clean":"2024-05-08T23:31:38.970412+0000","last_became_active":"2024-05-08T23:31:37.532428+0000","last_became_peered":"2024-05-08T23:31:37.532428+0000","last_unstale":"2024-05-08T23:31:38.970412+0000","last_undegraded":"2024-05-08T23:31:38.970412+0000","last_fullsized":"2024-05-08T23:31:38.970412+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:41.831458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906206+0000","last_change":"2024-05-08T23:31:37.924622+0000","last_active":"2024-05-08T23:31:38.906206+0000","last_peered":"2024-05-08T23:31:38.906206+0000","last_clean":"2024-05-08T23:31:38.906206+0000","last_became_active":"2024-05-08T23:31:37.924377+0000","last_became_peered":"2024-05-08T23:31:37.924377+0000","last_unstale":"2024-05-08T23:31:38.906206+0000","last_undegraded":"2024-05-08T23:31:38.906206+0000","last_fullsized":"2024-05-08T23:31:38.906206+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:13.549112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969066+0000","last_change":"2024-05-08T23:31:37.537714+0000","last_active":"2024-05-08T23:31:38.969066+0000","last_peered":"2024-05-08T23:31:38.969066+0000","last_clean":"2024-05-08T23:31:38.969066+0000","last_became_active":"2024-05-08T23:31:37.537359+0000","last_became_peered":"2024-05-08T23:31:37.537359+0000","last_unstale":"2024-05-08T23:31:38.969066+0000","last_undegraded":"2024-05-08T23:31:38.969066+0000","last_fullsized":"2024-05-08T23:31:38.969066+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:48.852286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905146+0000","last_change":"2024-05-08T23:31:37.924407+0000","last_active":"2024-05-08T23:31:38.905146+0000","last_peered":"2024-05-08T23:31:38.905146+0000","last_clean":"2024-05-08T23:31:38.905146+0000","last_became_active":"2024-05-08T23:31:37.924132+0000","last_became_peered":"2024-05-08T23:31:37.924132+0000","last_unstale":"2024-05-08T23:31:38.905146+0000","last_undegraded":"2024-05-08T23:31:38.905146+0000","last_fullsized":"2024-05-08T23:31:38.905146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:35:12.810159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968530+0000","last_change":"2024-05-08T23:31:37.530961+0000","last_active":"2024-05-08T23:31:38.968530+0000","last_peered":"2024-05-08T23:31:38.968530+0000","last_clean":"2024-05-08T23:31:38.968530+0000","last_became_active":"2024-05-08T23:31:37.530656+0000","last_became_peered":"2024-05-08T23:31:37.530656+0000","last_unstale":"2024-05-08T23:31:38.968530+0000","last_undegraded":"2024-05-08T23:31:38.968530+0000","last_fullsized":"2024-05-08T23:31:38.968530+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:25.464997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.972342+0000","last_change":"2024-05-08T23:31:37.921784+0000","last_active":"2024-05-08T23:31:38.972342+0000","last_peered":"2024-05-08T23:31:38.972342+0000","last_clean":"2024-05-08T23:31:38.972342+0000","last_became_active":"2024-05-08T23:31:37.921385+0000","last_became_peered":"2024-05-08T23:31:37.921385+0000","last_unstale":"2024-05-08T23:31:38.972342+0000","last_undegraded":"2024-05-08T23:31:38.972342+0000","last_fullsized":"2024-05-08T23:31:38.972342+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:57.215271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906071+0000","last_change":"2024-05-08T23:31:37.529247+0000","last_active":"2024-05-08T23:31:38.906071+0000","last_peered":"2024-05-08T23:31:38.906071+0000","last_clean":"2024-05-08T23:31:38.906071+0000","last_became_active":"2024-05-08T23:31:37.528917+0000","last_became_peered":"2024-05-08T23:31:37.528917+0000","last_unstale":"2024-05-08T23:31:38.906071+0000","last_undegraded":"2024-05-08T23:31:38.906071+0000","last_fullsized":"2024-05-08T23:31:38.906071+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:30.372846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905833+0000","last_change":"2024-05-08T23:31:37.922597+0000","last_active":"2024-05-08T23:31:38.905833+0000","last_peered":"2024-05-08T23:31:38.905833+0000","last_clean":"2024-05-08T23:31:38.905833+0000","last_became_active":"2024-05-08T23:31:37.922316+0000","last_became_peered":"2024-05-08T23:31:37.922316+0000","last_unstale":"2024-05-08T23:31:38.905833+0000","last_undegraded":"2024-05-08T23:31:38.905833+0000","last_fullsized":"2024-05-08T23:31:38.905833+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:58.676646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.968237+0000","last_change":"2024-05-08T23:31:37.532231+0000","last_active":"2024-05-08T23:31:38.968237+0000","last_peered":"2024-05-08T23:31:38.968237+0000","last_clean":"2024-05-08T23:31:38.968237+0000","last_became_active":"2024-05-08T23:31:37.531950+0000","last_became_peered":"2024-05-08T23:31:37.531950+0000","last_unstale":"2024-05-08T23:31:38.968237+0000","last_undegraded":"2024-05-08T23:31:38.968237+0000","last_fullsized":"2024-05-08T23:31:38.968237+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:36.487313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970664+0000","last_change":"2024-05-08T23:31:37.925173+0000","last_active":"2024-05-08T23:31:38.970664+0000","last_peered":"2024-05-08T23:31:38.970664+0000","last_clean":"2024-05-08T23:31:38.970664+0000","last_became_active":"2024-05-08T23:31:37.924920+0000","last_became_peered":"2024-05-08T23:31:37.924920+0000","last_unstale":"2024-05-08T23:31:38.970664+0000","last_undegraded":"2024-05-08T23:31:38.970664+0000","last_fullsized":"2024-05-08T23:31:38.970664+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:33:35.017381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905954+0000","last_change":"2024-05-08T23:31:37.527972+0000","last_active":"2024-05-08T23:31:38.905954+0000","last_peered":"2024-05-08T23:31:38.905954+0000","last_clean":"2024-05-08T23:31:38.905954+0000","last_became_active":"2024-05-08T23:31:37.527641+0000","last_became_peered":"2024-05-08T23:31:37.527641+0000","last_unstale":"2024-05-08T23:31:38.905954+0000","last_undegraded":"2024-05-08T23:31:38.905954+0000","last_fullsized":"2024-05-08T23:31:38.905954+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:24.719550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.905821+0000","last_change":"2024-05-08T23:31:37.915918+0000","last_active":"2024-05-08T23:31:38.905821+0000","last_peered":"2024-05-08T23:31:38.905821+0000","last_clean":"2024-05-08T23:31:38.905821+0000","last_became_active":"2024-05-08T23:31:37.915687+0000","last_became_peered":"2024-05-08T23:31:37.915687+0000","last_unstale":"2024-05-08T23:31:38.905821+0000","last_undegraded":"2024-05-08T23:31:38.905821+0000","last_fullsized":"2024-05-08T23:31:38.905821+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:28.759487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.967722+0000","last_change":"2024-05-08T23:31:37.529483+0000","last_active":"2024-05-08T23:31:38.967722+0000","last_peered":"2024-05-08T23:31:38.967722+0000","last_clean":"2024-05-08T23:31:38.967722+0000","last_became_active":"2024-05-08T23:31:37.528480+0000","last_became_peered":"2024-05-08T23:31:37.528480+0000","last_unstale":"2024-05-08T23:31:38.967722+0000","last_undegraded":"2024-05-08T23:31:38.967722+0000","last_fullsized":"2024-05-08T23:31:38.967722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:38.591255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969726+0000","last_change":"2024-05-08T23:31:37.921085+0000","last_active":"2024-05-08T23:31:38.969726+0000","last_peered":"2024-05-08T23:31:38.969726+0000","last_clean":"2024-05-08T23:31:38.969726+0000","last_became_active":"2024-05-08T23:31:37.920833+0000","last_became_peered":"2024-05-08T23:31:37.920833+0000","last_unstale":"2024-05-08T23:31:38.969726+0000","last_undegraded":"2024-05-08T23:31:38.969726+0000","last_fullsized":"2024-05-08T23:31:38.969726+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:19.037204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970580+0000","last_change":"2024-05-08T23:31:37.534032+0000","last_active":"2024-05-08T23:31:38.970580+0000","last_peered":"2024-05-08T23:31:38.970580+0000","last_clean":"2024-05-08T23:31:38.970580+0000","last_became_active":"2024-05-08T23:31:37.532189+0000","last_became_peered":"2024-05-08T23:31:37.532189+0000","last_unstale":"2024-05-08T23:31:38.970580+0000","last_undegraded":"2024-05-08T23:31:38.970580+0000","last_fullsized":"2024-05-08T23:31:38.970580+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:26.197274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970223+0000","last_change":"2024-05-08T23:31:37.924638+0000","last_active":"2024-05-08T23:31:38.970223+0000","last_peered":"2024-05-08T23:31:38.970223+0000","last_clean":"2024-05-08T23:31:38.970223+0000","last_became_active":"2024-05-08T23:31:37.922881+0000","last_became_peered":"2024-05-08T23:31:37.922881+0000","last_unstale":"2024-05-08T23:31:38.970223+0000","last_undegraded":"2024-05-08T23:31:38.970223+0000","last_fullsized":"2024-05-08T23:31:38.970223+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:33.658532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.969845+0000","last_change":"2024-05-08T23:31:37.537638+0000","last_active":"2024-05-08T23:31:38.969845+0000","last_peered":"2024-05-08T23:31:38.969845+0000","last_clean":"2024-05-08T23:31:38.969845+0000","last_became_active":"2024-05-08T23:31:37.537360+0000","last_became_peered":"2024-05-08T23:31:37.537360+0000","last_unstale":"2024-05-08T23:31:38.969845+0000","last_undegraded":"2024-05-08T23:31:38.969845+0000","last_fullsized":"2024-05-08T23:31:38.969845+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:25.464997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.906666+0000","last_change":"2024-05-08T23:31:37.917985+0000","last_active":"2024-05-08T23:31:38.906666+0000","last_peered":"2024-05-08T23:31:38.906666+0000","last_clean":"2024-05-08T23:31:38.906666+0000","last_became_active":"2024-05-08T23:31:37.917678+0000","last_became_peered":"2024-05-08T23:31:37.917678+0000","last_unstale":"2024-05-08T23:31:38.906666+0000","last_undegraded":"2024-05-08T23:31:38.906666+0000","last_fullsized":"2024-05-08T23:31:38.906666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:37.501104+0000","last_clean_scrub_stamp":"2024-05-08T23:31:37.501104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:14.429360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T23:31:38.970885+0000","last_change":"2024-05-08T23:31:37.535656+0000","last_active":"2024-05-08T23:31:38.970885+0000","last_peered":"2024-05-08T23:31:38.970885+0000","last_clean":"2024-05-08T23:31:38.970885+0000","last_became_active":"2024-05-08T23:31:37.535231+0000","last_became_peered":"2024-05-08T23:31:37.535231+0000","last_unstale":"2024-05-08T23:31:38.970885+0000","last_undegraded":"2024-05-08T23:31:38.970885+0000","last_fullsized":"2024-05-08T23:31:38.970885+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:31:36.488285+0000","last_clean_scrub_stamp":"2024-05-08T23:31:36.488285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:36.141659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":175,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-08T23:31:37.902873+0000","last_change":"2024-05-08T23:30:27.226484+0000","last_active":"2024-05-08T23:31:37.902873+0000","last_peered":"2024-05-08T23:31:37.902873+0000","last_clean":"2024-05-08T23:31:37.902873+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:31:37.902873+0000","last_undegraded":"2024-05-08T23:31:37.902873+0000","last_fullsized":"2024-05-08T23:31:37.902873+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_clean_scrub_stamp":"2024-05-08T23:30:26.204118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:14.267287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":158427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.84599999999999997}]},{"osd":1,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.998}]},{"osd":2,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.1479999999999999}]},{"osd":3,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.79500000000000004}]},{"osd":4,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.94699999999999995}]},{"osd":5,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.105},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.89200000000000002}]},{"osd":7,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":1.048},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.018}]}]},{"osd":5,"up_from":15,"seq":64424509460,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":158483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.056},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.5449999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.90300000000000002}]},{"osd":1,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.752}]},{"osd":2,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.242}]},{"osd":3,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.03},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.1659999999999999}]},{"osd":4,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.464,"5min":2.464,"15min":2.464},"last":1.099}]},{"osd":6,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.1419999999999999}]},{"osd":7,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":1.4359999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":157915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:31:25 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.54200000000000004}]},{"osd":1,"last update":"Wed May 8 23:31:25 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.51300000000000001}]},{"osd":2,"last update":"Wed May 8 23:31:25 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed May 8 23:31:25 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.70999999999999996}]},{"osd":5,"last update":"Wed May 8 23:31:25 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.997}]},{"osd":6,"last update":"Wed May 8 23:31:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.77800000000000002}]},{"osd":7,"last update":"Wed May 8 23:31:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.036},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.95399999999999996}]}]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":159397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.48199999999999998}]},{"osd":1,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.56000000000000005}]},{"osd":2,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.99399999999999999}]},{"osd":4,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.86599999999999999}]},{"osd":5,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.0580000000000001}]},{"osd":6,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.90300000000000002}]},{"osd":7,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.92600000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":157881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.747},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.442}]},{"osd":1,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.621}]},{"osd":3,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":1.3420000000000001}]},{"osd":5,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.875}]},{"osd":6,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.0920000000000001}]},{"osd":7,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.056},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.492}]}]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":159397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.73399999999999999}]},{"osd":3,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.4740000000000002,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.85899999999999999}]},{"osd":5,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.03},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.5830000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.95899999999999996}]},{"osd":6,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.88}]},{"osd":7,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.91400000000000003}]}]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":741969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.73599999999999999}]},{"osd":1,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":1.016},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.871}]},{"osd":2,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.89700000000000002}]},{"osd":3,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":1.137},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.82099999999999995}]},{"osd":4,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":1.056},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.92900000000000005}]},{"osd":5,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.76900000000000002}]},{"osd":6,"last update":"Wed May 8 23:31:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.84099999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":741879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.69199999999999995}]},{"osd":2,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.012},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.58599999999999997}]},{"osd":3,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.91200000000000003}]},{"osd":4,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.87}]},{"osd":5,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.80700000000000005}]},{"osd":6,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":1.171},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.89000000000000001}]},{"osd":7,"last update":"Wed May 8 23:31:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.0860000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:31:42.273 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T23:31:42.273 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T23:31:42.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.396+0000 7f43b2b27700 1 Processor -- start 2024-05-08T23:31:42.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.396+0000 7f43b2b27700 1 -- start start 2024-05-08T23:31:42.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.396+0000 7f43b2b27700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f43ac157b30 0x7f43ac15e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.396+0000 7f43b2b27700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ac15a350 con 0x7f43ac0adf30 2024-05-08T23:31:42.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.396+0000 7f43b2b27700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ac15a5b0 con 0x7f43ac0abda0 2024-05-08T23:31:42.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.396+0000 7f43b2b27700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ac15e980 con 0x7f43ac157b30 2024-05-08T23:31:42.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.396+0000 7f43b1b25700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f43ac157b30 0x7f43ac15e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.396+0000 7f43b2326700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f43ac0adf30 0x7f43ac0b2fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58242/0 (socket says 172.21.15.81:58242) 2024-05-08T23:31:42.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.396+0000 7f43b2326700 1 -- 172.21.15.81:0/2678839214 learned_addr learned my addr 172.21.15.81:0/2678839214 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:42.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.396+0000 7f43b0b23700 1 -- 172.21.15.81:0/2678839214 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941031516 0 0) 0x7f43ac15a350 con 0x7f43ac0adf30 2024-05-08T23:31:42.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.396+0000 7f43b0b23700 1 -- 172.21.15.81:0/2678839214 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43940035d0 con 0x7f43ac0adf30 2024-05-08T23:31:42.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.400+0000 7f43b1b25700 1 -- 172.21.15.81:0/2678839214 >> v1:172.21.15.117:6789/0 conn(0x7f43ac0abda0 legacy=0x7f43ac0ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.400+0000 7f43b0b23700 1 -- 172.21.15.81:0/2678839214 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777196342 0 0) 0x7f43ac15a5b0 con 0x7f43ac0abda0 2024-05-08T23:31:42.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.400+0000 7f43b1b25700 1 -- 172.21.15.81:0/2678839214 >> v1:172.21.15.81:6789/0 conn(0x7f43ac0adf30 legacy=0x7f43ac0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.400+0000 7f43b1b25700 1 -- 172.21.15.81:0/2678839214 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac15a890 con 0x7f43ac157b30 2024-05-08T23:31:42.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.400+0000 7f43b1b25700 1 --2- 172.21.15.81:0/2678839214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43ac157b30 0x7f43ac15e350 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009850 tx=0x7f43a0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7a73e244b480f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:42.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.400+0000 7f43b0b23700 1 -- 172.21.15.81:0/2678839214 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0011830 con 0x7f43ac157b30 2024-05-08T23:31:42.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.400+0000 7f43b0b23700 1 -- 172.21.15.81:0/2678839214 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a0011990 con 0x7f43ac157b30 2024-05-08T23:31:42.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.400+0000 7f43b0b23700 1 -- 172.21.15.81:0/2678839214 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0011d40 con 0x7f43ac157b30 2024-05-08T23:31:42.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.400+0000 7f43b2b27700 1 -- 172.21.15.81:0/2678839214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43ac157b30 msgr2=0x7f43ac15e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.400+0000 7f43b2b27700 1 --2- 172.21.15.81:0/2678839214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43ac157b30 0x7f43ac15e350 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009850 tx=0x7f43a0009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:42.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.400+0000 7f43b2b27700 1 -- 172.21.15.81:0/2678839214 shutdown_connections 2024-05-08T23:31:42.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.400+0000 7f43b2b27700 1 --2- 172.21.15.81:0/2678839214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43ac157b30 0x7f43ac15e350 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.400+0000 7f43b2b27700 1 -- 172.21.15.81:0/2678839214 >> 172.21.15.81:0/2678839214 conn(0x7f43ac0b0de0 msgr2=0x7f43ac0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:42.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.400+0000 7f43b2b27700 1 -- 172.21.15.81:0/2678839214 shutdown_connections 2024-05-08T23:31:42.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.400+0000 7f43b2b27700 1 -- 172.21.15.81:0/2678839214 wait complete. 2024-05-08T23:31:42.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2b27700 1 Processor -- start 2024-05-08T23:31:42.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2b27700 1 -- start start 2024-05-08T23:31:42.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2b27700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43ac0abda0 0x7f43ac13d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2b27700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43ac0adf30 0x7f43ac151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2b27700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43ac157b30 0x7f43ac099d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2b27700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac14ebb0 con 0x7f43ac157b30 2024-05-08T23:31:42.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2b27700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac09be40 con 0x7f43ac0adf30 2024-05-08T23:31:42.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2b27700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac09c120 con 0x7f43ac0abda0 2024-05-08T23:31:42.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2326700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43ac157b30 0x7f43ac099d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2326700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43ac157b30 0x7f43ac099d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60596/0 (socket says 172.21.15.81:60596) 2024-05-08T23:31:42.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2326700 1 -- 172.21.15.81:0/1819147025 learned_addr learned my addr 172.21.15.81:0/1819147025 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:42.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b1b25700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43ac0abda0 0x7f43ac13d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b1324700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43ac0adf30 0x7f43ac151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2326700 1 -- 172.21.15.81:0/1819147025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43ac0abda0 msgr2=0x7f43ac13d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2326700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43ac0abda0 0x7f43ac13d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2326700 1 -- 172.21.15.81:0/1819147025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43ac0adf30 msgr2=0x7f43ac151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2326700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43ac0adf30 0x7f43ac151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2326700 1 -- 172.21.15.81:0/1819147025 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a00094b0 con 0x7f43ac157b30 2024-05-08T23:31:42.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b1b25700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43ac0abda0 0x7f43ac13d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:42.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b1324700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43ac0adf30 0x7f43ac151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:42.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2326700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43ac157b30 0x7f43ac099d10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f43a400c4f0 tx=0x7f43a400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:42.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43aa7fc700 1 -- 172.21.15.81:0/1819147025 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4005d90 con 0x7f43ac157b30 2024-05-08T23:31:42.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43aa7fc700 1 -- 172.21.15.81:0/1819147025 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a40070d0 con 0x7f43ac157b30 2024-05-08T23:31:42.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2b27700 1 -- 172.21.15.81:0/1819147025 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ac09c400 con 0x7f43ac157b30 2024-05-08T23:31:42.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43aa7fc700 1 -- 172.21.15.81:0/1819147025 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4007460 con 0x7f43ac157b30 2024-05-08T23:31:42.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.404+0000 7f43b2b27700 1 -- 172.21.15.81:0/1819147025 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ac09ba60 con 0x7f43ac157b30 2024-05-08T23:31:42.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.408+0000 7f43b2b27700 1 -- 172.21.15.81:0/1819147025 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ac09bbf0 con 0x7f43ac157b30 2024-05-08T23:31:42.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.408+0000 7f43aa7fc700 1 -- 172.21.15.81:0/1819147025 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f43a40075c0 con 0x7f43ac157b30 2024-05-08T23:31:42.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.408+0000 7f43aa7fc700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4398066730 0x7f4398068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.408+0000 7f43aa7fc700 1 -- 172.21.15.81:0/1819147025 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43a40836d0 con 0x7f43ac157b30 2024-05-08T23:31:42.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.408+0000 7f43b1b25700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4398066730 0x7f4398068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.408+0000 7f43b1b25700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4398066730 0x7f4398068bf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f43a00090b0 tx=0x7f43a001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:42.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.416+0000 7f43aa7fc700 1 -- 172.21.15.81:0/1819147025 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f43a4050a40 con 0x7f43ac157b30 2024-05-08T23:31:42.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.648+0000 7f43b2b27700 1 -- 172.21.15.81:0/1819147025 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f43ac0684b0 con 0x7f43ac157b30 2024-05-08T23:31:42.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.652+0000 7f43aa7fc700 1 -- 172.21.15.81:0/1819147025 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+0 (secure 0 0 0) 0x7f43a4054060 con 0x7f43ac157b30 2024-05-08T23:31:42.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.656+0000 7f43b2b27700 1 -- 172.21.15.81:0/1819147025 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4398066730 msgr2=0x7f4398068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.656+0000 7f43b2b27700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4398066730 0x7f4398068bf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f43a00090b0 tx=0x7f43a001c040 comp rx=0 tx=0).stop 2024-05-08T23:31:42.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.656+0000 7f43b2b27700 1 -- 172.21.15.81:0/1819147025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43ac157b30 msgr2=0x7f43ac099d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.656+0000 7f43b2b27700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43ac157b30 0x7f43ac099d10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f43a400c4f0 tx=0x7f43a400c8b0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.656+0000 7f43b2b27700 1 -- 172.21.15.81:0/1819147025 shutdown_connections 2024-05-08T23:31:42.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.656+0000 7f43b2b27700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4398066730 0x7f4398068bf0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.656+0000 7f43b2b27700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43ac157b30 0x7f43ac099d10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.656+0000 7f43b2b27700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43ac0adf30 0x7f43ac151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.656+0000 7f43b2b27700 1 --2- 172.21.15.81:0/1819147025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43ac0abda0 0x7f43ac13d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.656+0000 7f43b2b27700 1 -- 172.21.15.81:0/1819147025 >> 172.21.15.81:0/1819147025 conn(0x7f43ac0b0de0 msgr2=0x7f43ac0b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:42.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.656+0000 7f43b2b27700 1 -- 172.21.15.81:0/1819147025 shutdown_connections 2024-05-08T23:31:42.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.656+0000 7f43b2b27700 1 -- 172.21.15.81:0/1819147025 wait complete. 2024-05-08T23:31:42.659 INFO:teuthology.orchestra.run.smithi081.stderr:updated caps for client.0 2024-05-08T23:31:42.671 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:31:42.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.780+0000 7f82cbfff700 1 Processor -- start 2024-05-08T23:31:42.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.780+0000 7f82cbfff700 1 -- start start 2024-05-08T23:31:42.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cbfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f82cc15ef80 0x7f82cc071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cbfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f82cc0cfad0 0x7f82cc06a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cbfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82cc06a8a0 0x7f82cc06d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cbfff700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82cc06ee00 con 0x7f82cc15ef80 2024-05-08T23:31:42.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cbfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82cc0b36e0 con 0x7f82cc0cfad0 2024-05-08T23:31:42.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cbfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc06e810 con 0x7f82cc06a8a0 2024-05-08T23:31:42.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82caffd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f82cc15ef80 0x7f82cc071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82caffd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f82cc15ef80 0x7f82cc071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60608/0 (socket says 172.21.15.81:60608) 2024-05-08T23:31:42.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82caffd700 1 -- 172.21.15.81:0/3988793435 learned_addr learned my addr 172.21.15.81:0/3988793435 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:42.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82ca7fc700 1 --2- 172.21.15.81:0/3988793435 >> v2:172.21.15.117:3300/0 conn(0x7f82cc0cfad0 0x7f82cc06a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cb7fe700 1 --2- 172.21.15.81:0/3988793435 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82cc06a8a0 0x7f82cc06d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82caffd700 1 -- 172.21.15.81:0/3988793435 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82cc06a8a0 msgr2=0x7f82cc06d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:42.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82caffd700 1 --2- 172.21.15.81:0/3988793435 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82cc06a8a0 0x7f82cc06d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82caffd700 1 -- 172.21.15.81:0/3988793435 >> v2:172.21.15.117:3300/0 conn(0x7f82cc0cfad0 msgr2=0x7f82cc06a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:42.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82caffd700 1 --2- 172.21.15.81:0/3988793435 >> v2:172.21.15.117:3300/0 conn(0x7f82cc0cfad0 0x7f82cc06a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82caffd700 1 -- 172.21.15.81:0/3988793435 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc0b2e40 con 0x7f82cc15ef80 2024-05-08T23:31:42.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82caffd700 1 --2- 172.21.15.81:0/3988793435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82cc15ef80 0x7f82cc071930 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0d0d90 tx=0x7f82b4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8060591b6974e207 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:42.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82c9ffb700 1 -- 172.21.15.81:0/3988793435 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4011830 con 0x7f82cc15ef80 2024-05-08T23:31:42.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82c9ffb700 1 -- 172.21.15.81:0/3988793435 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b4011990 con 0x7f82cc15ef80 2024-05-08T23:31:42.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82c9ffb700 1 -- 172.21.15.81:0/3988793435 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4011d20 con 0x7f82cc15ef80 2024-05-08T23:31:42.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cbfff700 1 -- 172.21.15.81:0/3988793435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82cc15ef80 msgr2=0x7f82cc071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cbfff700 1 --2- 172.21.15.81:0/3988793435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82cc15ef80 0x7f82cc071930 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0d0d90 tx=0x7f82b4009bf0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cbfff700 1 -- 172.21.15.81:0/3988793435 shutdown_connections 2024-05-08T23:31:42.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cbfff700 1 --2- 172.21.15.81:0/3988793435 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82cc06a8a0 0x7f82cc06d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cbfff700 1 --2- 172.21.15.81:0/3988793435 >> v2:172.21.15.117:3300/0 conn(0x7f82cc0cfad0 0x7f82cc06a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cbfff700 1 --2- 172.21.15.81:0/3988793435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82cc15ef80 0x7f82cc071930 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cbfff700 1 -- 172.21.15.81:0/3988793435 >> 172.21.15.81:0/3988793435 conn(0x7f82cc06b470 msgr2=0x7f82cc06b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:42.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cbfff700 1 -- 172.21.15.81:0/3988793435 shutdown_connections 2024-05-08T23:31:42.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.784+0000 7f82cbfff700 1 -- 172.21.15.81:0/3988793435 wait complete. 2024-05-08T23:31:42.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.788+0000 7f82cbfff700 1 Processor -- start 2024-05-08T23:31:42.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.788+0000 7f82cbfff700 1 -- start start 2024-05-08T23:31:42.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82cbfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82cc06a8a0 0x7f82cc150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82cbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82cc0cfad0 0x7f82cc1603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82cbfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82cc15ef80 0x7f82cc09a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82cbfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc09d230 con 0x7f82cc0cfad0 2024-05-08T23:31:42.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82cbfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc1507d0 con 0x7f82cc15ef80 2024-05-08T23:31:42.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82cbfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc14dcf0 con 0x7f82cc06a8a0 2024-05-08T23:31:42.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82ca7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82cc0cfad0 0x7f82cc1603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82ca7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82cc0cfad0 0x7f82cc1603c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60620/0 (socket says 172.21.15.81:60620) 2024-05-08T23:31:42.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82ca7fc700 1 -- 172.21.15.81:0/619610234 learned_addr learned my addr 172.21.15.81:0/619610234 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:42.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82cb7fe700 1 --2- 172.21.15.81:0/619610234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82cc15ef80 0x7f82cc09a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82caffd700 1 --2- 172.21.15.81:0/619610234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82cc06a8a0 0x7f82cc150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82ca7fc700 1 -- 172.21.15.81:0/619610234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82cc06a8a0 msgr2=0x7f82cc150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82ca7fc700 1 --2- 172.21.15.81:0/619610234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82cc06a8a0 0x7f82cc150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82ca7fc700 1 -- 172.21.15.81:0/619610234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82cc15ef80 msgr2=0x7f82cc09a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:42.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82ca7fc700 1 --2- 172.21.15.81:0/619610234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82cc15ef80 0x7f82cc09a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:42.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82ca7fc700 1 -- 172.21.15.81:0/619610234 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b4009570 con 0x7f82cc0cfad0 2024-05-08T23:31:42.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82ca7fc700 1 --2- 172.21.15.81:0/619610234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82cc0cfad0 0x7f82cc1603c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00efc0 tx=0x7f82bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:42.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82c37fe700 1 -- 172.21.15.81:0/619610234 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00cd10 con 0x7f82cc0cfad0 2024-05-08T23:31:42.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82c37fe700 1 -- 172.21.15.81:0/619610234 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc00f040 con 0x7f82cc0cfad0 2024-05-08T23:31:42.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82cbfff700 1 -- 172.21.15.81:0/619610234 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82cc14dfd0 con 0x7f82cc0cfad0 2024-05-08T23:31:42.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82c37fe700 1 -- 172.21.15.81:0/619610234 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc0146a0 con 0x7f82cc0cfad0 2024-05-08T23:31:42.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82cbfff700 1 -- 172.21.15.81:0/619610234 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc14e590 con 0x7f82cc0cfad0 2024-05-08T23:31:42.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.792+0000 7f82cbfff700 1 -- 172.21.15.81:0/619610234 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82cc0672a0 con 0x7f82cc0cfad0 2024-05-08T23:31:42.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.796+0000 7f82c37fe700 1 -- 172.21.15.81:0/619610234 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f82bc014800 con 0x7f82cc0cfad0 2024-05-08T23:31:42.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.796+0000 7f82c37fe700 1 --2- 172.21.15.81:0/619610234 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f82ac066730 0x7f82ac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:42.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.796+0000 7f82c37fe700 1 -- 172.21.15.81:0/619610234 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82bc084b60 con 0x7f82cc0cfad0 2024-05-08T23:31:42.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.796+0000 7f82caffd700 1 --2- 172.21.15.81:0/619610234 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f82ac066730 0x7f82ac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:42.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.796+0000 7f82caffd700 1 --2- 172.21.15.81:0/619610234 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f82ac066730 0x7f82ac068bf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f82b4000c00 tx=0x7f82b4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:42.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:42.804+0000 7f82c37fe700 1 -- 172.21.15.81:0/619610234 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f82bc051ed0 con 0x7f82cc0cfad0 2024-05-08T23:31:43.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.008+0000 7f82cbfff700 1 -- 172.21.15.81:0/619610234 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82cc1627f0 con 0x7f82cc0cfad0 2024-05-08T23:31:43.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.012+0000 7f82c37fe700 1 -- 172.21.15.81:0/619610234 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f82bc0554f0 con 0x7f82cc0cfad0 2024-05-08T23:31:43.012 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:43.012 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1243037934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1243037934},{"type":"v1","addr":"172.21.15.81:6835","nonce":1243037934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3124115182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3124115182},{"type":"v1","addr":"172.21.15.81:6837","nonce":3124115182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1423349691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":1423349691},{"type":"v1","addr":"172.21.15.117:6834","nonce":1423349691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:31:37.889298+0000","modified":"2024-05-08T23:31:40.903919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14852":{"gid":14852,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/3091184157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3091184157},{"type":"v1","addr":"172.21.15.117:6835","nonce":3091184157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:31:43.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.012+0000 7f82cbfff700 1 -- 172.21.15.81:0/619610234 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f82ac066730 msgr2=0x7f82ac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.012+0000 7f82cbfff700 1 --2- 172.21.15.81:0/619610234 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f82ac066730 0x7f82ac068bf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f82b4000c00 tx=0x7f82b4008030 comp rx=0 tx=0).stop 2024-05-08T23:31:43.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.012+0000 7f82cbfff700 1 -- 172.21.15.81:0/619610234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82cc0cfad0 msgr2=0x7f82cc1603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.012+0000 7f82cbfff700 1 --2- 172.21.15.81:0/619610234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82cc0cfad0 0x7f82cc1603c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00efc0 tx=0x7f82bc00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.012+0000 7f82cbfff700 1 -- 172.21.15.81:0/619610234 shutdown_connections 2024-05-08T23:31:43.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.012+0000 7f82cbfff700 1 --2- 172.21.15.81:0/619610234 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f82ac066730 0x7f82ac068bf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.012+0000 7f82cbfff700 1 --2- 172.21.15.81:0/619610234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82cc15ef80 0x7f82cc09a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.012+0000 7f82cbfff700 1 --2- 172.21.15.81:0/619610234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82cc0cfad0 0x7f82cc1603c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.012+0000 7f82cbfff700 1 --2- 172.21.15.81:0/619610234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82cc06a8a0 0x7f82cc150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.012+0000 7f82cbfff700 1 -- 172.21.15.81:0/619610234 >> 172.21.15.81:0/619610234 conn(0x7f82cc06b470 msgr2=0x7f82cc163fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:43.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.012+0000 7f82cbfff700 1 -- 172.21.15.81:0/619610234 shutdown_connections 2024-05-08T23:31:43.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.012+0000 7f82cbfff700 1 -- 172.21.15.81:0/619610234 wait complete. 2024-05-08T23:31:43.016 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 14 2024-05-08T23:31:43.429 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:31:37.889298+0000', 'modified': '2024-05-08T23:31:40.903919+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 14852}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14852': {'gid': 14852, 'name': 'd', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6835/3091184157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6832', 'nonce': 3091184157}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 3091184157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:31:43.429 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T23:31:43.429 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T23:31:43.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.552+0000 7f6704b4e700 1 Processor -- start 2024-05-08T23:31:43.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f6704b4e700 1 -- start start 2024-05-08T23:31:43.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f6704b4e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6700160ab0 0x7f67000b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f6704b4e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67000d3e80 con 0x7f67000b4610 2024-05-08T23:31:43.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f6704b4e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67000ce780 con 0x7f67000d00d0 2024-05-08T23:31:43.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f6704b4e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6700160e90 con 0x7f6700160ab0 2024-05-08T23:31:43.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66feffd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f67000b4610 0x7f67000cf9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58256/0 (socket says 172.21.15.81:58256) 2024-05-08T23:31:43.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66feffd700 1 -- 172.21.15.81:0/4026798277 learned_addr learned my addr 172.21.15.81:0/4026798277 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:43.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/4026798277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6700160ab0 0x7f67000b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499866416 0 0) 0x7f67000d3e80 con 0x7f67000b4610 2024-05-08T23:31:43.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66e80035d0 con 0x7f67000b4610 2024-05-08T23:31:43.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145580746 0 0) 0x7f67000ce780 con 0x7f67000d00d0 2024-05-08T23:31:43.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67000d3e80 con 0x7f67000d00d0 2024-05-08T23:31:43.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3198750811 0 0) 0x7f66e80035d0 con 0x7f67000b4610 2024-05-08T23:31:43.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6700160ab0 msgr2=0x7f67000b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:43.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 --2- 172.21.15.81:0/4026798277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6700160ab0 0x7f67000b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 >> v1:172.21.15.117:6789/0 conn(0x7f67000d00d0 legacy=0x7f67000b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67000cea60 con 0x7f67000b4610 2024-05-08T23:31:43.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f66f4001670 con 0x7f67000b4610 2024-05-08T23:31:43.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 >> v1:172.21.15.81:6789/0 conn(0x7f67000b4610 legacy=0x7f67000cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 --2- 172.21.15.81:0/4026798277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6700160ab0 0x7f66e8003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 --2- 172.21.15.81:0/4026798277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66e8004140 0x7f66e8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 --2- 172.21.15.81:0/4026798277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e8004db0 0x7f66e8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6700160e90 con 0x7f66e8004140 2024-05-08T23:31:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e800bb10 con 0x7f66e8004db0 2024-05-08T23:31:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e800bcd0 con 0x7f6700160ab0 2024-05-08T23:31:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66feffd700 1 --2- 172.21.15.81:0/4026798277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66e8004140 0x7f66e8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/4026798277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6700160ab0 0x7f66e8003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fffff700 1 --2- 172.21.15.81:0/4026798277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e8004db0 0x7f66e8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66feffd700 1 -- 172.21.15.81:0/4026798277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6700160ab0 msgr2=0x7f66e8003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66feffd700 1 --2- 172.21.15.81:0/4026798277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6700160ab0 0x7f66e8003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66feffd700 1 -- 172.21.15.81:0/4026798277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e8004db0 msgr2=0x7f66e8005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66feffd700 1 --2- 172.21.15.81:0/4026798277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e8004db0 0x7f66e8005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66feffd700 1 -- 172.21.15.81:0/4026798277 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e80035d0 con 0x7f66e8004140 2024-05-08T23:31:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66feffd700 1 --2- 172.21.15.81:0/4026798277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66e8004140 0x7f66e8004870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f67000b4a30 tx=0x7f66e80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c7f829d3cba499 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f400ccf0 con 0x7f66e8004140 2024-05-08T23:31:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f4001af0 con 0x7f66e8004140 2024-05-08T23:31:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.556+0000 7f66fe7fc700 1 -- 172.21.15.81:0/4026798277 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4005bc0 con 0x7f66e8004140 2024-05-08T23:31:43.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.560+0000 7f6704b4e700 1 -- 172.21.15.81:0/4026798277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66e8004140 msgr2=0x7f66e8004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.560+0000 7f6704b4e700 1 --2- 172.21.15.81:0/4026798277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66e8004140 0x7f66e8004870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f67000b4a30 tx=0x7f66e80058a0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.560+0000 7f66fffff700 1 -- 172.21.15.81:0/4026798277 reap_dead start 2024-05-08T23:31:43.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.560+0000 7f6704b4e700 1 -- 172.21.15.81:0/4026798277 shutdown_connections 2024-05-08T23:31:43.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.560+0000 7f6704b4e700 1 -- 172.21.15.81:0/4026798277 >> 172.21.15.81:0/4026798277 conn(0x7f670009c390 msgr2=0x7f67000d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:43.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.560+0000 7f6704b4e700 1 -- 172.21.15.81:0/4026798277 shutdown_connections 2024-05-08T23:31:43.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.560+0000 7f6704b4e700 1 -- 172.21.15.81:0/4026798277 wait complete. 2024-05-08T23:31:43.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f6704b4e700 1 Processor -- start 2024-05-08T23:31:43.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f6704b4e700 1 -- start start 2024-05-08T23:31:43.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f6704b4e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e8004140 0x7f6700150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f6704b4e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67000a0870 0x7f67000cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f6704b4e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67001f3d60 0x7f67001f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f6704b4e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67001f4720 con 0x7f67001f3d60 2024-05-08T23:31:43.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f6704b4e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67000a0cd0 con 0x7f66e8004140 2024-05-08T23:31:43.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f6704b4e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670014fd70 con 0x7f67000a0870 2024-05-08T23:31:43.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f66fffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67001f3d60 0x7f67001f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f66ff7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e8004140 0x7f6700150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f66fffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67001f3d60 0x7f67001f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60642/0 (socket says 172.21.15.81:60642) 2024-05-08T23:31:43.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f66ff7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e8004140 0x7f6700150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:36800/0 (socket says 172.21.15.81:36800) 2024-05-08T23:31:43.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f66fffff700 1 -- 172.21.15.81:0/2518804681 learned_addr learned my addr 172.21.15.81:0/2518804681 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:43.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f66feffd700 1 --2- 172.21.15.81:0/2518804681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67000a0870 0x7f67000cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f66fffff700 1 -- 172.21.15.81:0/2518804681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67000a0870 msgr2=0x7f67000cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f66fffff700 1 --2- 172.21.15.81:0/2518804681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67000a0870 0x7f67000cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f66fffff700 1 -- 172.21.15.81:0/2518804681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e8004140 msgr2=0x7f6700150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f66fffff700 1 --2- 172.21.15.81:0/2518804681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e8004140 0x7f6700150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f66fffff700 1 -- 172.21.15.81:0/2518804681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67000ccbd0 con 0x7f67001f3d60 2024-05-08T23:31:43.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/2518804681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e8004140 0x7f6700150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:43.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.564+0000 7f66fffff700 1 --2- 172.21.15.81:0/2518804681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67001f3d60 0x7f67001f41e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f67000cf620 tx=0x7f66f800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:43.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.568+0000 7f66e7fff700 1 -- 172.21.15.81:0/2518804681 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f800fed0 con 0x7f67001f3d60 2024-05-08T23:31:43.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.568+0000 7f66e7fff700 1 -- 172.21.15.81:0/2518804681 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f8003c30 con 0x7f67001f3d60 2024-05-08T23:31:43.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.568+0000 7f6704b4e700 1 -- 172.21.15.81:0/2518804681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6700150840 con 0x7f67001f3d60 2024-05-08T23:31:43.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.568+0000 7f66e7fff700 1 -- 172.21.15.81:0/2518804681 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f800d930 con 0x7f67001f3d60 2024-05-08T23:31:43.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.568+0000 7f6704b4e700 1 -- 172.21.15.81:0/2518804681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6700150e00 con 0x7f67001f3d60 2024-05-08T23:31:43.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.568+0000 7f66e7fff700 1 -- 172.21.15.81:0/2518804681 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f66f800da90 con 0x7f67001f3d60 2024-05-08T23:31:43.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.568+0000 7f66e7fff700 1 --2- 172.21.15.81:0/2518804681 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f66ec066730 0x7f66ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.568+0000 7f66e7fff700 1 -- 172.21.15.81:0/2518804681 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66f80842a0 con 0x7f67001f3d60 2024-05-08T23:31:43.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.568+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/2518804681 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f66ec066730 0x7f66ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.568+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/2518804681 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f66ec066730 0x7f66ec068bf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009f80 tx=0x7f66f0009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:43.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 Processor -- start 2024-05-08T23:31:43.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 -- start start 2024-05-08T23:31:43.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 0x1177ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x116ebe0 0x113f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x1178420 0x113f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x109ed80 con 0x1178420 2024-05-08T23:31:43.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x109c510 con 0x116ebe0 2024-05-08T23:31:43.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x1097e00 con 0x1177b00 2024-05-08T23:31:43.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f66fffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x1178420 0x113f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f66fffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x1178420 0x113f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60646/0 (socket says 172.21.15.81:60646) 2024-05-08T23:31:43.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f66fffff700 1 -- 172.21.15.81:0/687954012 learned_addr learned my addr 172.21.15.81:0/687954012 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:43.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/687954012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 0x1177ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f66feffd700 1 --2- 172.21.15.81:0/687954012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x116ebe0 0x113f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f66fffff700 1 -- 172.21.15.81:0/687954012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 msgr2=0x1177ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:43.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f66fffff700 1 --2- 172.21.15.81:0/687954012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 0x1177ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f66fffff700 1 -- 172.21.15.81:0/687954012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x116ebe0 msgr2=0x113f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:43.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f66fffff700 1 --2- 172.21.15.81:0/687954012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x116ebe0 0x113f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f66fffff700 1 -- 172.21.15.81:0/687954012 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xf0a960 con 0x1178420 2024-05-08T23:31:43.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/687954012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 0x1177ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:43.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f66fffff700 1 --2- 172.21.15.81:0/687954012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x1178420 0x113f9b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f66f80827f0 tx=0x7f66f8054cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7224c4ea4f66a89d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:43.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f66e57fa700 1 -- 172.21.15.81:0/687954012 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f805d6c0 con 0x1178420 2024-05-08T23:31:43.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f66e57fa700 1 -- 172.21.15.81:0/687954012 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f8053410 con 0x1178420 2024-05-08T23:31:43.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f66e57fa700 1 -- 172.21.15.81:0/687954012 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f80519e0 con 0x1178420 2024-05-08T23:31:43.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 -- 172.21.15.81:0/687954012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x1178420 msgr2=0x113f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 --2- 172.21.15.81:0/687954012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x1178420 0x113f9b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f66f80827f0 tx=0x7f66f8054cb0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 -- 172.21.15.81:0/687954012 shutdown_connections 2024-05-08T23:31:43.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 --2- 172.21.15.81:0/687954012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x1178420 0x113f9b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 --2- 172.21.15.81:0/687954012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x116ebe0 0x113f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 --2- 172.21.15.81:0/687954012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 0x1177ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 -- 172.21.15.81:0/687954012 >> 172.21.15.81:0/687954012 conn(0xebf4d0 msgr2=0x1159250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:43.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 -- 172.21.15.81:0/687954012 shutdown_connections 2024-05-08T23:31:43.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.572+0000 7f670905f740 1 -- 172.21.15.81:0/687954012 wait complete. 2024-05-08T23:31:43.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f670905f740 1 -- start start 2024-05-08T23:31:43.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f670905f740 1 Processor -- start 2024-05-08T23:31:43.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f670905f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x1131d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f670905f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x1177b00 0x1130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f670905f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1178420 0x117dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f670905f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x1055730 con 0x116ebe0 2024-05-08T23:31:43.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f670905f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x105b5b0 con 0x1177b00 2024-05-08T23:31:43.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f670905f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0xf43aa0 con 0x1178420 2024-05-08T23:31:43.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f670905f740 10 client.0 _ll_register_callbacks cb 0x1091610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:43.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f66ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x1131d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f66ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x1131d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60660/0 (socket says 172.21.15.81:60660) 2024-05-08T23:31:43.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f66ff7fe700 1 -- 172.21.15.81:0/1343916293 learned_addr learned my addr 172.21.15.81:0/1343916293 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:43.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f66fffff700 1 --2- 172.21.15.81:0/1343916293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1178420 0x117dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f66feffd700 1 --2- 172.21.15.81:0/1343916293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x1177b00 0x1130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f66ff7fe700 1 -- 172.21.15.81:0/1343916293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1178420 msgr2=0x117dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:43.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/1343916293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1178420 0x117dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f66ff7fe700 1 -- 172.21.15.81:0/1343916293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x1177b00 msgr2=0x1130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:43.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/1343916293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x1177b00 0x1130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f66ff7fe700 1 -- 172.21.15.81:0/1343916293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6700150e00 con 0x116ebe0 2024-05-08T23:31:43.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/1343916293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x1131d00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f66f0022b10 tx=0x7f66f0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b9efd04178c6d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:43.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.576+0000 7f66de7fc700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-08T23:31:43.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f66de7fc700 1 -- 172.21.15.81:0/1343916293 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0015630 con 0x116ebe0 2024-05-08T23:31:43.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f66de7fc700 1 -- 172.21.15.81:0/1343916293 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66f0015790 con 0x116ebe0 2024-05-08T23:31:43.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 1 -- 172.21.15.81:0/1343916293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xecc1e0 con 0x116ebe0 2024-05-08T23:31:43.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f66de7fc700 1 -- 172.21.15.81:0/1343916293 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66f0015950 con 0x116ebe0 2024-05-08T23:31:43.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 10 client.5391 fetch_fsmap learned FSMap version 14 2024-05-08T23:31:43.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 1 -- 172.21.15.81:0/1343916293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x105e220 con 0x116ebe0 2024-05-08T23:31:43.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f66de7fc700 1 -- 172.21.15.81:0/1343916293 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xecc1e0 con 0x116ebe0 2024-05-08T23:31:43.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f66de7fc700 1 -- 172.21.15.81:0/1343916293 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 14) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f66f0015ab0 con 0x116ebe0 2024-05-08T23:31:43.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 10 client.5391 fetch_fsmap finished waiting for FSMap version 14 2024-05-08T23:31:43.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 10 client.5391 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T23:31:43.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 20 client.5391 populate_metadata read hostname 'smithi081' 2024-05-08T23:31:43.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 1 --2- 172.21.15.81:0/1343916293 >> [v2:172.21.15.117:6832/3091184157,v1:172.21.15.117:6835/3091184157] conn(0x11705f0 0x118b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 4 client.5391 mds_command: new command op to 14852 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:31:43.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 1 -- 172.21.15.81:0/1343916293 --> [v2:172.21.15.117:6832/3091184157,v1:172.21.15.117:6835/3091184157] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xf16460 con 0x11705f0 2024-05-08T23:31:43.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f66feffd700 1 --2- 172.21.15.81:0/1343916293 >> [v2:172.21.15.117:6832/3091184157,v1:172.21.15.117:6835/3091184157] conn(0x11705f0 0x118b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f66de7fc700 10 client.5391 ms_handle_connect on v2:172.21.15.117:6832/3091184157 2024-05-08T23:31:43.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f66feffd700 1 --2- 172.21.15.81:0/1343916293 >> [v2:172.21.15.117:6832/3091184157,v1:172.21.15.117:6835/3091184157] conn(0x11705f0 0x118b4d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92432fa928d1a36c server_cookie=27fb2a1695245a50 in_seq=0 out_seq=0 2024-05-08T23:31:43.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f66de7fc700 1 -- 172.21.15.81:0/1343916293 <== mds.0 v2:172.21.15.117:6832/3091184157 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0xf16460 con 0x11705f0 2024-05-08T23:31:43.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f66de7fc700 10 client.5391 handle_command_reply: tid=0 2024-05-08T23:31:43.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 1 client.5391 shutdown 2024-05-08T23:31:43.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 1 -- 172.21.15.81:0/1343916293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 msgr2=0x1131d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 1 --2- 172.21.15.81:0/1343916293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x1131d00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f66f0022b10 tx=0x7f66f0004770 comp rx=0 tx=0).stop 2024-05-08T23:31:43.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 1 -- 172.21.15.81:0/1343916293 shutdown_connections 2024-05-08T23:31:43.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 1 --2- 172.21.15.81:0/1343916293 >> [v2:172.21.15.117:6832/3091184157,v1:172.21.15.117:6835/3091184157] conn(0x11705f0 0x118b4d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 1 --2- 172.21.15.81:0/1343916293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1178420 0x117dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f66de7fc700 0 client.5391 ms_handle_reset on v2:172.21.15.117:6832/3091184157 2024-05-08T23:31:43.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 1 --2- 172.21.15.81:0/1343916293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x1177b00 0x1130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 1 --2- 172.21.15.81:0/1343916293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x1131d00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.580+0000 7f670905f740 1 -- 172.21.15.81:0/1343916293 >> 172.21.15.81:0/1343916293 conn(0xebf4d0 msgr2=0x1158990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:43.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.584+0000 7f670905f740 1 -- 172.21.15.81:0/1343916293 shutdown_connections 2024-05-08T23:31:43.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.584+0000 7f670905f740 1 -- 172.21.15.81:0/1343916293 wait complete. 2024-05-08T23:31:43.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.584+0000 7f670905f740 20 client.5391 trim_cache size 0 max 16384 2024-05-08T23:31:43.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f670905f740 1 Processor -- start 2024-05-08T23:31:43.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f670905f740 1 -- start start 2024-05-08T23:31:43.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f670905f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x11759a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f670905f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 0x1175ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f670905f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x1178420 0x11868a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f670905f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x1130ea0 con 0x116ebe0 2024-05-08T23:31:43.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f670905f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x10c6ec0 con 0x1178420 2024-05-08T23:31:43.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f670905f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x10624f0 con 0x1177b00 2024-05-08T23:31:43.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f66ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x11759a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f66ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x11759a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60666/0 (socket says 172.21.15.81:60666) 2024-05-08T23:31:43.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f66ff7fe700 1 -- 172.21.15.81:0/1155797070 learned_addr learned my addr 172.21.15.81:0/1155797070 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:43.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f66feffd700 1 --2- 172.21.15.81:0/1155797070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 0x1175ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f66fffff700 1 --2- 172.21.15.81:0/1155797070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x1178420 0x11868a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f66ff7fe700 1 -- 172.21.15.81:0/1155797070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 msgr2=0x1175ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:43.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/1155797070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 0x1175ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f66ff7fe700 1 -- 172.21.15.81:0/1155797070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x1178420 msgr2=0x11868a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:43.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/1155797070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x1178420 0x11868a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f66ff7fe700 1 -- 172.21.15.81:0/1155797070 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1044450 con 0x116ebe0 2024-05-08T23:31:43.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f66feffd700 1 --2- 172.21.15.81:0/1155797070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 0x1175ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:43.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/1155797070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x11759a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f66f0022a90 tx=0x7f66f0024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca620a1eabc19b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:43.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f66e5ffb700 1 -- 172.21.15.81:0/1155797070 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0022230 con 0x116ebe0 2024-05-08T23:31:43.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f66e5ffb700 1 -- 172.21.15.81:0/1155797070 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f0019070 con 0x116ebe0 2024-05-08T23:31:43.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.604+0000 7f66e5ffb700 1 -- 172.21.15.81:0/1155797070 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0015830 con 0x116ebe0 2024-05-08T23:31:43.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.608+0000 7f670905f740 1 -- 172.21.15.81:0/1155797070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 msgr2=0x11759a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.608+0000 7f670905f740 1 --2- 172.21.15.81:0/1155797070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x11759a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f66f0022a90 tx=0x7f66f0024040 comp rx=0 tx=0).stop 2024-05-08T23:31:43.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.608+0000 7f670905f740 1 -- 172.21.15.81:0/1155797070 shutdown_connections 2024-05-08T23:31:43.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.608+0000 7f670905f740 1 --2- 172.21.15.81:0/1155797070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x1178420 0x11868a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.608+0000 7f670905f740 1 --2- 172.21.15.81:0/1155797070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 0x1175ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.608+0000 7f670905f740 1 --2- 172.21.15.81:0/1155797070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x11759a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.608+0000 7f670905f740 1 -- 172.21.15.81:0/1155797070 >> 172.21.15.81:0/1155797070 conn(0xebf4d0 msgr2=0x118fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:43.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.608+0000 7f670905f740 1 -- 172.21.15.81:0/1155797070 shutdown_connections 2024-05-08T23:31:43.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.608+0000 7f670905f740 1 -- 172.21.15.81:0/1155797070 wait complete. 2024-05-08T23:31:43.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.608+0000 7f670905f740 1 -- start start 2024-05-08T23:31:43.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.608+0000 7f670905f740 1 Processor -- start 2024-05-08T23:31:43.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x117da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 0x112f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x1178420 0x112fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x1142840 con 0x116ebe0 2024-05-08T23:31:43.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x1176760 con 0x1178420 2024-05-08T23:31:43.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x117d6e0 con 0x1177b00 2024-05-08T23:31:43.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 10 client.0 _ll_register_callbacks cb 0x7f66f8051610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:43.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x117da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66fffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x1178420 0x112fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66feffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 0x112f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x117da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60682/0 (socket says 172.21.15.81:60682) 2024-05-08T23:31:43.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66ff7fe700 1 -- 172.21.15.81:0/3584101838 learned_addr learned my addr 172.21.15.81:0/3584101838 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:43.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66ff7fe700 1 -- 172.21.15.81:0/3584101838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 msgr2=0x112f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:43.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/3584101838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 0x112f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66ff7fe700 1 -- 172.21.15.81:0/3584101838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x1178420 msgr2=0x112fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:43.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/3584101838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x1178420 0x112fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66ff7fe700 1 -- 172.21.15.81:0/3584101838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e80035d0 con 0x116ebe0 2024-05-08T23:31:43.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66ff7fe700 1 --2- 172.21.15.81:0/3584101838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x117da80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f66f002d450 tx=0x7f66f001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35629404e0767d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:43.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66de7fc700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-08T23:31:43.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66de7fc700 1 -- 172.21.15.81:0/3584101838 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0015a40 con 0x116ebe0 2024-05-08T23:31:43.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66de7fc700 1 -- 172.21.15.81:0/3584101838 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66f0015ba0 con 0x116ebe0 2024-05-08T23:31:43.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 1 -- 172.21.15.81:0/3584101838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66f8053410 con 0x116ebe0 2024-05-08T23:31:43.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66de7fc700 1 -- 172.21.15.81:0/3584101838 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66f0005a40 con 0x116ebe0 2024-05-08T23:31:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 10 client.5397 fetch_fsmap learned FSMap version 14 2024-05-08T23:31:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 1 -- 172.21.15.81:0/3584101838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x105e220 con 0x116ebe0 2024-05-08T23:31:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66de7fc700 1 -- 172.21.15.81:0/3584101838 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f8053410 con 0x116ebe0 2024-05-08T23:31:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66de7fc700 1 -- 172.21.15.81:0/3584101838 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 14) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f66f0005ba0 con 0x116ebe0 2024-05-08T23:31:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 10 client.5397 fetch_fsmap finished waiting for FSMap version 14 2024-05-08T23:31:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 10 client.5397 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T23:31:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 20 client.5397 populate_metadata read hostname 'smithi081' 2024-05-08T23:31:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 1 --2- 172.21.15.81:0/3584101838 >> [v2:172.21.15.117:6832/3091184157,v1:172.21.15.117:6835/3091184157] conn(0x11304e0 0x112d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 4 client.5397 mds_command: new command op to 14852 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:31:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f670905f740 1 -- 172.21.15.81:0/3584101838 --> [v2:172.21.15.117:6832/3091184157,v1:172.21.15.117:6835/3091184157] -- command(tid 0: {"prefix": "status"}) v1 -- 0xf3b1e0 con 0x11304e0 2024-05-08T23:31:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66feffd700 1 --2- 172.21.15.81:0/3584101838 >> [v2:172.21.15.117:6832/3091184157,v1:172.21.15.117:6835/3091184157] conn(0x11304e0 0x112d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66feffd700 1 --2- 172.21.15.81:0/3584101838 >> [v2:172.21.15.117:6832/3091184157,v1:172.21.15.117:6835/3091184157] conn(0x11304e0 0x112d710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=519d4493918ea773 server_cookie=c7ed8a7d588b5cb2 in_seq=0 out_seq=0 2024-05-08T23:31:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.612+0000 7f66de7fc700 10 client.5397 ms_handle_connect on v2:172.21.15.117:6832/3091184157 2024-05-08T23:31:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f66de7fc700 1 -- 172.21.15.81:0/3584101838 <== mds.0 v2:172.21.15.117:6832/3091184157 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0xf3b1e0 con 0x11304e0 2024-05-08T23:31:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f66de7fc700 10 client.5397 handle_command_reply: tid=0 2024-05-08T23:31:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f670905f740 1 client.5397 shutdown 2024-05-08T23:31:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f670905f740 1 -- 172.21.15.81:0/3584101838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 msgr2=0x117da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f670905f740 1 --2- 172.21.15.81:0/3584101838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x117da80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f66f002d450 tx=0x7f66f001c000 comp rx=0 tx=0).stop 2024-05-08T23:31:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f670905f740 1 -- 172.21.15.81:0/3584101838 shutdown_connections 2024-05-08T23:31:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f670905f740 1 --2- 172.21.15.81:0/3584101838 >> [v2:172.21.15.117:6832/3091184157,v1:172.21.15.117:6835/3091184157] conn(0x11304e0 0x112d710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f670905f740 1 --2- 172.21.15.81:0/3584101838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x1178420 0x112fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f670905f740 1 --2- 172.21.15.81:0/3584101838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x1177b00 0x112f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f66de7fc700 0 client.5397 ms_handle_reset on v2:172.21.15.117:6832/3091184157 2024-05-08T23:31:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f670905f740 1 --2- 172.21.15.81:0/3584101838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x116ebe0 0x117da80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f670905f740 1 -- 172.21.15.81:0/3584101838 >> 172.21.15.81:0/3584101838 conn(0xebf4d0 msgr2=0x11868a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f670905f740 1 -- 172.21.15.81:0/3584101838 shutdown_connections 2024-05-08T23:31:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f670905f740 1 -- 172.21.15.81:0/3584101838 wait complete. 2024-05-08T23:31:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f670905f740 20 client.5397 trim_cache size 0 max 16384 2024-05-08T23:31:43.619 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:31:43.619 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_fsid": "dee3de5e-bc15-439a-9578-b5583aefc890", 2024-05-08T23:31:43.619 INFO:teuthology.orchestra.run.smithi081.stdout: "whoami": 0, 2024-05-08T23:31:43.619 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 14852, 2024-05-08T23:31:43.619 INFO:teuthology.orchestra.run.smithi081.stdout: "want_state": "up:active", 2024-05-08T23:31:43.619 INFO:teuthology.orchestra.run.smithi081.stdout: "state": "up:active", 2024-05-08T23:31:43.619 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs", 2024-05-08T23:31:43.619 INFO:teuthology.orchestra.run.smithi081.stdout: "rank_uptime": 5.7223669470000003, 2024-05-08T23:31:43.619 INFO:teuthology.orchestra.run.smithi081.stdout: "mdsmap_epoch": 14, 2024-05-08T23:31:43.619 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch": 36, 2024-05-08T23:31:43.619 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch_barrier": 36, 2024-05-08T23:31:43.619 INFO:teuthology.orchestra.run.smithi081.stdout: "uptime": 13.044774986 2024-05-08T23:31:43.620 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:31:43.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f6704b4e700 1 -- 172.21.15.81:0/2518804681 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f66ec066730 msgr2=0x7f66ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f6704b4e700 1 --2- 172.21.15.81:0/2518804681 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f66ec066730 0x7f66ec068bf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009f80 tx=0x7f66f0009240 comp rx=0 tx=0).stop 2024-05-08T23:31:43.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f6704b4e700 1 -- 172.21.15.81:0/2518804681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67001f3d60 msgr2=0x7f67001f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f6704b4e700 1 --2- 172.21.15.81:0/2518804681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67001f3d60 0x7f67001f41e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f67000cf620 tx=0x7f66f800d430 comp rx=0 tx=0).stop 2024-05-08T23:31:43.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f6704b4e700 1 -- 172.21.15.81:0/2518804681 shutdown_connections 2024-05-08T23:31:43.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f6704b4e700 1 --2- 172.21.15.81:0/2518804681 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f66ec066730 0x7f66ec068bf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f6704b4e700 1 --2- 172.21.15.81:0/2518804681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67001f3d60 0x7f67001f41e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f6704b4e700 1 --2- 172.21.15.81:0/2518804681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67000a0870 0x7f67000cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f6704b4e700 1 --2- 172.21.15.81:0/2518804681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e8004140 0x7f6700150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f6704b4e700 1 -- 172.21.15.81:0/2518804681 >> 172.21.15.81:0/2518804681 conn(0x7f670009c390 msgr2=0x7f670006bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:43.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f6704b4e700 1 -- 172.21.15.81:0/2518804681 shutdown_connections 2024-05-08T23:31:43.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.616+0000 7f6704b4e700 1 -- 172.21.15.81:0/2518804681 wait complete. 2024-05-08T23:31:43.632 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:31:43.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150ed19700 1 Processor -- start 2024-05-08T23:31:43.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150ed19700 1 -- start start 2024-05-08T23:31:43.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150ed19700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f15080cb3b0 0x7f15081478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150ed19700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1508144d40 0x7f15081451b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150ed19700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15081656a0 con 0x7f1508165240 2024-05-08T23:31:43.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150ed19700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1508161c50 con 0x7f15080cb3b0 2024-05-08T23:31:43.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150ed19700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1508166210 con 0x7f1508144d40 2024-05-08T23:31:43.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150cab5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f15080cb3b0 0x7f15081478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150cab5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f15080cb3b0 0x7f15081478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60686/0 (socket says 172.21.15.81:60686) 2024-05-08T23:31:43.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150cab5700 1 -- 172.21.15.81:0/893366924 learned_addr learned my addr 172.21.15.81:0/893366924 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:43.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150d2b6700 1 --2- 172.21.15.81:0/893366924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1508144d40 0x7f15081451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.747 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150cab5700 1 -- 172.21.15.81:0/893366924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1508144d40 msgr2=0x7f15081451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:43.747 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150cab5700 1 --2- 172.21.15.81:0/893366924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1508144d40 0x7f15081451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.747 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150cab5700 1 -- 172.21.15.81:0/893366924 >> v1:172.21.15.117:6789/0 conn(0x7f1508165240 legacy=0x7f1508162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.747 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f15077fe700 1 -- 172.21.15.81:0/893366924 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4200300370 0 0) 0x7f15081656a0 con 0x7f1508165240 2024-05-08T23:31:43.747 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150cab5700 1 -- 172.21.15.81:0/893366924 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1508165970 con 0x7f15080cb3b0 2024-05-08T23:31:43.747 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150cab5700 1 --2- 172.21.15.81:0/893366924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15080cb3b0 0x7f15081478c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1500009e50 tx=0x7f150000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1110daf691c17b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:43.747 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f15077fe700 1 -- 172.21.15.81:0/893366924 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150000e070 con 0x7f15080cb3b0 2024-05-08T23:31:43.747 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f15077fe700 1 -- 172.21.15.81:0/893366924 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150000b910 con 0x7f15080cb3b0 2024-05-08T23:31:43.747 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f15077fe700 1 -- 172.21.15.81:0/893366924 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150000bcc0 con 0x7f15080cb3b0 2024-05-08T23:31:43.748 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150ed19700 1 -- 172.21.15.81:0/893366924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15080cb3b0 msgr2=0x7f15081478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.748 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.744+0000 7f150ed19700 1 --2- 172.21.15.81:0/893366924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15080cb3b0 0x7f15081478c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1500009e50 tx=0x7f150000b330 comp rx=0 tx=0).stop 2024-05-08T23:31:43.748 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.748+0000 7f150ed19700 1 -- 172.21.15.81:0/893366924 shutdown_connections 2024-05-08T23:31:43.748 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.748+0000 7f150ed19700 1 --2- 172.21.15.81:0/893366924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1508144d40 0x7f15081451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.748+0000 7f150ed19700 1 --2- 172.21.15.81:0/893366924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15080cb3b0 0x7f15081478c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.748+0000 7f150ed19700 1 -- 172.21.15.81:0/893366924 >> 172.21.15.81:0/893366924 conn(0x7f150806a360 msgr2=0x7f150806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:43.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.748+0000 7f150ed19700 1 -- 172.21.15.81:0/893366924 shutdown_connections 2024-05-08T23:31:43.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.748+0000 7f150ed19700 1 -- 172.21.15.81:0/893366924 wait complete. 2024-05-08T23:31:43.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.748+0000 7f150ed19700 1 Processor -- start 2024-05-08T23:31:43.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.748+0000 7f150ed19700 1 -- start start 2024-05-08T23:31:43.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150ed19700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15080cb3b0 0x7f1508148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150ed19700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1508144d40 0x7f1508157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150ed19700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1508165240 0x7f150815e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150ed19700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150814a260 con 0x7f15080cb3b0 2024-05-08T23:31:43.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150ed19700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1508149070 con 0x7f1508165240 2024-05-08T23:31:43.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150ed19700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1508158340 con 0x7f1508144d40 2024-05-08T23:31:43.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150cab5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15080cb3b0 0x7f1508148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150cab5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15080cb3b0 0x7f1508148b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60688/0 (socket says 172.21.15.81:60688) 2024-05-08T23:31:43.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150cab5700 1 -- 172.21.15.81:0/4109798013 learned_addr learned my addr 172.21.15.81:0/4109798013 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:43.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150d2b6700 1 --2- 172.21.15.81:0/4109798013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1508165240 0x7f150815e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f1507fff700 1 --2- 172.21.15.81:0/4109798013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1508144d40 0x7f1508157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150cab5700 1 -- 172.21.15.81:0/4109798013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1508144d40 msgr2=0x7f1508157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150cab5700 1 --2- 172.21.15.81:0/4109798013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1508144d40 0x7f1508157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150cab5700 1 -- 172.21.15.81:0/4109798013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1508165240 msgr2=0x7f150815e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150cab5700 1 --2- 172.21.15.81:0/4109798013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1508165240 0x7f150815e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150cab5700 1 -- 172.21.15.81:0/4109798013 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1500009b30 con 0x7f15080cb3b0 2024-05-08T23:31:43.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150d2b6700 1 --2- 172.21.15.81:0/4109798013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1508165240 0x7f150815e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:43.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150cab5700 1 --2- 172.21.15.81:0/4109798013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15080cb3b0 0x7f1508148b30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f150000b370 tx=0x7f1500004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:43.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f15057fa700 1 -- 172.21.15.81:0/4109798013 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15000099b0 con 0x7f15080cb3b0 2024-05-08T23:31:43.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f15057fa700 1 -- 172.21.15.81:0/4109798013 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15000092e0 con 0x7f15080cb3b0 2024-05-08T23:31:43.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150ed19700 1 -- 172.21.15.81:0/4109798013 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15081495e0 con 0x7f15080cb3b0 2024-05-08T23:31:43.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f15057fa700 1 -- 172.21.15.81:0/4109798013 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150001c640 con 0x7f15080cb3b0 2024-05-08T23:31:43.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150ed19700 1 -- 172.21.15.81:0/4109798013 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1508149ba0 con 0x7f15080cb3b0 2024-05-08T23:31:43.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f15057fa700 1 -- 172.21.15.81:0/4109798013 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f150001c7a0 con 0x7f15080cb3b0 2024-05-08T23:31:43.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f150ed19700 1 -- 172.21.15.81:0/4109798013 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1508071730 con 0x7f15080cb3b0 2024-05-08T23:31:43.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f15057fa700 1 --2- 172.21.15.81:0/4109798013 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f14e8066730 0x7f14e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.752+0000 7f15057fa700 1 -- 172.21.15.81:0/4109798013 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1500085e40 con 0x7f15080cb3b0 2024-05-08T23:31:43.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.756+0000 7f1507fff700 1 --2- 172.21.15.81:0/4109798013 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f14e8066730 0x7f14e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.756+0000 7f1507fff700 1 --2- 172.21.15.81:0/4109798013 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f14e8066730 0x7f14e8068bf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1508161640 tx=0x7f14f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:43.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.760+0000 7f15057fa700 1 -- 172.21.15.81:0/4109798013 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1500053240 con 0x7f15080cb3b0 2024-05-08T23:31:43.969 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.968+0000 7f150ed19700 1 -- 172.21.15.81:0/4109798013 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f15080684b0 con 0x7f15080cb3b0 2024-05-08T23:31:43.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.968+0000 7f15057fa700 1 -- 172.21.15.81:0/4109798013 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1500056860 con 0x7f15080cb3b0 2024-05-08T23:31:43.970 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:31:43.972 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.972+0000 7f150ed19700 1 -- 172.21.15.81:0/4109798013 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f14e8066730 msgr2=0x7f14e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.972 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.972+0000 7f150ed19700 1 --2- 172.21.15.81:0/4109798013 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f14e8066730 0x7f14e8068bf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1508161640 tx=0x7f14f0009040 comp rx=0 tx=0).stop 2024-05-08T23:31:43.972 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.972+0000 7f150ed19700 1 -- 172.21.15.81:0/4109798013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15080cb3b0 msgr2=0x7f1508148b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.972+0000 7f150ed19700 1 --2- 172.21.15.81:0/4109798013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15080cb3b0 0x7f1508148b30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f150000b370 tx=0x7f1500004af0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.972+0000 7f150ed19700 1 -- 172.21.15.81:0/4109798013 shutdown_connections 2024-05-08T23:31:43.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.972+0000 7f150ed19700 1 --2- 172.21.15.81:0/4109798013 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f14e8066730 0x7f14e8068bf0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.972+0000 7f150ed19700 1 --2- 172.21.15.81:0/4109798013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1508165240 0x7f150815e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.972+0000 7f150ed19700 1 --2- 172.21.15.81:0/4109798013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1508144d40 0x7f1508157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.972+0000 7f150ed19700 1 --2- 172.21.15.81:0/4109798013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15080cb3b0 0x7f1508148b30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.972+0000 7f150ed19700 1 -- 172.21.15.81:0/4109798013 >> 172.21.15.81:0/4109798013 conn(0x7f150806a360 msgr2=0x7f1508145780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:43.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.972+0000 7f150ed19700 1 -- 172.21.15.81:0/4109798013 shutdown_connections 2024-05-08T23:31:43.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:43.972+0000 7f150ed19700 1 -- 172.21.15.81:0/4109798013 wait complete. 2024-05-08T23:31:43.984 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:31:43.984 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:31:43.984 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2024-05-08T23:31:43.984 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:31:43.984 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:31:43.985 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:31:43.985 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:31:43.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:31:43.985 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:31:43.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:43.985 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2024-05-08T23:31:43.990 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:31:43.990 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:31:43.990 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:31:43.990 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:43.990 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-05-08T23:31:43.990 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:43.990 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:31:43.990 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:43.991 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:31:43.991 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:43.991 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:31:43.991 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:43.991 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:31:43.991 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:43.991 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T23:31:43.991 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42512sec preferred_lft 42512sec 2024-05-08T23:31:43.991 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2024-05-08T23:31:43.991 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:43.991 INFO:teuthology.orchestra.run.smithi081.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T23:31:43.991 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 02:42:22:62:7a:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:43.991 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T23:31:43.991 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:43.992 INFO:teuthology.orchestra.run.smithi081.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:31:43.992 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:2b:10:6b:08:e3 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:43.992 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:31:43.992 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:43.992 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::48fc:c7ff:fe41:6d94/64 scope link 2024-05-08T23:31:43.992 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:43.992 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:31:43.992 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:2b:10:6b:08:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:31:43.992 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::702b:10ff:fe6b:8e3/64 scope link 2024-05-08T23:31:43.992 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:43.992 INFO:teuthology.orchestra.run.smithi081.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:31:43.992 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:9d:1b:4e:6b:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:31:43.992 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::709d:1bff:fe4e:6bd5/64 scope link 2024-05-08T23:31:43.992 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:43.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:43.993 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:31:44.046 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-05-08T23:31:44.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:44.048 DEBUG:teuthology.orchestra.run.smithi081:> route 2024-05-08T23:31:44.057 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2024-05-08T23:31:44.057 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:31:44.057 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T23:31:44.057 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T23:31:44.057 INFO:teuthology.orchestra.run.smithi081.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T23:31:44.057 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T23:31:44.057 INFO:teuthology.orchestra.run.smithi081.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T23:31:44.057 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:31:44.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:44.058 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-05-08T23:31:44.058 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-05-08T23:31:44.058 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T23:31:44.058 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T23:31:44.058 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T23:31:44.058 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-05-08T23:31:44.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:44.093 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-05-08T23:31:44.100 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T23:31:44.100 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:31:44.100 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:31:44.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:44.100 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2024-05-08T23:31:44.159 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T23:31:44.160 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:31:44.160 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:31:44.167 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:31:44.167 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:31:44.167 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:31:44.174 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:31:44.231 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:31:44.231 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:31:44.244 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:31:44.245 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:31:44.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:44.246 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:31:44.251 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T23:31:44.251 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T23:31:44.253 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:31:44.270 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:31:44.270 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:31:44.328 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:31:44.329 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:31:44.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:44.330 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:31:44.340 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2024-05-08T23:31:44.340+0000 7fb28ea28640 -1 init, newargv = 0x55cb72e5bb70 newargc=15 2024-05-08T23:31:44.341 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[36330]: starting ceph client 2024-05-08T23:31:44.348 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[36330]: starting fuse 2024-05-08T23:31:45.436 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:31:45.455 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:31:45.456 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:31:45.517 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:31:45.518 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:31:45.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:45.519 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:31:45.525 INFO:teuthology.orchestra.run.smithi081.stdout:52 2024-05-08T23:31:45.525 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T23:31:45.525 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:31:45.525 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2024-05-08T23:31:45.526 DEBUG:teuthology.orchestra.run.smithi081:> import re 2024-05-08T23:31:45.526 DEBUG:teuthology.orchestra.run.smithi081:> import os 2024-05-08T23:31:45.526 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2024-05-08T23:31:45.526 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:31:45.526 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2024-05-08T23:31:45.526 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T23:31:45.526 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2024-05-08T23:31:45.526 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T23:31:45.526 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:31:45.526 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2024-05-08T23:31:45.526 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2024-05-08T23:31:45.526 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2024-05-08T23:31:45.526 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2024-05-08T23:31:45.526 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:31:45.526 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2024-05-08T23:31:45.527 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T23:31:45.527 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T23:31:45.527 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T23:31:45.527 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2024-05-08T23:31:45.527 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2024-05-08T23:31:45.527 DEBUG:teuthology.orchestra.run.smithi081:> return f 2024-05-08T23:31:45.527 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T23:31:45.527 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:31:45.527 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2024-05-08T23:31:45.527 DEBUG:teuthology.orchestra.run.smithi081:> ' 2024-05-08T23:31:45.608 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.36330.asok 2024-05-08T23:31:45.795 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36330.asok 2024-05-08T23:31:45.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:45.795 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36330.asok status 2024-05-08T23:31:45.949 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:31:45.949 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2024-05-08T23:31:45.949 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T23:31:45.949 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:31:45.949 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2024-05-08T23:31:45.949 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2024-05-08T23:31:45.949 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T23:31:45.949 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "36330", 2024-05-08T23:31:45.949 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2024-05-08T23:31:45.949 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:45.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2024-05-08T23:31:45.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2024-05-08T23:31:45.949 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 5412, 2024-05-08T23:31:45.950 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2024-05-08T23:31:45.950 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2024-05-08T23:31:45.950 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2024-05-08T23:31:45.950 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 5412 2024-05-08T23:31:45.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:45.950 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:31:45.950 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:31:45.950 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:31:45.950 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3630142461 2024-05-08T23:31:45.950 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:31:45.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:45.950 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:31:45.950 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:31:45.950 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:31:45.950 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3630142461 2024-05-08T23:31:45.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:31:45.951 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.5412 192.168.0.1:0/3630142461", 2024-05-08T23:31:45.951 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/3630142461", 2024-05-08T23:31:45.951 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2024-05-08T23:31:45.951 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 14, 2024-05-08T23:31:45.951 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 36, 2024-05-08T23:31:45.951 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2024-05-08T23:31:45.951 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2024-05-08T23:31:45.951 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2024-05-08T23:31:45.951 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:31:45.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:45.959 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2024-05-08T23:31:45.959 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T23:31:45.965 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2024-05-08T23:31:45.965 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T23:31:45.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:45.965 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:31:45.977 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2024-05-08T23:31:46.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051eb3c700 1 Processor -- start 2024-05-08T23:31:46.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051eb3c700 1 -- start start 2024-05-08T23:31:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051eb3c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f05180cb1c0 0x7f05181590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051eb3c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180b40d0 0x7f0518159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:46.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051eb3c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051815e200 con 0x7f05180b3cf0 2024-05-08T23:31:46.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051eb3c700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f051815d460 con 0x7f05180cb1c0 2024-05-08T23:31:46.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051eb3c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051815d740 con 0x7f05180b40d0 2024-05-08T23:31:46.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051e33b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180b40d0 0x7f0518159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:46.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051d339700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f05180cb1c0 0x7f05181590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051db3a700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f05180b3cf0 0x7f0518071c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:60310/0 (socket says 172.21.15.81:60310) 2024-05-08T23:31:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051db3a700 1 -- 172.21.15.81:0/2109553801 learned_addr learned my addr 172.21.15.81:0/2109553801 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051cb38700 1 -- 172.21.15.81:0/2109553801 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43600740 0 0) 0x7f051815e200 con 0x7f05180b3cf0 2024-05-08T23:31:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051cb38700 1 -- 172.21.15.81:0/2109553801 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05040035d0 con 0x7f05180b3cf0 2024-05-08T23:31:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051e33b700 1 -- 172.21.15.81:0/2109553801 >> v2:172.21.15.117:3300/0 conn(0x7f05180cb1c0 msgr2=0x7f05181590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051e33b700 1 --2- 172.21.15.81:0/2109553801 >> v2:172.21.15.117:3300/0 conn(0x7f05180cb1c0 0x7f05181590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051e33b700 1 -- 172.21.15.81:0/2109553801 >> v1:172.21.15.117:6789/0 conn(0x7f05180b3cf0 legacy=0x7f0518071c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051e33b700 1 -- 172.21.15.81:0/2109553801 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051815da20 con 0x7f05180b40d0 2024-05-08T23:31:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051d339700 1 --2- 172.21.15.81:0/2109553801 >> v2:172.21.15.117:3300/0 conn(0x7f05180cb1c0 0x7f05181590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051e33b700 1 --2- 172.21.15.81:0/2109553801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180b40d0 0x7f0518159630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0510009d90 tx=0x7f051000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bde651e3c07e27f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051cb38700 1 -- 172.21.15.81:0/2109553801 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051000e070 con 0x7f05180b40d0 2024-05-08T23:31:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051cb38700 1 -- 172.21.15.81:0/2109553801 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051000bb20 con 0x7f05180b40d0 2024-05-08T23:31:46.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.100+0000 7f051cb38700 1 -- 172.21.15.81:0/2109553801 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510012430 con 0x7f05180b40d0 2024-05-08T23:31:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.104+0000 7f051eb3c700 1 -- 172.21.15.81:0/2109553801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180b40d0 msgr2=0x7f0518159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.104+0000 7f051eb3c700 1 --2- 172.21.15.81:0/2109553801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180b40d0 0x7f0518159630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0510009d90 tx=0x7f051000b330 comp rx=0 tx=0).stop 2024-05-08T23:31:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.104+0000 7f051eb3c700 1 -- 172.21.15.81:0/2109553801 shutdown_connections 2024-05-08T23:31:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.104+0000 7f051eb3c700 1 --2- 172.21.15.81:0/2109553801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180b40d0 0x7f0518159630 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.104+0000 7f051eb3c700 1 --2- 172.21.15.81:0/2109553801 >> v2:172.21.15.117:3300/0 conn(0x7f05180cb1c0 0x7f05181590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.104+0000 7f051eb3c700 1 -- 172.21.15.81:0/2109553801 >> 172.21.15.81:0/2109553801 conn(0x7f051806d650 msgr2=0x7f051806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.104+0000 7f051eb3c700 1 -- 172.21.15.81:0/2109553801 shutdown_connections 2024-05-08T23:31:46.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.104+0000 7f051eb3c700 1 -- 172.21.15.81:0/2109553801 wait complete. 2024-05-08T23:31:46.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.104+0000 7f051eb3c700 1 Processor -- start 2024-05-08T23:31:46.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.104+0000 7f051eb3c700 1 -- start start 2024-05-08T23:31:46.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.104+0000 7f051eb3c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180b3cf0 0x7f051813e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:46.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.104+0000 7f051eb3c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05180b40d0 0x7f051814fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:46.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051eb3c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05180cb1c0 0x7f05181456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:46.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051eb3c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051814f6b0 con 0x7f05180b40d0 2024-05-08T23:31:46.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051eb3c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051814f380 con 0x7f05180cb1c0 2024-05-08T23:31:46.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051eb3c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051815cb70 con 0x7f05180b3cf0 2024-05-08T23:31:46.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051d339700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05180b40d0 0x7f051814fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:46.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051d339700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05180b40d0 0x7f051814fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58256/0 (socket says 172.21.15.81:58256) 2024-05-08T23:31:46.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051d339700 1 -- 172.21.15.81:0/4158191460 learned_addr learned my addr 172.21.15.81:0/4158191460 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:46.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051e33b700 1 --2- 172.21.15.81:0/4158191460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05180cb1c0 0x7f05181456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:46.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051db3a700 1 --2- 172.21.15.81:0/4158191460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180b3cf0 0x7f051813e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:46.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051e33b700 1 -- 172.21.15.81:0/4158191460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180b3cf0 msgr2=0x7f051813e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051e33b700 1 --2- 172.21.15.81:0/4158191460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180b3cf0 0x7f051813e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051e33b700 1 -- 172.21.15.81:0/4158191460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05180b40d0 msgr2=0x7f051814fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051e33b700 1 --2- 172.21.15.81:0/4158191460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05180b40d0 0x7f051814fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051e33b700 1 -- 172.21.15.81:0/4158191460 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05080094b0 con 0x7f05180cb1c0 2024-05-08T23:31:46.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051e33b700 1 --2- 172.21.15.81:0/4158191460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05180cb1c0 0x7f05181456a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0510016000 tx=0x7f0510004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:46.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f050e7fc700 1 -- 172.21.15.81:0/4158191460 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510012430 con 0x7f05180cb1c0 2024-05-08T23:31:46.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f050e7fc700 1 -- 172.21.15.81:0/4158191460 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0510020070 con 0x7f05180cb1c0 2024-05-08T23:31:46.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051eb3c700 1 -- 172.21.15.81:0/4158191460 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0510009a70 con 0x7f05180cb1c0 2024-05-08T23:31:46.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f050e7fc700 1 -- 172.21.15.81:0/4158191460 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05100098f0 con 0x7f05180cb1c0 2024-05-08T23:31:46.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051eb3c700 1 -- 172.21.15.81:0/4158191460 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0518149ea0 con 0x7f05180cb1c0 2024-05-08T23:31:46.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.108+0000 7f051eb3c700 1 -- 172.21.15.81:0/4158191460 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f051814f7f0 con 0x7f05180cb1c0 2024-05-08T23:31:46.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.112+0000 7f050e7fc700 1 -- 172.21.15.81:0/4158191460 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0510004040 con 0x7f05180cb1c0 2024-05-08T23:31:46.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.112+0000 7f050e7fc700 1 --2- 172.21.15.81:0/4158191460 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f04fc066760 0x7f04fc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:46.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.112+0000 7f050e7fc700 1 -- 172.21.15.81:0/4158191460 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0510022070 con 0x7f05180cb1c0 2024-05-08T23:31:46.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.112+0000 7f051db3a700 1 --2- 172.21.15.81:0/4158191460 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f04fc066760 0x7f04fc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:46.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.112+0000 7f051db3a700 1 --2- 172.21.15.81:0/4158191460 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f04fc066760 0x7f04fc068c20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0514007fd0 tx=0x7f0514007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:46.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.124+0000 7f050e7fc700 1 -- 172.21.15.81:0/4158191460 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f051814f7f0 con 0x7f05180cb1c0 2024-05-08T23:31:46.294 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.292+0000 7f051eb3c700 1 -- 172.21.15.81:0/4158191460 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) v1 -- 0x7f051814f7f0 con 0x7f05180cb1c0 2024-05-08T23:31:46.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.948+0000 7f050e7fc700 1 -- 172.21.15.81:0/4158191460 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v37) v1 ==== 124+0+0 (secure 0 0 0) 0x7f051814f7f0 con 0x7f05180cb1c0 2024-05-08T23:31:46.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.952+0000 7f051eb3c700 1 -- 172.21.15.81:0/4158191460 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f04fc066760 msgr2=0x7f04fc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.952+0000 7f051eb3c700 1 --2- 172.21.15.81:0/4158191460 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f04fc066760 0x7f04fc068c20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0514007fd0 tx=0x7f0514007de0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.952+0000 7f051eb3c700 1 -- 172.21.15.81:0/4158191460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05180cb1c0 msgr2=0x7f05181456a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.952+0000 7f051eb3c700 1 --2- 172.21.15.81:0/4158191460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05180cb1c0 0x7f05181456a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0510016000 tx=0x7f0510004af0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.952+0000 7f051eb3c700 1 -- 172.21.15.81:0/4158191460 shutdown_connections 2024-05-08T23:31:46.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.952+0000 7f051eb3c700 1 --2- 172.21.15.81:0/4158191460 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f04fc066760 0x7f04fc068c20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.952+0000 7f051eb3c700 1 --2- 172.21.15.81:0/4158191460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05180cb1c0 0x7f05181456a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.952+0000 7f051eb3c700 1 --2- 172.21.15.81:0/4158191460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05180b40d0 0x7f051814fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.952+0000 7f051eb3c700 1 --2- 172.21.15.81:0/4158191460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180b3cf0 0x7f051813e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.952+0000 7f051eb3c700 1 -- 172.21.15.81:0/4158191460 >> 172.21.15.81:0/4158191460 conn(0x7f051806d650 msgr2=0x7f05180ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:46.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.956+0000 7f051eb3c700 1 -- 172.21.15.81:0/4158191460 shutdown_connections 2024-05-08T23:31:46.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:46.956+0000 7f051eb3c700 1 -- 172.21.15.81:0/4158191460 wait complete. 2024-05-08T23:31:46.957 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'data_new' created 2024-05-08T23:31:46.968 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs data_new 2024-05-08T23:31:47.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.076+0000 7f2e3f59e700 1 Processor -- start 2024-05-08T23:31:47.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.076+0000 7f2e3f59e700 1 -- start start 2024-05-08T23:31:47.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.076+0000 7f2e3f59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2e400abe80 0x7f2e4006a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.076+0000 7f2e3f59e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e400b4160 con 0x7f2e4015af20 2024-05-08T23:31:47.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.076+0000 7f2e3f59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4006b0f0 con 0x7f2e4006acc0 2024-05-08T23:31:47.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.076+0000 7f2e3f59e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e400adb50 con 0x7f2e400abe80 2024-05-08T23:31:47.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.076+0000 7f2e37fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2e400abe80 0x7f2e4006a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.076+0000 7f2e37fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2e400abe80 0x7f2e4006a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58268/0 (socket says 172.21.15.81:58268) 2024-05-08T23:31:47.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.076+0000 7f2e37fff700 1 -- 172.21.15.81:0/1141801698 learned_addr learned my addr 172.21.15.81:0/1141801698 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:47.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.076+0000 7f2e3e59c700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2e4015af20 0x7f2e400b3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:48080/0 (socket says 172.21.15.81:48080) 2024-05-08T23:31:47.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.076+0000 7f2e37fff700 1 -- 172.21.15.81:0/1141801698 >> v1:172.21.15.117:6789/0 conn(0x7f2e4006acc0 legacy=0x7f2e400b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.076+0000 7f2e37fff700 1 -- 172.21.15.81:0/1141801698 >> v1:172.21.15.81:6789/0 conn(0x7f2e4015af20 legacy=0x7f2e400b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.076+0000 7f2e3dd9b700 1 -- 172.21.15.81:0/1141801698 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316530491 0 0) 0x7f2e400b4160 con 0x7f2e4015af20 2024-05-08T23:31:47.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.076+0000 7f2e37fff700 1 -- 172.21.15.81:0/1141801698 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e400ade30 con 0x7f2e400abe80 2024-05-08T23:31:47.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e37fff700 1 --2- 172.21.15.81:0/1141801698 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e400abe80 0x7f2e4006a780 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009910 tx=0x7f2e28009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3563a1a76ab204e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:47.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3dd9b700 1 -- 172.21.15.81:0/1141801698 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2800ba60 con 0x7f2e400abe80 2024-05-08T23:31:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3dd9b700 1 -- 172.21.15.81:0/1141801698 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2800bbc0 con 0x7f2e400abe80 2024-05-08T23:31:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3dd9b700 1 -- 172.21.15.81:0/1141801698 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28009e60 con 0x7f2e400abe80 2024-05-08T23:31:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3f59e700 1 -- 172.21.15.81:0/1141801698 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e400abe80 msgr2=0x7f2e4006a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3f59e700 1 --2- 172.21.15.81:0/1141801698 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e400abe80 0x7f2e4006a780 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009910 tx=0x7f2e28009c20 comp rx=0 tx=0).stop 2024-05-08T23:31:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3f59e700 1 -- 172.21.15.81:0/1141801698 shutdown_connections 2024-05-08T23:31:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3f59e700 1 --2- 172.21.15.81:0/1141801698 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e400abe80 0x7f2e4006a780 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3f59e700 1 -- 172.21.15.81:0/1141801698 >> 172.21.15.81:0/1141801698 conn(0x7f2e400b0e00 msgr2=0x7f2e400b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3f59e700 1 -- 172.21.15.81:0/1141801698 shutdown_connections 2024-05-08T23:31:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3f59e700 1 -- 172.21.15.81:0/1141801698 wait complete. 2024-05-08T23:31:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3f59e700 1 Processor -- start 2024-05-08T23:31:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3f59e700 1 -- start start 2024-05-08T23:31:47.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3f59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4006acc0 0x7f2e40143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3f59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e400abe80 0x7f2e401f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3f59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4015af20 0x7f2e401f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3f59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e401527e0 con 0x7f2e4015af20 2024-05-08T23:31:47.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3f59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e40154340 con 0x7f2e400abe80 2024-05-08T23:31:47.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.080+0000 7f2e3f59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e40146eb0 con 0x7f2e4006acc0 2024-05-08T23:31:47.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e3ed9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4015af20 0x7f2e401f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e3ed9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4015af20 0x7f2e401f6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58280/0 (socket says 172.21.15.81:58280) 2024-05-08T23:31:47.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e3ed9d700 1 -- 172.21.15.81:0/2313391452 learned_addr learned my addr 172.21.15.81:0/2313391452 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:47.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e37fff700 1 --2- 172.21.15.81:0/2313391452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e400abe80 0x7f2e401f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e3e59c700 1 --2- 172.21.15.81:0/2313391452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4006acc0 0x7f2e40143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e3ed9d700 1 -- 172.21.15.81:0/2313391452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4006acc0 msgr2=0x7f2e40143980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e3ed9d700 1 --2- 172.21.15.81:0/2313391452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4006acc0 0x7f2e40143980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e3ed9d700 1 -- 172.21.15.81:0/2313391452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e400abe80 msgr2=0x7f2e401f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e3ed9d700 1 --2- 172.21.15.81:0/2313391452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e400abe80 0x7f2e401f5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e3ed9d700 1 -- 172.21.15.81:0/2313391452 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e28009570 con 0x7f2e4015af20 2024-05-08T23:31:47.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e3ed9d700 1 --2- 172.21.15.81:0/2313391452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4015af20 0x7f2e401f6350 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800cbd0 tx=0x7f2e3800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:47.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e36ffd700 1 -- 172.21.15.81:0/2313391452 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38003c40 con 0x7f2e4015af20 2024-05-08T23:31:47.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e36ffd700 1 -- 172.21.15.81:0/2313391452 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e38003da0 con 0x7f2e4015af20 2024-05-08T23:31:47.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e36ffd700 1 -- 172.21.15.81:0/2313391452 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3800ab40 con 0x7f2e4015af20 2024-05-08T23:31:47.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e3f59e700 1 -- 172.21.15.81:0/2313391452 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e40147190 con 0x7f2e4015af20 2024-05-08T23:31:47.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e3f59e700 1 -- 172.21.15.81:0/2313391452 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e40152400 con 0x7f2e4015af20 2024-05-08T23:31:47.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e3f59e700 1 -- 172.21.15.81:0/2313391452 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e400677c0 con 0x7f2e4015af20 2024-05-08T23:31:47.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.084+0000 7f2e36ffd700 1 -- 172.21.15.81:0/2313391452 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2e3800aca0 con 0x7f2e4015af20 2024-05-08T23:31:47.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.088+0000 7f2e36ffd700 1 --2- 172.21.15.81:0/2313391452 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e20066730 0x7f2e20068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.088+0000 7f2e36ffd700 1 -- 172.21.15.81:0/2313391452 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6072+0+0 (secure 0 0 0) 0x7f2e38084290 con 0x7f2e4015af20 2024-05-08T23:31:47.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.088+0000 7f2e3e59c700 1 --2- 172.21.15.81:0/2313391452 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e20066730 0x7f2e20068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.088+0000 7f2e3e59c700 1 --2- 172.21.15.81:0/2313391452 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e20066730 0x7f2e20068bf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2e300070a0 tx=0x7f2e30000f10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:47.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.092+0000 7f2e36ffd700 1 -- 172.21.15.81:0/2313391452 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2e38051520 con 0x7f2e4015af20 2024-05-08T23:31:47.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.304+0000 7f2e3f59e700 1 -- 172.21.15.81:0/2313391452 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"} v 0) v1 -- 0x7f2e40152920 con 0x7f2e4015af20 2024-05-08T23:31:47.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.316+0000 7f2e36ffd700 1 -- 172.21.15.81:0/2313391452 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"}]=0 added data pool 6 to fsmap v15) v1 ==== 131+0+0 (secure 0 0 0) 0x7f2e40152920 con 0x7f2e4015af20 2024-05-08T23:31:47.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.320+0000 7f2e3f59e700 1 -- 172.21.15.81:0/2313391452 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e20066730 msgr2=0x7f2e20068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.320+0000 7f2e3f59e700 1 --2- 172.21.15.81:0/2313391452 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e20066730 0x7f2e20068bf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2e300070a0 tx=0x7f2e30000f10 comp rx=0 tx=0).stop 2024-05-08T23:31:47.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.320+0000 7f2e3f59e700 1 -- 172.21.15.81:0/2313391452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4015af20 msgr2=0x7f2e401f6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.320+0000 7f2e3f59e700 1 --2- 172.21.15.81:0/2313391452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4015af20 0x7f2e401f6350 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800cbd0 tx=0x7f2e3800cee0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.324+0000 7f2e3f59e700 1 -- 172.21.15.81:0/2313391452 shutdown_connections 2024-05-08T23:31:47.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.324+0000 7f2e3f59e700 1 --2- 172.21.15.81:0/2313391452 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e20066730 0x7f2e20068bf0 secure :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2e300070a0 tx=0x7f2e30000f10 comp rx=0 tx=0).stop 2024-05-08T23:31:47.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.324+0000 7f2e3f59e700 1 --2- 172.21.15.81:0/2313391452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4015af20 0x7f2e401f6350 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.324+0000 7f2e3f59e700 1 --2- 172.21.15.81:0/2313391452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e400abe80 0x7f2e401f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.324+0000 7f2e3f59e700 1 --2- 172.21.15.81:0/2313391452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4006acc0 0x7f2e40143980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.324+0000 7f2e3f59e700 1 -- 172.21.15.81:0/2313391452 >> 172.21.15.81:0/2313391452 conn(0x7f2e400b0e00 msgr2=0x7f2e400ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:47.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.324+0000 7f2e3f59e700 1 -- 172.21.15.81:0/2313391452 shutdown_connections 2024-05-08T23:31:47.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.324+0000 7f2e3f59e700 1 -- 172.21.15.81:0/2313391452 wait complete. 2024-05-08T23:31:47.328 INFO:teuthology.orchestra.run.smithi081.stderr:added data pool 6 to fsmap 2024-05-08T23:31:47.340 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:31:47.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f72527f1700 1 Processor -- start 2024-05-08T23:31:47.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f72527f1700 1 -- start start 2024-05-08T23:31:47.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f72527f1700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7244033fb0 0x7f72440eb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f72527f1700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7244048210 0x7f72440eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f72527f1700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724404fe20 con 0x7f7244047cc0 2024-05-08T23:31:47.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f72527f1700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724404fb80 con 0x7f7244048210 2024-05-08T23:31:47.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f72527f1700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72440502b0 con 0x7f7244033fb0 2024-05-08T23:31:47.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f7250fee700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7244048210 0x7f72440eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f7250fee700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7244048210 0x7f72440eb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58282/0 (socket says 172.21.15.81:58282) 2024-05-08T23:31:47.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f7250fee700 1 -- 172.21.15.81:0/1326072849 learned_addr learned my addr 172.21.15.81:0/1326072849 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:47.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f72517ef700 1 --2- 172.21.15.81:0/1326072849 >> v2:172.21.15.117:3300/0 conn(0x7f7244033fb0 0x7f72440eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f7250fee700 1 -- 172.21.15.81:0/1326072849 >> v2:172.21.15.117:3300/0 conn(0x7f7244033fb0 msgr2=0x7f72440eb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:47.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f7250fee700 1 --2- 172.21.15.81:0/1326072849 >> v2:172.21.15.117:3300/0 conn(0x7f7244033fb0 0x7f72440eb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f7250fee700 1 -- 172.21.15.81:0/1326072849 >> v1:172.21.15.117:6789/0 conn(0x7f7244047cc0 legacy=0x7f7244005f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:47.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f7250fee700 1 -- 172.21.15.81:0/1326072849 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72440ec0c0 con 0x7f7244048210 2024-05-08T23:31:47.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f72517ef700 1 --2- 172.21.15.81:0/1326072849 >> v2:172.21.15.117:3300/0 conn(0x7f7244033fb0 0x7f72440eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:47.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f7250fee700 1 --2- 172.21.15.81:0/1326072849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7244048210 0x7f72440eb760 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7244007b10 tx=0x7f7240009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d279158fcd0c5402 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:47.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f723ffff700 1 -- 172.21.15.81:0/1326072849 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240004230 con 0x7f7244048210 2024-05-08T23:31:47.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f723ffff700 1 -- 172.21.15.81:0/1326072849 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7240004390 con 0x7f7244048210 2024-05-08T23:31:47.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f723ffff700 1 -- 172.21.15.81:0/1326072849 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72400036c0 con 0x7f7244048210 2024-05-08T23:31:47.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f72527f1700 1 -- 172.21.15.81:0/1326072849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7244048210 msgr2=0x7f72440eb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f72527f1700 1 --2- 172.21.15.81:0/1326072849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7244048210 0x7f72440eb760 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7244007b10 tx=0x7f7240009bf0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f72527f1700 1 -- 172.21.15.81:0/1326072849 shutdown_connections 2024-05-08T23:31:47.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.444+0000 7f72527f1700 1 --2- 172.21.15.81:0/1326072849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7244048210 0x7f72440eb760 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f72527f1700 1 --2- 172.21.15.81:0/1326072849 >> v2:172.21.15.117:3300/0 conn(0x7f7244033fb0 0x7f72440eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f72527f1700 1 -- 172.21.15.81:0/1326072849 >> 172.21.15.81:0/1326072849 conn(0x7f7244054b70 msgr2=0x7f7244054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:47.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f72527f1700 1 -- 172.21.15.81:0/1326072849 shutdown_connections 2024-05-08T23:31:47.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f72527f1700 1 -- 172.21.15.81:0/1326072849 wait complete. 2024-05-08T23:31:47.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f72527f1700 1 Processor -- start 2024-05-08T23:31:47.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f72527f1700 1 -- start start 2024-05-08T23:31:47.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f72527f1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7244033fb0 0x7f7244036550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f72527f1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7244047cc0 0x7f7244034e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f72527f1700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7244048210 0x7f72440dc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f72527f1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72440e8180 con 0x7f7244047cc0 2024-05-08T23:31:47.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f72527f1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72440e8460 con 0x7f7244033fb0 2024-05-08T23:31:47.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f72527f1700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72440e8740 con 0x7f7244048210 2024-05-08T23:31:47.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f7250fee700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7244047cc0 0x7f7244034e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f7250fee700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7244047cc0 0x7f7244034e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58288/0 (socket says 172.21.15.81:58288) 2024-05-08T23:31:47.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f7250fee700 1 -- 172.21.15.81:0/1598277889 learned_addr learned my addr 172.21.15.81:0/1598277889 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:47.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f72517ef700 1 --2- 172.21.15.81:0/1598277889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7244033fb0 0x7f7244036550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f7251ff0700 1 --2- 172.21.15.81:0/1598277889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7244048210 0x7f72440dc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f7250fee700 1 -- 172.21.15.81:0/1598277889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7244048210 msgr2=0x7f72440dc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f7250fee700 1 --2- 172.21.15.81:0/1598277889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7244048210 0x7f72440dc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f7250fee700 1 -- 172.21.15.81:0/1598277889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7244033fb0 msgr2=0x7f7244036550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f7250fee700 1 --2- 172.21.15.81:0/1598277889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7244033fb0 0x7f7244036550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f7250fee700 1 -- 172.21.15.81:0/1598277889 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7240009570 con 0x7f7244047cc0 2024-05-08T23:31:47.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f7250fee700 1 --2- 172.21.15.81:0/1598277889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7244047cc0 0x7f7244034e40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7240004000 tx=0x7f7240004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:47.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f723dffb700 1 -- 172.21.15.81:0/1598277889 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724000b870 con 0x7f7244047cc0 2024-05-08T23:31:47.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f723dffb700 1 -- 172.21.15.81:0/1598277889 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724000b9d0 con 0x7f7244047cc0 2024-05-08T23:31:47.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.448+0000 7f72527f1700 1 -- 172.21.15.81:0/1598277889 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72440e8a20 con 0x7f7244047cc0 2024-05-08T23:31:47.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.452+0000 7f723dffb700 1 -- 172.21.15.81:0/1598277889 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724000bd60 con 0x7f7244047cc0 2024-05-08T23:31:47.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.452+0000 7f72527f1700 1 -- 172.21.15.81:0/1598277889 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7244035380 con 0x7f7244047cc0 2024-05-08T23:31:47.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.452+0000 7f72527f1700 1 -- 172.21.15.81:0/1598277889 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72440e8f40 con 0x7f7244047cc0 2024-05-08T23:31:47.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.452+0000 7f723dffb700 1 -- 172.21.15.81:0/1598277889 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7240021070 con 0x7f7244047cc0 2024-05-08T23:31:47.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.452+0000 7f723dffb700 1 --2- 172.21.15.81:0/1598277889 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7230066760 0x7f7230068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.452+0000 7f723dffb700 1 -- 172.21.15.81:0/1598277889 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f7240085e70 con 0x7f7244047cc0 2024-05-08T23:31:47.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.452+0000 7f72517ef700 1 --2- 172.21.15.81:0/1598277889 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7230066760 0x7f7230068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.456+0000 7f72517ef700 1 --2- 172.21.15.81:0/1598277889 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7230066760 0x7f7230068c20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f724800a9a0 tx=0x7f7248008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:47.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.460+0000 7f723dffb700 1 -- 172.21.15.81:0/1598277889 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f72440e8f40 con 0x7f7244047cc0 2024-05-08T23:31:47.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.672+0000 7f72527f1700 1 -- 172.21.15.81:0/1598277889 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72440e8f40 con 0x7f7244047cc0 2024-05-08T23:31:47.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.672+0000 7f723dffb700 1 -- 172.21.15.81:0/1598277889 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f72440e8f40 con 0x7f7244047cc0 2024-05-08T23:31:47.674 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:47.674 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1243037934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1243037934},{"type":"v1","addr":"172.21.15.81:6835","nonce":1243037934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3124115182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3124115182},{"type":"v1","addr":"172.21.15.81:6837","nonce":3124115182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1423349691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":1423349691},{"type":"v1","addr":"172.21.15.117:6834","nonce":1423349691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:31:37.889298+0000","modified":"2024-05-08T23:31:47.307677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14852":{"gid":14852,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/3091184157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3091184157},{"type":"v1","addr":"172.21.15.117:6835","nonce":3091184157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:31:47.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.676+0000 7f72527f1700 1 -- 172.21.15.81:0/1598277889 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7230066760 msgr2=0x7f7230068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.676+0000 7f72527f1700 1 --2- 172.21.15.81:0/1598277889 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7230066760 0x7f7230068c20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f724800a9a0 tx=0x7f7248008040 comp rx=0 tx=0).stop 2024-05-08T23:31:47.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.676+0000 7f72527f1700 1 -- 172.21.15.81:0/1598277889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7244047cc0 msgr2=0x7f7244034e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.676+0000 7f72527f1700 1 --2- 172.21.15.81:0/1598277889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7244047cc0 0x7f7244034e40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7240004000 tx=0x7f7240004770 comp rx=0 tx=0).stop 2024-05-08T23:31:47.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.676+0000 7f72527f1700 1 -- 172.21.15.81:0/1598277889 shutdown_connections 2024-05-08T23:31:47.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.676+0000 7f72527f1700 1 --2- 172.21.15.81:0/1598277889 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7230066760 0x7f7230068c20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.676+0000 7f72527f1700 1 --2- 172.21.15.81:0/1598277889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7244048210 0x7f72440dc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.676+0000 7f72527f1700 1 --2- 172.21.15.81:0/1598277889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7244047cc0 0x7f7244034e40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.676+0000 7f72527f1700 1 --2- 172.21.15.81:0/1598277889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7244033fb0 0x7f7244036550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.676+0000 7f72527f1700 1 -- 172.21.15.81:0/1598277889 >> 172.21.15.81:0/1598277889 conn(0x7f7244054b70 msgr2=0x7f72440e5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:47.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.676+0000 7f72527f1700 1 -- 172.21.15.81:0/1598277889 shutdown_connections 2024-05-08T23:31:47.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.676+0000 7f72527f1700 1 -- 172.21.15.81:0/1598277889 wait complete. 2024-05-08T23:31:47.678 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 15 2024-05-08T23:31:47.689 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:47.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50b0c45700 1 Processor -- start 2024-05-08T23:31:47.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50b0c45700 1 -- start start 2024-05-08T23:31:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50b0c45700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f50ac071ab0 0x7f50ac0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50b0c45700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50ac15ac70 0x7f50ac13d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50b0c45700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ac0cea50 con 0x7f50ac071e90 2024-05-08T23:31:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50b0c45700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50ac0ce080 con 0x7f50ac071ab0 2024-05-08T23:31:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50b0c45700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac0cd7c0 con 0x7f50ac15ac70 2024-05-08T23:31:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50aaffd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f50ac071e90 0x7f50ac15a560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:48092/0 (socket says 172.21.15.81:48092) 2024-05-08T23:31:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50aaffd700 1 -- 172.21.15.81:0/904660975 learned_addr learned my addr 172.21.15.81:0/904660975 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50ab7fe700 1 --2- 172.21.15.81:0/904660975 >> v2:172.21.15.117:3300/0 conn(0x7f50ac071ab0 0x7f50ac0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50abfff700 1 --2- 172.21.15.81:0/904660975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50ac15ac70 0x7f50ac13d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50aa7fc700 1 -- 172.21.15.81:0/904660975 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355899103 0 0) 0x7f5094001110 con 0x7f50ac071e90 2024-05-08T23:31:47.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50aa7fc700 1 -- 172.21.15.81:0/904660975 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f508c0035d0 con 0x7f50ac071e90 2024-05-08T23:31:47.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50ab7fe700 1 -- 172.21.15.81:0/904660975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50ac15ac70 msgr2=0x7f50ac13d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:47.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50ab7fe700 1 --2- 172.21.15.81:0/904660975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50ac15ac70 0x7f50ac13d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50ab7fe700 1 -- 172.21.15.81:0/904660975 >> v1:172.21.15.81:6789/0 conn(0x7f50ac071e90 legacy=0x7f50ac15a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50aa7fc700 1 -- 172.21.15.81:0/904660975 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 586748852 0 0) 0x7f508c0035d0 con 0x7f50ac071e90 2024-05-08T23:31:47.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.804+0000 7f50ab7fe700 1 -- 172.21.15.81:0/904660975 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac0cda30 con 0x7f50ac071ab0 2024-05-08T23:31:47.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.808+0000 7f50ab7fe700 1 --2- 172.21.15.81:0/904660975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ac071ab0 0x7f50ac0ce510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f509c009880 tx=0x7f509c009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=481fb0ee10b43f60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:47.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.808+0000 7f50aa7fc700 1 -- 172.21.15.81:0/904660975 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c011830 con 0x7f50ac071ab0 2024-05-08T23:31:47.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.808+0000 7f50aa7fc700 1 -- 172.21.15.81:0/904660975 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c011990 con 0x7f50ac071ab0 2024-05-08T23:31:47.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.808+0000 7f50aa7fc700 1 -- 172.21.15.81:0/904660975 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c011d00 con 0x7f50ac071ab0 2024-05-08T23:31:47.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.808+0000 7f50b0c45700 1 -- 172.21.15.81:0/904660975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ac071ab0 msgr2=0x7f50ac0ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.808+0000 7f50b0c45700 1 --2- 172.21.15.81:0/904660975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ac071ab0 0x7f50ac0ce510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f509c009880 tx=0x7f509c009b90 comp rx=0 tx=0).stop 2024-05-08T23:31:47.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.808+0000 7f50b0c45700 1 -- 172.21.15.81:0/904660975 shutdown_connections 2024-05-08T23:31:47.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.808+0000 7f50b0c45700 1 --2- 172.21.15.81:0/904660975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50ac15ac70 0x7f50ac13d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.808+0000 7f50b0c45700 1 --2- 172.21.15.81:0/904660975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ac071ab0 0x7f50ac0ce510 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.808+0000 7f50b0c45700 1 -- 172.21.15.81:0/904660975 >> 172.21.15.81:0/904660975 conn(0x7f50ac06c330 msgr2=0x7f50ac06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:47.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.808+0000 7f50b0c45700 1 -- 172.21.15.81:0/904660975 shutdown_connections 2024-05-08T23:31:47.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.808+0000 7f50b0c45700 1 -- 172.21.15.81:0/904660975 wait complete. 2024-05-08T23:31:47.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50b0c45700 1 Processor -- start 2024-05-08T23:31:47.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50b0c45700 1 -- start start 2024-05-08T23:31:47.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50b0c45700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ac071ab0 0x7f50ac132810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50b0c45700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50ac071e90 0x7f50ac13e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50b0c45700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50ac15ac70 0x7f50ac13e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50b0c45700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac155f70 con 0x7f50ac15ac70 2024-05-08T23:31:47.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50b0c45700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac13efc0 con 0x7f50ac071ab0 2024-05-08T23:31:47.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50b0c45700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac13f2a0 con 0x7f50ac071e90 2024-05-08T23:31:47.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50abfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50ac15ac70 0x7f50ac13e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50abfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50ac15ac70 0x7f50ac13e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58294/0 (socket says 172.21.15.81:58294) 2024-05-08T23:31:47.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50abfff700 1 -- 172.21.15.81:0/2535081601 learned_addr learned my addr 172.21.15.81:0/2535081601 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:47.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50ab7fe700 1 --2- 172.21.15.81:0/2535081601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ac071ab0 0x7f50ac132810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50aaffd700 1 --2- 172.21.15.81:0/2535081601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50ac071e90 0x7f50ac13e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50abfff700 1 -- 172.21.15.81:0/2535081601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50ac071e90 msgr2=0x7f50ac13e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50abfff700 1 --2- 172.21.15.81:0/2535081601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50ac071e90 0x7f50ac13e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50abfff700 1 -- 172.21.15.81:0/2535081601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ac071ab0 msgr2=0x7f50ac132810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50abfff700 1 --2- 172.21.15.81:0/2535081601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ac071ab0 0x7f50ac132810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50abfff700 1 -- 172.21.15.81:0/2535081601 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c0094b0 con 0x7f50ac15ac70 2024-05-08T23:31:47.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50aaffd700 1 --2- 172.21.15.81:0/2535081601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50ac071e90 0x7f50ac13e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:47.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50abfff700 1 --2- 172.21.15.81:0/2535081601 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50ac15ac70 0x7f50ac13e990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f50a400efc0 tx=0x7f50a400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:47.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50a3fff700 1 -- 172.21.15.81:0/2535081601 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a400cb80 con 0x7f50ac15ac70 2024-05-08T23:31:47.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50b0c45700 1 -- 172.21.15.81:0/2535081601 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ac13f580 con 0x7f50ac15ac70 2024-05-08T23:31:47.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.812+0000 7f50a3fff700 1 -- 172.21.15.81:0/2535081601 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a400f040 con 0x7f50ac15ac70 2024-05-08T23:31:47.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.816+0000 7f50b0c45700 1 -- 172.21.15.81:0/2535081601 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac13fb40 con 0x7f50ac15ac70 2024-05-08T23:31:47.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.816+0000 7f50a3fff700 1 -- 172.21.15.81:0/2535081601 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a400cee0 con 0x7f50ac15ac70 2024-05-08T23:31:47.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.816+0000 7f50a1ffb700 1 -- 172.21.15.81:0/2535081601 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5088005310 con 0x7f50ac15ac70 2024-05-08T23:31:47.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.816+0000 7f50a3fff700 1 -- 172.21.15.81:0/2535081601 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f50a4014480 con 0x7f50ac15ac70 2024-05-08T23:31:47.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.816+0000 7f50a3fff700 1 --2- 172.21.15.81:0/2535081601 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5090066730 0x7f5090068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.816+0000 7f50a3fff700 1 -- 172.21.15.81:0/2535081601 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f50a4084bc0 con 0x7f50ac15ac70 2024-05-08T23:31:47.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.816+0000 7f50ab7fe700 1 --2- 172.21.15.81:0/2535081601 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5090066730 0x7f5090068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.816+0000 7f50ab7fe700 1 --2- 172.21.15.81:0/2535081601 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5090066730 0x7f5090068bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f509c0090b0 tx=0x7f509c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:47.823 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.820+0000 7f50a3fff700 1 -- 172.21.15.81:0/2535081601 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f50a4051c90 con 0x7f50ac15ac70 2024-05-08T23:31:47.990 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.988+0000 7f50a1ffb700 1 -- 172.21.15.81:0/2535081601 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5088005890 con 0x7f50ac15ac70 2024-05-08T23:31:47.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.992+0000 7f50a3fff700 1 -- 172.21.15.81:0/2535081601 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+17893 (secure 0 0 0) 0x7f50a407ce60 con 0x7f50ac15ac70 2024-05-08T23:31:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:47.994 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":38,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:47.307068+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:31:35.620423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:31:36.878524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-05-08T23:31:46.337484+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:47.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.996+0000 7f50a1ffb700 1 -- 172.21.15.81:0/2535081601 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5090066730 msgr2=0x7f5090068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.996+0000 7f50a1ffb700 1 --2- 172.21.15.81:0/2535081601 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5090066730 0x7f5090068bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f509c0090b0 tx=0x7f509c008030 comp rx=0 tx=0).stop 2024-05-08T23:31:47.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.996+0000 7f50a1ffb700 1 -- 172.21.15.81:0/2535081601 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50ac15ac70 msgr2=0x7f50ac13e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.996+0000 7f50a1ffb700 1 --2- 172.21.15.81:0/2535081601 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50ac15ac70 0x7f50ac13e990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f50a400efc0 tx=0x7f50a400c0c0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.996+0000 7f50a1ffb700 1 -- 172.21.15.81:0/2535081601 shutdown_connections 2024-05-08T23:31:47.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.996+0000 7f50a1ffb700 1 --2- 172.21.15.81:0/2535081601 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5090066730 0x7f5090068bf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.996+0000 7f50a1ffb700 1 --2- 172.21.15.81:0/2535081601 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50ac15ac70 0x7f50ac13e990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.996+0000 7f50a1ffb700 1 --2- 172.21.15.81:0/2535081601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50ac071e90 0x7f50ac13e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.996+0000 7f50a1ffb700 1 --2- 172.21.15.81:0/2535081601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50ac071ab0 0x7f50ac132810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.996+0000 7f50a1ffb700 1 -- 172.21.15.81:0/2535081601 >> 172.21.15.81:0/2535081601 conn(0x7f50ac06c330 msgr2=0x7f50ac06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:47.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.996+0000 7f50a1ffb700 1 -- 172.21.15.81:0/2535081601 shutdown_connections 2024-05-08T23:31:47.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:47.996+0000 7f50a1ffb700 1 -- 172.21.15.81:0/2535081601 wait complete. 2024-05-08T23:31:48.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:48.013 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch layoutfile) 2024-05-08T23:31:48.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:48.031 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir layoutdir) 2024-05-08T23:31:48.085 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rw' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2024-05-08T23:31:48.214 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.212+0000 7f27de833700 1 Processor -- start 2024-05-08T23:31:48.214 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.212+0000 7f27de833700 1 -- start start 2024-05-08T23:31:48.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.212+0000 7f27de833700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f27d8160920 0x7f27d80ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.212+0000 7f27de833700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d80d3d40 con 0x7f27d80d0050 2024-05-08T23:31:48.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.212+0000 7f27de833700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d80d0d00 con 0x7f27d80d1da0 2024-05-08T23:31:48.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.212+0000 7f27de833700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27d80d0fe0 con 0x7f27d8160920 2024-05-08T23:31:48.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.212+0000 7f27dd831700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f27d8160920 0x7f27d80ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.212+0000 7f27de032700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f27d80d0050 0x7f27d80b5e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:48100/0 (socket says 172.21.15.81:48100) 2024-05-08T23:31:48.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.212+0000 7f27de032700 1 -- 172.21.15.81:0/3540505219 learned_addr learned my addr 172.21.15.81:0/3540505219 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:48.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27dc82f700 1 -- 172.21.15.81:0/3540505219 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437469648 0 0) 0x7f27d80d3d40 con 0x7f27d80d0050 2024-05-08T23:31:48.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27dc82f700 1 -- 172.21.15.81:0/3540505219 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27c40035d0 con 0x7f27d80d0050 2024-05-08T23:31:48.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27dd831700 1 -- 172.21.15.81:0/3540505219 >> v1:172.21.15.117:6789/0 conn(0x7f27d80d1da0 legacy=0x7f27d80cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27dc82f700 1 -- 172.21.15.81:0/3540505219 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970263853 0 0) 0x7f27d80d0d00 con 0x7f27d80d1da0 2024-05-08T23:31:48.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27dd831700 1 -- 172.21.15.81:0/3540505219 >> v1:172.21.15.81:6789/0 conn(0x7f27d80d0050 legacy=0x7f27d80b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.218 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27dd831700 1 -- 172.21.15.81:0/3540505219 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d80d12c0 con 0x7f27d8160920 2024-05-08T23:31:48.218 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27dd831700 1 --2- 172.21.15.81:0/3540505219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27d8160920 0x7f27d80ce6f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009850 tx=0x7f27c8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be347267d1b71a5b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:48.218 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27dc82f700 1 -- 172.21.15.81:0/3540505219 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8011830 con 0x7f27d8160920 2024-05-08T23:31:48.218 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27dc82f700 1 -- 172.21.15.81:0/3540505219 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c8011990 con 0x7f27d8160920 2024-05-08T23:31:48.218 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27dc82f700 1 -- 172.21.15.81:0/3540505219 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8011d40 con 0x7f27d8160920 2024-05-08T23:31:48.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27de833700 1 -- 172.21.15.81:0/3540505219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27d8160920 msgr2=0x7f27d80ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27de833700 1 --2- 172.21.15.81:0/3540505219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27d8160920 0x7f27d80ce6f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009850 tx=0x7f27c8009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:48.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27de833700 1 -- 172.21.15.81:0/3540505219 shutdown_connections 2024-05-08T23:31:48.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27de833700 1 --2- 172.21.15.81:0/3540505219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27d8160920 0x7f27d80ce6f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27de833700 1 -- 172.21.15.81:0/3540505219 >> 172.21.15.81:0/3540505219 conn(0x7f27d809c340 msgr2=0x7f27d80d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:48.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27de833700 1 -- 172.21.15.81:0/3540505219 shutdown_connections 2024-05-08T23:31:48.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.216+0000 7f27de833700 1 -- 172.21.15.81:0/3540505219 wait complete. 2024-05-08T23:31:48.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27de833700 1 Processor -- start 2024-05-08T23:31:48.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27de833700 1 -- start start 2024-05-08T23:31:48.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27de833700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27d80d0050 0x7f27d814c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27de833700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27d80d1da0 0x7f27d81f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27de833700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27d8160920 0x7f27d815d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27de833700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d81593a0 con 0x7f27d80d1da0 2024-05-08T23:31:48.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27de833700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d815f910 con 0x7f27d8160920 2024-05-08T23:31:48.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27de833700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d815e0d0 con 0x7f27d80d0050 2024-05-08T23:31:48.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27dd030700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27d80d1da0 0x7f27d81f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27dd030700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27d80d1da0 0x7f27d81f2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58308/0 (socket says 172.21.15.81:58308) 2024-05-08T23:31:48.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27dd831700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27d80d0050 0x7f27d814c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27dd030700 1 -- 172.21.15.81:0/3058087880 learned_addr learned my addr 172.21.15.81:0/3058087880 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:48.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27de032700 1 --2- 172.21.15.81:0/3058087880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27d8160920 0x7f27d815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27dd030700 1 -- 172.21.15.81:0/3058087880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27d80d0050 msgr2=0x7f27d814c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27dd030700 1 --2- 172.21.15.81:0/3058087880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27d80d0050 0x7f27d814c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27dd030700 1 -- 172.21.15.81:0/3058087880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27d8160920 msgr2=0x7f27d815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27dd030700 1 --2- 172.21.15.81:0/3058087880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27d8160920 0x7f27d815d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27dd030700 1 -- 172.21.15.81:0/3058087880 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c80094b0 con 0x7f27d80d1da0 2024-05-08T23:31:48.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27dd030700 1 --2- 172.21.15.81:0/3058087880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27d80d1da0 0x7f27d81f2f80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f27d400c520 tx=0x7f27d400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:48.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.220+0000 7f27ce7fc700 1 -- 172.21.15.81:0/3058087880 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d40054e0 con 0x7f27d80d1da0 2024-05-08T23:31:48.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.224+0000 7f27ce7fc700 1 -- 172.21.15.81:0/3058087880 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d40036a0 con 0x7f27d80d1da0 2024-05-08T23:31:48.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.224+0000 7f27de833700 1 -- 172.21.15.81:0/3058087880 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d8160e80 con 0x7f27d80d1da0 2024-05-08T23:31:48.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.224+0000 7f27ce7fc700 1 -- 172.21.15.81:0/3058087880 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4003a50 con 0x7f27d80d1da0 2024-05-08T23:31:48.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.224+0000 7f27de833700 1 -- 172.21.15.81:0/3058087880 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d8158d50 con 0x7f27d80d1da0 2024-05-08T23:31:48.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.224+0000 7f27ce7fc700 1 -- 172.21.15.81:0/3058087880 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f27d4003bb0 con 0x7f27d80d1da0 2024-05-08T23:31:48.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.224+0000 7f27de833700 1 -- 172.21.15.81:0/3058087880 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d8159510 con 0x7f27d80d1da0 2024-05-08T23:31:48.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.224+0000 7f27ce7fc700 1 --2- 172.21.15.81:0/3058087880 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f27bc066730 0x7f27bc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.228+0000 7f27dd831700 1 --2- 172.21.15.81:0/3058087880 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f27bc066730 0x7f27bc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.228+0000 7f27ce7fc700 1 -- 172.21.15.81:0/3058087880 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f27d40836a0 con 0x7f27d80d1da0 2024-05-08T23:31:48.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.228+0000 7f27dd831700 1 --2- 172.21.15.81:0/3058087880 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f27bc066730 0x7f27bc068bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f27c80090b0 tx=0x7f27c801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:48.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.232+0000 7f27ce7fc700 1 -- 172.21.15.81:0/3058087880 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f27d4050880 con 0x7f27d80d1da0 2024-05-08T23:31:48.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.464+0000 7f27de833700 1 -- 172.21.15.81:0/3058087880 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7f27d80684b0 con 0x7f27d80d1da0 2024-05-08T23:31:48.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.472+0000 7f27ce7fc700 1 -- 172.21.15.81:0/3058087880 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v11) v1 ==== 210+0+0 (secure 0 0 0) 0x7f27d4081e60 con 0x7f27d80d1da0 2024-05-08T23:31:48.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.472+0000 7f27de833700 1 -- 172.21.15.81:0/3058087880 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f27bc066730 msgr2=0x7f27bc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.472+0000 7f27de833700 1 --2- 172.21.15.81:0/3058087880 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f27bc066730 0x7f27bc068bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f27c80090b0 tx=0x7f27c801c040 comp rx=0 tx=0).stop 2024-05-08T23:31:48.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.472+0000 7f27de833700 1 -- 172.21.15.81:0/3058087880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27d80d1da0 msgr2=0x7f27d81f2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.472+0000 7f27de833700 1 --2- 172.21.15.81:0/3058087880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27d80d1da0 0x7f27d81f2f80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f27d400c520 tx=0x7f27d400c830 comp rx=0 tx=0).stop 2024-05-08T23:31:48.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.472+0000 7f27de833700 1 -- 172.21.15.81:0/3058087880 shutdown_connections 2024-05-08T23:31:48.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.472+0000 7f27de833700 1 --2- 172.21.15.81:0/3058087880 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f27bc066730 0x7f27bc068bf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.472+0000 7f27de833700 1 --2- 172.21.15.81:0/3058087880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27d8160920 0x7f27d815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.472+0000 7f27de833700 1 --2- 172.21.15.81:0/3058087880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27d80d1da0 0x7f27d81f2f80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.472+0000 7f27de833700 1 --2- 172.21.15.81:0/3058087880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27d80d0050 0x7f27d814c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.472+0000 7f27de833700 1 -- 172.21.15.81:0/3058087880 >> 172.21.15.81:0/3058087880 conn(0x7f27d809c340 msgr2=0x7f27d80b7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:48.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.472+0000 7f27de833700 1 -- 172.21.15.81:0/3058087880 shutdown_connections 2024-05-08T23:31:48.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.472+0000 7f27de833700 1 -- 172.21.15.81:0/3058087880 wait complete. 2024-05-08T23:31:48.476 INFO:teuthology.orchestra.run.smithi081.stderr:updated caps for client.0 2024-05-08T23:31:48.488 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:31:48.488 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:31:48.496 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:48.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d909b700 1 Processor -- start 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d909b700 1 -- start start 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d909b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04d4159e80 0x7f04d4070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d909b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f04d4071180 0x7f04d40cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d909b700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d40cc950 con 0x7f04d4142bf0 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d909b700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d40ccbb0 con 0x7f04d4071180 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d909b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d40719e0 con 0x7f04d4159e80 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d889a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f04d4071180 0x7f04d40cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d889a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f04d4071180 0x7f04d40cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58314/0 (socket says 172.21.15.81:58314) 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d889a700 1 -- 172.21.15.81:0/1283101553 learned_addr learned my addr 172.21.15.81:0/1283101553 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d37fe700 1 --2- 172.21.15.81:0/1283101553 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04d4159e80 0x7f04d4070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d889a700 1 -- 172.21.15.81:0/1283101553 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04d4159e80 msgr2=0x7f04d4070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d889a700 1 --2- 172.21.15.81:0/1283101553 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04d4159e80 0x7f04d4070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d889a700 1 -- 172.21.15.81:0/1283101553 >> v1:172.21.15.117:6789/0 conn(0x7f04d4142bf0 legacy=0x7f04d4159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d889a700 1 -- 172.21.15.81:0/1283101553 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d40cce70 con 0x7f04d4071180 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d889a700 1 --2- 172.21.15.81:0/1283101553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d4071180 0x7f04d40cd670 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009e80 tx=0x7f04cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5c69ac28cc9db7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d2ffd700 1 -- 172.21.15.81:0/1283101553 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc00e070 con 0x7f04d4071180 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d2ffd700 1 -- 172.21.15.81:0/1283101553 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04cc00b8d0 con 0x7f04d4071180 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d2ffd700 1 -- 172.21.15.81:0/1283101553 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc00bc80 con 0x7f04d4071180 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d909b700 1 -- 172.21.15.81:0/1283101553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d4071180 msgr2=0x7f04d40cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d909b700 1 --2- 172.21.15.81:0/1283101553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d4071180 0x7f04d40cd670 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009e80 tx=0x7f04cc00b330 comp rx=0 tx=0).stop 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.604+0000 7f04d909b700 1 -- 172.21.15.81:0/1283101553 shutdown_connections 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d909b700 1 --2- 172.21.15.81:0/1283101553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d4071180 0x7f04d40cd670 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d909b700 1 --2- 172.21.15.81:0/1283101553 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04d4159e80 0x7f04d4070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d909b700 1 -- 172.21.15.81:0/1283101553 >> 172.21.15.81:0/1283101553 conn(0x7f04d406c330 msgr2=0x7f04d406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d909b700 1 -- 172.21.15.81:0/1283101553 shutdown_connections 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d909b700 1 -- 172.21.15.81:0/1283101553 wait complete. 2024-05-08T23:31:48.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d909b700 1 Processor -- start 2024-05-08T23:31:48.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d909b700 1 -- start start 2024-05-08T23:31:48.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d909b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d4071180 0x7f04d4158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d909b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04d4142bf0 0x7f04d40b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d909b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d4159e80 0x7f04d415ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d909b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d40b3c30 con 0x7f04d4071180 2024-05-08T23:31:48.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d909b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d40add70 con 0x7f04d4159e80 2024-05-08T23:31:48.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d909b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d40ae050 con 0x7f04d4142bf0 2024-05-08T23:31:48.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d4071180 0x7f04d4158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d4071180 0x7f04d4158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58324/0 (socket says 172.21.15.81:58324) 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d3fff700 1 -- 172.21.15.81:0/3252300867 learned_addr learned my addr 172.21.15.81:0/3252300867 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d37fe700 1 --2- 172.21.15.81:0/3252300867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04d4142bf0 0x7f04d40b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d889a700 1 --2- 172.21.15.81:0/3252300867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d4159e80 0x7f04d415ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d3fff700 1 -- 172.21.15.81:0/3252300867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04d4142bf0 msgr2=0x7f04d40b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d3fff700 1 --2- 172.21.15.81:0/3252300867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04d4142bf0 0x7f04d40b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d3fff700 1 -- 172.21.15.81:0/3252300867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d4159e80 msgr2=0x7f04d415ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d3fff700 1 --2- 172.21.15.81:0/3252300867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d4159e80 0x7f04d415ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.608+0000 7f04d3fff700 1 -- 172.21.15.81:0/3252300867 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc009b30 con 0x7f04d4071180 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.612+0000 7f04d37fe700 1 --2- 172.21.15.81:0/3252300867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04d4142bf0 0x7f04d40b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.612+0000 7f04d3fff700 1 --2- 172.21.15.81:0/3252300867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d4071180 0x7f04d4158270 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f04c800c500 tx=0x7f04c800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.612+0000 7f04d0ff9700 1 -- 172.21.15.81:0/3252300867 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c80052d0 con 0x7f04d4071180 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.612+0000 7f04d0ff9700 1 -- 172.21.15.81:0/3252300867 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c8005430 con 0x7f04d4071180 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.612+0000 7f04d909b700 1 -- 172.21.15.81:0/3252300867 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d40ae330 con 0x7f04d4071180 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.612+0000 7f04d0ff9700 1 -- 172.21.15.81:0/3252300867 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c80055a0 con 0x7f04d4071180 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.612+0000 7f04d909b700 1 -- 172.21.15.81:0/3252300867 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d40b20b0 con 0x7f04d4071180 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.612+0000 7f04d909b700 1 -- 172.21.15.81:0/3252300867 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d40684b0 con 0x7f04d4071180 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.612+0000 7f04d0ff9700 1 -- 172.21.15.81:0/3252300867 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f04c80038e0 con 0x7f04d4071180 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.612+0000 7f04d0ff9700 1 --2- 172.21.15.81:0/3252300867 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f04c0066730 0x7f04c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.616+0000 7f04d0ff9700 1 -- 172.21.15.81:0/3252300867 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f04c8083690 con 0x7f04d4071180 2024-05-08T23:31:48.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.616+0000 7f04d37fe700 1 --2- 172.21.15.81:0/3252300867 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f04c0066730 0x7f04c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.616+0000 7f04d37fe700 1 --2- 172.21.15.81:0/3252300867 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f04c0066730 0x7f04c0068bf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f04d41587b0 tx=0x7f04c4009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.620+0000 7f04d0ff9700 1 -- 172.21.15.81:0/3252300867 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f04c8050870 con 0x7f04d4071180 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.788+0000 7f04d909b700 1 -- 172.21.15.81:0/3252300867 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04d4134f70 con 0x7f04d4071180 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.792+0000 7f04d0ff9700 1 -- 172.21.15.81:0/3252300867 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+17880 (secure 0 0 0) 0x7f04c8053e90 con 0x7f04d4071180 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":39,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:48.310750+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:31:35.620423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:31:36.878524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-05-08T23:31:46.337484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:48.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.796+0000 7f04d909b700 1 -- 172.21.15.81:0/3252300867 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f04c0066730 msgr2=0x7f04c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.796+0000 7f04d909b700 1 --2- 172.21.15.81:0/3252300867 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f04c0066730 0x7f04c0068bf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f04d41587b0 tx=0x7f04c4009240 comp rx=0 tx=0).stop 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.796+0000 7f04d909b700 1 -- 172.21.15.81:0/3252300867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d4071180 msgr2=0x7f04d4158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.796+0000 7f04d909b700 1 --2- 172.21.15.81:0/3252300867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d4071180 0x7f04d4158270 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f04c800c500 tx=0x7f04c800c810 comp rx=0 tx=0).stop 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.796+0000 7f04d909b700 1 -- 172.21.15.81:0/3252300867 shutdown_connections 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.796+0000 7f04d909b700 1 --2- 172.21.15.81:0/3252300867 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f04c0066730 0x7f04c0068bf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.796+0000 7f04d909b700 1 --2- 172.21.15.81:0/3252300867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d4159e80 0x7f04d415ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.796+0000 7f04d909b700 1 --2- 172.21.15.81:0/3252300867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04d4142bf0 0x7f04d40b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.796+0000 7f04d909b700 1 --2- 172.21.15.81:0/3252300867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d4071180 0x7f04d4158270 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.796+0000 7f04d909b700 1 -- 172.21.15.81:0/3252300867 >> 172.21.15.81:0/3252300867 conn(0x7f04d406c330 msgr2=0x7f04d40aeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.796+0000 7f04d909b700 1 -- 172.21.15.81:0/3252300867 shutdown_connections 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.796+0000 7f04d909b700 1 -- 172.21.15.81:0/3252300867 wait complete. 2024-05-08T23:31:48.811 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:31:35.620423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:31:36.878524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-05-08T23:31:46.337484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:48.812 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.924+0000 7fc09127a700 1 Processor -- start 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.924+0000 7fc09127a700 1 -- start start 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc09127a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc08c15d640 0x7fc08c0ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc09127a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc08c0d2bd0 0x7fc08c0d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc09127a700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08c0d1890 con 0x7fc08c0b4bf0 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc09127a700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc08c0d3030 con 0x7fc08c0d2bd0 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc09127a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0d1b30 con 0x7fc08c15d640 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc08b7fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc08c0b4bf0 0x7fc08c06b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:48106/0 (socket says 172.21.15.81:48106) 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc08b7fe700 1 -- 172.21.15.81:0/2324503435 learned_addr learned my addr 172.21.15.81:0/2324503435 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc08a7fc700 1 --2- 172.21.15.81:0/2324503435 >> v2:172.21.15.117:3300/0 conn(0x7fc08c0d2bd0 0x7fc08c0d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc08affd700 1 --2- 172.21.15.81:0/2324503435 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc08c15d640 0x7fc08c0ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc089ffb700 1 -- 172.21.15.81:0/2324503435 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823172413 0 0) 0x7fc08c0d1890 con 0x7fc08c0b4bf0 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc089ffb700 1 -- 172.21.15.81:0/2324503435 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0740035d0 con 0x7fc08c0b4bf0 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc08a7fc700 1 -- 172.21.15.81:0/2324503435 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc08c15d640 msgr2=0x7fc08c0ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc08a7fc700 1 --2- 172.21.15.81:0/2324503435 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc08c15d640 0x7fc08c0ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc08a7fc700 1 -- 172.21.15.81:0/2324503435 >> v1:172.21.15.81:6789/0 conn(0x7fc08c0b4bf0 legacy=0x7fc08c06b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc08a7fc700 1 -- 172.21.15.81:0/2324503435 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c0d1e10 con 0x7fc08c0d2bd0 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc08a7fc700 1 --2- 172.21.15.81:0/2324503435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c0d2bd0 0x7fc08c0d1350 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009850 tx=0x7fc07c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0cd8910a890c8ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc089ffb700 1 -- 172.21.15.81:0/2324503435 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c011830 con 0x7fc08c0d2bd0 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc089ffb700 1 -- 172.21.15.81:0/2324503435 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07c011990 con 0x7fc08c0d2bd0 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc089ffb700 1 -- 172.21.15.81:0/2324503435 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c011d00 con 0x7fc08c0d2bd0 2024-05-08T23:31:48.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc09127a700 1 -- 172.21.15.81:0/2324503435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c0d2bd0 msgr2=0x7fc08c0d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc09127a700 1 --2- 172.21.15.81:0/2324503435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c0d2bd0 0x7fc08c0d1350 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009850 tx=0x7fc07c009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:48.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc09127a700 1 -- 172.21.15.81:0/2324503435 shutdown_connections 2024-05-08T23:31:48.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc09127a700 1 --2- 172.21.15.81:0/2324503435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c0d2bd0 0x7fc08c0d1350 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc09127a700 1 --2- 172.21.15.81:0/2324503435 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc08c15d640 0x7fc08c0ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc09127a700 1 -- 172.21.15.81:0/2324503435 >> 172.21.15.81:0/2324503435 conn(0x7fc08c073cf0 msgr2=0x7fc08c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:48.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc09127a700 1 -- 172.21.15.81:0/2324503435 shutdown_connections 2024-05-08T23:31:48.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.928+0000 7fc09127a700 1 -- 172.21.15.81:0/2324503435 wait complete. 2024-05-08T23:31:48.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc09127a700 1 Processor -- start 2024-05-08T23:31:48.934 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc09127a700 1 -- start start 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc09127a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc08c0b4bf0 0x7fc08c0ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc09127a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c0d2bd0 0x7fc08c0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc09127a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc08c15d640 0x7fc08c0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc09127a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0a9b60 con 0x7fc08c0b4bf0 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc09127a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0aa120 con 0x7fc08c0d2bd0 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc09127a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c152c80 con 0x7fc08c15d640 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc08affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc08c0b4bf0 0x7fc08c0ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc08affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc08c0b4bf0 0x7fc08c0ac730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58330/0 (socket says 172.21.15.81:58330) 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc08affd700 1 -- 172.21.15.81:0/3462939033 learned_addr learned my addr 172.21.15.81:0/3462939033 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc08a7fc700 1 --2- 172.21.15.81:0/3462939033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c0d2bd0 0x7fc08c0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc08affd700 1 -- 172.21.15.81:0/3462939033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc08c15d640 msgr2=0x7fc08c0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc08b7fe700 1 --2- 172.21.15.81:0/3462939033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc08c15d640 0x7fc08c0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc08affd700 1 --2- 172.21.15.81:0/3462939033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc08c15d640 0x7fc08c0cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc08affd700 1 -- 172.21.15.81:0/3462939033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c0d2bd0 msgr2=0x7fc08c0cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc08affd700 1 --2- 172.21.15.81:0/3462939033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c0d2bd0 0x7fc08c0cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc08affd700 1 -- 172.21.15.81:0/3462939033 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c0094b0 con 0x7fc08c0b4bf0 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc08a7fc700 1 --2- 172.21.15.81:0/3462939033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c0d2bd0 0x7fc08c0cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc08affd700 1 --2- 172.21.15.81:0/3462939033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc08c0b4bf0 0x7fc08c0ac730 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc08000e870 tx=0x7fc08000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.932+0000 7fc0737fe700 1 -- 172.21.15.81:0/3462939033 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0800095a0 con 0x7fc08c0b4bf0 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.936+0000 7fc09127a700 1 -- 172.21.15.81:0/3462939033 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08c0cbbc0 con 0x7fc08c0b4bf0 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.936+0000 7fc0737fe700 1 -- 172.21.15.81:0/3462939033 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08000ee70 con 0x7fc08c0b4bf0 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.936+0000 7fc09127a700 1 -- 172.21.15.81:0/3462939033 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c0a9db0 con 0x7fc08c0b4bf0 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.936+0000 7fc0737fe700 1 -- 172.21.15.81:0/3462939033 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080005c40 con 0x7fc08c0b4bf0 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.936+0000 7fc09127a700 1 -- 172.21.15.81:0/3462939033 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08c0684b0 con 0x7fc08c0b4bf0 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.936+0000 7fc0737fe700 1 -- 172.21.15.81:0/3462939033 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc08000c850 con 0x7fc08c0b4bf0 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.936+0000 7fc0737fe700 1 --2- 172.21.15.81:0/3462939033 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc078066730 0x7fc078068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.936+0000 7fc0737fe700 1 -- 172.21.15.81:0/3462939033 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fc080084270 con 0x7fc08c0b4bf0 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.936+0000 7fc08a7fc700 1 --2- 172.21.15.81:0/3462939033 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc078066730 0x7fc078068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.936+0000 7fc08a7fc700 1 --2- 172.21.15.81:0/3462939033 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc078066730 0x7fc078068bf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0cd390 tx=0x7fc07c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:48.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:48.944+0000 7fc0737fe700 1 -- 172.21.15.81:0/3462939033 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc080051460 con 0x7fc08c0b4bf0 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.112+0000 7fc09127a700 1 -- 172.21.15.81:0/3462939033 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc08c0b39b0 con 0x7fc08c0b4bf0 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.112+0000 7fc0737fe700 1 -- 172.21.15.81:0/3462939033 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc080054a80 con 0x7fc08c0b4bf0 2024-05-08T23:31:49.114 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.116+0000 7fc09127a700 1 -- 172.21.15.81:0/3462939033 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc078066730 msgr2=0x7fc078068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.116+0000 7fc09127a700 1 --2- 172.21.15.81:0/3462939033 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc078066730 0x7fc078068bf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0cd390 tx=0x7fc07c008030 comp rx=0 tx=0).stop 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.116+0000 7fc09127a700 1 -- 172.21.15.81:0/3462939033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc08c0b4bf0 msgr2=0x7fc08c0ac730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.116+0000 7fc09127a700 1 --2- 172.21.15.81:0/3462939033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc08c0b4bf0 0x7fc08c0ac730 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc08000e870 tx=0x7fc08000ec30 comp rx=0 tx=0).stop 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.116+0000 7fc09127a700 1 -- 172.21.15.81:0/3462939033 shutdown_connections 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.116+0000 7fc09127a700 1 --2- 172.21.15.81:0/3462939033 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc078066730 0x7fc078068bf0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.116+0000 7fc09127a700 1 --2- 172.21.15.81:0/3462939033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc08c15d640 0x7fc08c0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.116+0000 7fc09127a700 1 --2- 172.21.15.81:0/3462939033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c0d2bd0 0x7fc08c0cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.116+0000 7fc09127a700 1 --2- 172.21.15.81:0/3462939033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc08c0b4bf0 0x7fc08c0ac730 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.116+0000 7fc09127a700 1 -- 172.21.15.81:0/3462939033 >> 172.21.15.81:0/3462939033 conn(0x7fc08c073cf0 msgr2=0x7fc08c0d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.116+0000 7fc09127a700 1 -- 172.21.15.81:0/3462939033 shutdown_connections 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.116+0000 7fc09127a700 1 -- 172.21.15.81:0/3462939033 wait complete. 2024-05-08T23:31:49.133 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:49.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f611c881700 1 Processor -- start 2024-05-08T23:31:49.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f611c881700 1 -- start start 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f611c881700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f61180b45f0 0x7f611815a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f611c881700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f61180ce520 0x7f61180b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f611c881700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f611806db00 con 0x7f611815a190 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f611c881700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611815aab0 con 0x7f61180ce520 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f611c881700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61180cd760 con 0x7f61180b45f0 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f6117fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f61180ce520 0x7f61180b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f61177fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f611815a190 0x7f61180b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:48116/0 (socket says 172.21.15.81:48116) 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f6117fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f61180ce520 0x7f61180b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58340/0 (socket says 172.21.15.81:58340) 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f61177fe700 1 -- 172.21.15.81:0/2833280761 learned_addr learned my addr 172.21.15.81:0/2833280761 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f6116ffd700 1 --2- 172.21.15.81:0/2833280761 >> v2:172.21.15.117:3300/0 conn(0x7f61180b45f0 0x7f611815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f6117fff700 1 -- 172.21.15.81:0/2833280761 >> v2:172.21.15.117:3300/0 conn(0x7f61180b45f0 msgr2=0x7f611815a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f6117fff700 1 --2- 172.21.15.81:0/2833280761 >> v2:172.21.15.117:3300/0 conn(0x7f61180b45f0 0x7f611815a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f6117fff700 1 -- 172.21.15.81:0/2833280761 >> v1:172.21.15.81:6789/0 conn(0x7f611815a190 legacy=0x7f61180b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.248+0000 7f6117fff700 1 -- 172.21.15.81:0/2833280761 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61180cda40 con 0x7f61180ce520 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.252+0000 7f6117fff700 1 --2- 172.21.15.81:0/2833280761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61180ce520 0x7f61180b25a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f61180cee70 tx=0x7f611000d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bd3190270f63a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:49.253 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.252+0000 7f61167fc700 1 -- 172.21.15.81:0/2833280761 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110010070 con 0x7f61180ce520 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.252+0000 7f61167fc700 1 -- 172.21.15.81:0/2833280761 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611000b2e0 con 0x7f61180ce520 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.252+0000 7f61167fc700 1 -- 172.21.15.81:0/2833280761 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611000dc10 con 0x7f61180ce520 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.252+0000 7f611c881700 1 -- 172.21.15.81:0/2833280761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61180ce520 msgr2=0x7f61180b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.252+0000 7f611c881700 1 --2- 172.21.15.81:0/2833280761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61180ce520 0x7f61180b25a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f61180cee70 tx=0x7f611000d330 comp rx=0 tx=0).stop 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.252+0000 7f611c881700 1 -- 172.21.15.81:0/2833280761 shutdown_connections 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.252+0000 7f611c881700 1 --2- 172.21.15.81:0/2833280761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61180ce520 0x7f61180b25a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.252+0000 7f611c881700 1 --2- 172.21.15.81:0/2833280761 >> v2:172.21.15.117:3300/0 conn(0x7f61180b45f0 0x7f611815a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.252+0000 7f611c881700 1 -- 172.21.15.81:0/2833280761 >> 172.21.15.81:0/2833280761 conn(0x7f61180b16c0 msgr2=0x7f61180b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.252+0000 7f611c881700 1 -- 172.21.15.81:0/2833280761 shutdown_connections 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.252+0000 7f611c881700 1 -- 172.21.15.81:0/2833280761 wait complete. 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.252+0000 7f611c881700 1 Processor -- start 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.252+0000 7f611c881700 1 -- start start 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f611c881700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61180b45f0 0x7f6118150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f611c881700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61180ce520 0x7f61181f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f611c881700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f611815a190 0x7f611815c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f611c881700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61181f65d0 con 0x7f611815a190 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f611c881700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61180ad6c0 con 0x7f61180b45f0 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f611c881700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61180ad9a0 con 0x7f61180ce520 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f6117fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f611815a190 0x7f611815c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f6117fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f611815a190 0x7f611815c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58350/0 (socket says 172.21.15.81:58350) 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f6117fff700 1 -- 172.21.15.81:0/3759732770 learned_addr learned my addr 172.21.15.81:0/3759732770 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f61177fe700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61180b45f0 0x7f6118150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f6117fff700 1 -- 172.21.15.81:0/3759732770 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61180ce520 msgr2=0x7f61181f6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f6116ffd700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61180ce520 0x7f61181f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f6117fff700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61180ce520 0x7f61181f6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f6117fff700 1 -- 172.21.15.81:0/3759732770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61180b45f0 msgr2=0x7f6118150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f6117fff700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61180b45f0 0x7f6118150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f6117fff700 1 -- 172.21.15.81:0/3759732770 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611000bc80 con 0x7f611815a190 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f61177fe700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61180b45f0 0x7f6118150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f6116ffd700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61180ce520 0x7f61181f6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f6117fff700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f611815a190 0x7f611815c4e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f611806cb40 tx=0x7f6110004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f60fffff700 1 -- 172.21.15.81:0/3759732770 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110004450 con 0x7f611815a190 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f60fffff700 1 -- 172.21.15.81:0/3759732770 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61100045b0 con 0x7f611815a190 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f611c881700 1 -- 172.21.15.81:0/3759732770 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61180adc80 con 0x7f611815a190 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f60fffff700 1 -- 172.21.15.81:0/3759732770 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110004960 con 0x7f611815a190 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f611c881700 1 -- 172.21.15.81:0/3759732770 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61181f6fa0 con 0x7f611815a190 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.256+0000 7f611c881700 1 -- 172.21.15.81:0/3759732770 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61181f6740 con 0x7f611815a190 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.260+0000 7f60fffff700 1 -- 172.21.15.81:0/3759732770 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6110026450 con 0x7f611815a190 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.260+0000 7f60fffff700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6104066730 0x7f6104068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.260+0000 7f61177fe700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6104066730 0x7f6104068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.260+0000 7f60fffff700 1 -- 172.21.15.81:0/3759732770 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f6110086080 con 0x7f611815a190 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.260+0000 7f61177fe700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6104066730 0x7f6104068bf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f610c005930 tx=0x7f610c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:49.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.264+0000 7f60fffff700 1 -- 172.21.15.81:0/3759732770 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f61100b7cf0 con 0x7f611815a190 2024-05-08T23:31:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.432+0000 7f611c881700 1 -- 172.21.15.81:0/3759732770 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f61180684b0 con 0x7f611815a190 2024-05-08T23:31:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.436+0000 7f60fffff700 1 -- 172.21.15.81:0/3759732770 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6110056d80 con 0x7f611815a190 2024-05-08T23:31:49.437 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.436+0000 7f611c881700 1 -- 172.21.15.81:0/3759732770 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6104066730 msgr2=0x7f6104068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.436+0000 7f611c881700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6104066730 0x7f6104068bf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f610c005930 tx=0x7f610c009040 comp rx=0 tx=0).stop 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.436+0000 7f611c881700 1 -- 172.21.15.81:0/3759732770 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f611815a190 msgr2=0x7f611815c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.436+0000 7f611c881700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f611815a190 0x7f611815c4e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f611806cb40 tx=0x7f6110004af0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.440+0000 7f611c881700 1 -- 172.21.15.81:0/3759732770 shutdown_connections 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.440+0000 7f611c881700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6104066730 0x7f6104068bf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.440+0000 7f611c881700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f611815a190 0x7f611815c4e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.440+0000 7f611c881700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61180ce520 0x7f61181f6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.440+0000 7f611c881700 1 --2- 172.21.15.81:0/3759732770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61180b45f0 0x7f6118150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.440+0000 7f611c881700 1 -- 172.21.15.81:0/3759732770 >> 172.21.15.81:0/3759732770 conn(0x7f61180b16c0 msgr2=0x7f61181521d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.440+0000 7f611c881700 1 -- 172.21.15.81:0/3759732770 shutdown_connections 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.440+0000 7f611c881700 1 -- 172.21.15.81:0/3759732770 wait complete. 2024-05-08T23:31:49.458 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:49.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d25809700 1 Processor -- start 2024-05-08T23:31:49.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d25809700 1 -- start start 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d25809700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6d20160a30 0x7f6d200d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d25809700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d200d3c50 0x7f6d200d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d25809700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d200b66d0 con 0x7f6d200d1cc0 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d25809700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d200d2170 con 0x7f6d20160a30 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d25809700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d200d11c0 con 0x7f6d200d3c50 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d1effd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6d20160a30 0x7f6d200d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d1effd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6d20160a30 0x7f6d200d0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58356/0 (socket says 172.21.15.81:58356) 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d1effd700 1 -- 172.21.15.81:0/1593405494 learned_addr learned my addr 172.21.15.81:0/1593405494 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d1e7fc700 1 --2- 172.21.15.81:0/1593405494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d200d3c50 0x7f6d200d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d1effd700 1 -- 172.21.15.81:0/1593405494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d200d3c50 msgr2=0x7f6d200d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d1effd700 1 --2- 172.21.15.81:0/1593405494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d200d3c50 0x7f6d200d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d1effd700 1 -- 172.21.15.81:0/1593405494 >> v1:172.21.15.117:6789/0 conn(0x7f6d200d1cc0 legacy=0x7f6d200cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d1effd700 1 -- 172.21.15.81:0/1593405494 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d200d14a0 con 0x7f6d20160a30 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d1e7fc700 1 --2- 172.21.15.81:0/1593405494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d200d3c50 0x7f6d200d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d1effd700 1 --2- 172.21.15.81:0/1593405494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d20160a30 0x7f6d200d0480 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009910 tx=0x7f6d08009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0dfb7339b8905b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d1dffb700 1 -- 172.21.15.81:0/1593405494 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0800ba60 con 0x7f6d20160a30 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d1dffb700 1 -- 172.21.15.81:0/1593405494 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0800bbc0 con 0x7f6d20160a30 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.580+0000 7f6d1dffb700 1 -- 172.21.15.81:0/1593405494 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d08009e60 con 0x7f6d20160a30 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.584+0000 7f6d25809700 1 -- 172.21.15.81:0/1593405494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d20160a30 msgr2=0x7f6d200d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.584+0000 7f6d25809700 1 --2- 172.21.15.81:0/1593405494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d20160a30 0x7f6d200d0480 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009910 tx=0x7f6d08009c20 comp rx=0 tx=0).stop 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.584+0000 7f6d25809700 1 -- 172.21.15.81:0/1593405494 shutdown_connections 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.584+0000 7f6d25809700 1 --2- 172.21.15.81:0/1593405494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d200d3c50 0x7f6d200d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.584+0000 7f6d25809700 1 --2- 172.21.15.81:0/1593405494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d20160a30 0x7f6d200d0480 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.584+0000 7f6d25809700 1 -- 172.21.15.81:0/1593405494 >> 172.21.15.81:0/1593405494 conn(0x7f6d2009c500 msgr2=0x7f6d200d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.584+0000 7f6d25809700 1 -- 172.21.15.81:0/1593405494 shutdown_connections 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.584+0000 7f6d25809700 1 -- 172.21.15.81:0/1593405494 wait complete. 2024-05-08T23:31:49.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.584+0000 7f6d25809700 1 Processor -- start 2024-05-08T23:31:49.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.584+0000 7f6d25809700 1 -- start start 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d25809700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d200d1cc0 0x7f6d2015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d25809700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d200d3c50 0x7f6d200c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d25809700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d20160a30 0x7f6d200c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d25809700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d200a0a20 con 0x7f6d200d3c50 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d25809700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2015cd10 con 0x7f6d20160a30 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d25809700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d201f6fe0 con 0x7f6d200d1cc0 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d1e7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d200d3c50 0x7f6d200c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d1e7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d200d3c50 0x7f6d200c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58364/0 (socket says 172.21.15.81:58364) 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d1e7fc700 1 -- 172.21.15.81:0/1987712506 learned_addr learned my addr 172.21.15.81:0/1987712506 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d1effd700 1 --2- 172.21.15.81:0/1987712506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d200d1cc0 0x7f6d2015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d1f7fe700 1 --2- 172.21.15.81:0/1987712506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d20160a30 0x7f6d200c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d1e7fc700 1 -- 172.21.15.81:0/1987712506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d200d1cc0 msgr2=0x7f6d2015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d1e7fc700 1 --2- 172.21.15.81:0/1987712506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d200d1cc0 0x7f6d2015c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d1e7fc700 1 -- 172.21.15.81:0/1987712506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d20160a30 msgr2=0x7f6d200c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d1e7fc700 1 --2- 172.21.15.81:0/1987712506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d20160a30 0x7f6d200c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d1e7fc700 1 -- 172.21.15.81:0/1987712506 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d08009570 con 0x7f6d200d3c50 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d1e7fc700 1 --2- 172.21.15.81:0/1987712506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d200d3c50 0x7f6d200c8d60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000e790 tx=0x7f6d1000eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d17fff700 1 -- 172.21.15.81:0/1987712506 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1000c070 con 0x7f6d200d3c50 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d17fff700 1 -- 172.21.15.81:0/1987712506 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d10004230 con 0x7f6d200d3c50 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d25809700 1 -- 172.21.15.81:0/1987712506 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d200a0360 con 0x7f6d200d3c50 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d17fff700 1 -- 172.21.15.81:0/1987712506 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d100045c0 con 0x7f6d200d3c50 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d25809700 1 -- 172.21.15.81:0/1987712506 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d200a0e40 con 0x7f6d200d3c50 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.588+0000 7f6d25809700 1 -- 172.21.15.81:0/1987712506 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d200684b0 con 0x7f6d200d3c50 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.592+0000 7f6d17fff700 1 -- 172.21.15.81:0/1987712506 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6d100036c0 con 0x7f6d200d3c50 2024-05-08T23:31:49.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.592+0000 7f6d17fff700 1 --2- 172.21.15.81:0/1987712506 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6d00066730 0x7f6d00068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.592+0000 7f6d1effd700 1 --2- 172.21.15.81:0/1987712506 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6d00066730 0x7f6d00068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.592+0000 7f6d17fff700 1 -- 172.21.15.81:0/1987712506 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f6d10083f70 con 0x7f6d200d3c50 2024-05-08T23:31:49.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.592+0000 7f6d1effd700 1 --2- 172.21.15.81:0/1987712506 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6d00066730 0x7f6d00068bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6d08000c00 tx=0x7f6d08008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:49.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.600+0000 7f6d17fff700 1 -- 172.21.15.81:0/1987712506 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6d10051150 con 0x7f6d200d3c50 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.764+0000 7f6d25809700 1 -- 172.21.15.81:0/1987712506 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6d2015d9e0 con 0x7f6d200d3c50 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.764+0000 7f6d17fff700 1 -- 172.21.15.81:0/1987712506 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6d100094e0 con 0x7f6d200d3c50 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.768+0000 7f6d25809700 1 -- 172.21.15.81:0/1987712506 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6d00066730 msgr2=0x7f6d00068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.768+0000 7f6d25809700 1 --2- 172.21.15.81:0/1987712506 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6d00066730 0x7f6d00068bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6d08000c00 tx=0x7f6d08008030 comp rx=0 tx=0).stop 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.768+0000 7f6d25809700 1 -- 172.21.15.81:0/1987712506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d200d3c50 msgr2=0x7f6d200c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.768+0000 7f6d25809700 1 --2- 172.21.15.81:0/1987712506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d200d3c50 0x7f6d200c8d60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000e790 tx=0x7f6d1000eaa0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.768+0000 7f6d25809700 1 -- 172.21.15.81:0/1987712506 shutdown_connections 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.768+0000 7f6d25809700 1 --2- 172.21.15.81:0/1987712506 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6d00066730 0x7f6d00068bf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.768+0000 7f6d25809700 1 --2- 172.21.15.81:0/1987712506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d20160a30 0x7f6d200c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.768+0000 7f6d25809700 1 --2- 172.21.15.81:0/1987712506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d200d3c50 0x7f6d200c8d60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.768+0000 7f6d25809700 1 --2- 172.21.15.81:0/1987712506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d200d1cc0 0x7f6d2015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.768+0000 7f6d25809700 1 -- 172.21.15.81:0/1987712506 >> 172.21.15.81:0/1987712506 conn(0x7f6d2009c500 msgr2=0x7f6d200cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.768+0000 7f6d25809700 1 -- 172.21.15.81:0/1987712506 shutdown_connections 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.768+0000 7f6d25809700 1 -- 172.21.15.81:0/1987712506 wait complete. 2024-05-08T23:31:49.782 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.900+0000 7f036268f700 1 Processor -- start 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.900+0000 7f036268f700 1 -- start start 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.900+0000 7f036268f700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f035c157b30 0x7f035c15e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.900+0000 7f036268f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f035c0abda0 0x7f035c0ad820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.900+0000 7f036268f700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035c15a350 con 0x7f035c0add60 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.900+0000 7f036268f700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f035c15e980 con 0x7f035c157b30 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.900+0000 7f036268f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035c15b370 con 0x7f035c0abda0 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.900+0000 7f036168d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f035c157b30 0x7f035c15e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.900+0000 7f0360e8c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f035c0abda0 0x7f035c0ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.900+0000 7f0361e8e700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f035c0add60 0x7f035c0b1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:60362/0 (socket says 172.21.15.81:60362) 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.900+0000 7f0361e8e700 1 -- 172.21.15.81:0/795899154 learned_addr learned my addr 172.21.15.81:0/795899154 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f036168d700 1 -- 172.21.15.81:0/795899154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f035c0abda0 msgr2=0x7f035c0ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f036168d700 1 --2- 172.21.15.81:0/795899154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f035c0abda0 0x7f035c0ad820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f036168d700 1 -- 172.21.15.81:0/795899154 >> v1:172.21.15.117:6789/0 conn(0x7f035c0add60 legacy=0x7f035c0b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f036168d700 1 -- 172.21.15.81:0/795899154 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c15a5b0 con 0x7f035c157b30 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f036168d700 1 --2- 172.21.15.81:0/795899154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f035c157b30 0x7f035c15e350 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f034c009850 tx=0x7f034c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a42b36a11174e320 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f035bfff700 1 -- 172.21.15.81:0/795899154 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c004230 con 0x7f035c157b30 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f035bfff700 1 -- 172.21.15.81:0/795899154 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c004390 con 0x7f035c157b30 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f035bfff700 1 -- 172.21.15.81:0/795899154 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c0036c0 con 0x7f035c157b30 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f036268f700 1 -- 172.21.15.81:0/795899154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f035c157b30 msgr2=0x7f035c15e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f036268f700 1 --2- 172.21.15.81:0/795899154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f035c157b30 0x7f035c15e350 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f034c009850 tx=0x7f034c009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f036268f700 1 -- 172.21.15.81:0/795899154 shutdown_connections 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f036268f700 1 --2- 172.21.15.81:0/795899154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f035c0abda0 0x7f035c0ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f036268f700 1 --2- 172.21.15.81:0/795899154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f035c157b30 0x7f035c15e350 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f036268f700 1 -- 172.21.15.81:0/795899154 >> 172.21.15.81:0/795899154 conn(0x7f035c0b0de0 msgr2=0x7f035c0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f036268f700 1 -- 172.21.15.81:0/795899154 shutdown_connections 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f036268f700 1 -- 172.21.15.81:0/795899154 wait complete. 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.904+0000 7f036268f700 1 Processor -- start 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f036268f700 1 -- start start 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f036268f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f035c0abda0 0x7f035c147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f036268f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f035c0add60 0x7f035c158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f036268f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f035c157b30 0x7f035c06a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f036268f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c1589c0 con 0x7f035c157b30 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f036268f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c1580d0 con 0x7f035c0abda0 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f036268f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035c158370 con 0x7f035c0add60 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f0361e8e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f035c157b30 0x7f035c06a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f0361e8e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f035c157b30 0x7f035c06a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58378/0 (socket says 172.21.15.81:58378) 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f0361e8e700 1 -- 172.21.15.81:0/2441412210 learned_addr learned my addr 172.21.15.81:0/2441412210 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f036168d700 1 --2- 172.21.15.81:0/2441412210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f035c0abda0 0x7f035c147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f0360e8c700 1 --2- 172.21.15.81:0/2441412210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f035c0add60 0x7f035c158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f0361e8e700 1 -- 172.21.15.81:0/2441412210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f035c0add60 msgr2=0x7f035c158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f0361e8e700 1 --2- 172.21.15.81:0/2441412210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f035c0add60 0x7f035c158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f0361e8e700 1 -- 172.21.15.81:0/2441412210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f035c0abda0 msgr2=0x7f035c147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f0361e8e700 1 --2- 172.21.15.81:0/2441412210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f035c0abda0 0x7f035c147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f0361e8e700 1 -- 172.21.15.81:0/2441412210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c0094b0 con 0x7f035c157b30 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f0361e8e700 1 --2- 172.21.15.81:0/2441412210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f035c157b30 0x7f035c06a360 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0354002060 tx=0x7f035400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f0359ffb700 1 -- 172.21.15.81:0/2441412210 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035400ac90 con 0x7f035c157b30 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f0359ffb700 1 -- 172.21.15.81:0/2441412210 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035400adf0 con 0x7f035c157b30 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f036268f700 1 -- 172.21.15.81:0/2441412210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035c158650 con 0x7f035c157b30 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f0359ffb700 1 -- 172.21.15.81:0/2441412210 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03540072f0 con 0x7f035c157b30 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.908+0000 7f036268f700 1 -- 172.21.15.81:0/2441412210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035c159390 con 0x7f035c157b30 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.912+0000 7f036268f700 1 -- 172.21.15.81:0/2441412210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f035c158b30 con 0x7f035c157b30 2024-05-08T23:31:49.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.912+0000 7f0359ffb700 1 -- 172.21.15.81:0/2441412210 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0354007450 con 0x7f035c157b30 2024-05-08T23:31:49.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.912+0000 7f0359ffb700 1 --2- 172.21.15.81:0/2441412210 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0348066730 0x7f0348068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.912+0000 7f036168d700 1 --2- 172.21.15.81:0/2441412210 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0348066730 0x7f0348068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.912+0000 7f0359ffb700 1 -- 172.21.15.81:0/2441412210 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f0354084190 con 0x7f035c157b30 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.912+0000 7f036168d700 1 --2- 172.21.15.81:0/2441412210 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0348066730 0x7f0348068bf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f034c009850 tx=0x7f034c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:49.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:49.924+0000 7f0359ffb700 1 -- 172.21.15.81:0/2441412210 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0354051400 con 0x7f035c157b30 2024-05-08T23:31:50.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.092+0000 7f036268f700 1 -- 172.21.15.81:0/2441412210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f035c0684b0 con 0x7f035c157b30 2024-05-08T23:31:50.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.092+0000 7f0359ffb700 1 -- 172.21.15.81:0/2441412210 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v39) v1 ==== 97+0+11 (secure 0 0 0) 0x7f0354054a20 con 0x7f035c157b30 2024-05-08T23:31:50.094 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:50.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.092+0000 7f036268f700 1 -- 172.21.15.81:0/2441412210 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0348066730 msgr2=0x7f0348068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.092+0000 7f036268f700 1 --2- 172.21.15.81:0/2441412210 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0348066730 0x7f0348068bf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f034c009850 tx=0x7f034c008030 comp rx=0 tx=0).stop 2024-05-08T23:31:50.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.092+0000 7f036268f700 1 -- 172.21.15.81:0/2441412210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f035c157b30 msgr2=0x7f035c06a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.092+0000 7f036268f700 1 --2- 172.21.15.81:0/2441412210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f035c157b30 0x7f035c06a360 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0354002060 tx=0x7f035400a340 comp rx=0 tx=0).stop 2024-05-08T23:31:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.096+0000 7f036268f700 1 -- 172.21.15.81:0/2441412210 shutdown_connections 2024-05-08T23:31:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.096+0000 7f036268f700 1 --2- 172.21.15.81:0/2441412210 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0348066730 0x7f0348068bf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.096+0000 7f036268f700 1 --2- 172.21.15.81:0/2441412210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f035c157b30 0x7f035c06a360 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.096+0000 7f036268f700 1 --2- 172.21.15.81:0/2441412210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f035c0add60 0x7f035c158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.096+0000 7f036268f700 1 --2- 172.21.15.81:0/2441412210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f035c0abda0 0x7f035c147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.096+0000 7f036268f700 1 -- 172.21.15.81:0/2441412210 >> 172.21.15.81:0/2441412210 conn(0x7f035c0b0de0 msgr2=0x7f035c0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.096+0000 7f036268f700 1 -- 172.21.15.81:0/2441412210 shutdown_connections 2024-05-08T23:31:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.096+0000 7f036268f700 1 -- 172.21.15.81:0/2441412210 wait complete. 2024-05-08T23:31:50.108 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:31:50.218 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa4516b9700 1 Processor -- start 2024-05-08T23:31:50.218 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa4516b9700 1 -- start start 2024-05-08T23:31:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa4516b9700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa44c0cf980 0x7fa44c15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa4516b9700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa44c0cddf0 0x7fa44c0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa4516b9700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c160de0 0x7fa44c161230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa4516b9700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44c15e830 con 0x7fa44c0cf980 2024-05-08T23:31:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa4516b9700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44c0d02a0 con 0x7fa44c0cddf0 2024-05-08T23:31:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa4516b9700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0ce2d0 con 0x7fa44c160de0 2024-05-08T23:31:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa44bfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa44c0cf980 0x7fa44c15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa44bfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa44c0cf980 0x7fa44c15e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58380/0 (socket says 172.21.15.81:58380) 2024-05-08T23:31:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa44bfff700 1 -- 172.21.15.81:0/2269305094 learned_addr learned my addr 172.21.15.81:0/2269305094 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa44b7fe700 1 --2- 172.21.15.81:0/2269305094 >> v2:172.21.15.117:3300/0 conn(0x7fa44c0cddf0 0x7fa44c0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa450eb8700 1 --2- 172.21.15.81:0/2269305094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c160de0 0x7fa44c161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa44bfff700 1 -- 172.21.15.81:0/2269305094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c160de0 msgr2=0x7fa44c161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa44bfff700 1 --2- 172.21.15.81:0/2269305094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c160de0 0x7fa44c161230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa44bfff700 1 -- 172.21.15.81:0/2269305094 >> v2:172.21.15.117:3300/0 conn(0x7fa44c0cddf0 msgr2=0x7fa44c0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:50.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa44bfff700 1 --2- 172.21.15.81:0/2269305094 >> v2:172.21.15.117:3300/0 conn(0x7fa44c0cddf0 0x7fa44c0cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa44bfff700 1 -- 172.21.15.81:0/2269305094 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c0ce5b0 con 0x7fa44c0cf980 2024-05-08T23:31:50.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa44b7fe700 1 --2- 172.21.15.81:0/2269305094 >> v2:172.21.15.117:3300/0 conn(0x7fa44c0cddf0 0x7fa44c0cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:50.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.216+0000 7fa450eb8700 1 --2- 172.21.15.81:0/2269305094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c160de0 0x7fa44c161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:50.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.220+0000 7fa44bfff700 1 --2- 172.21.15.81:0/2269305094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa44c0cf980 0x7fa44c15e2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa44c08c4a0 tx=0x7fa44400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2533be4c3ca48 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:50.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.220+0000 7fa44affd700 1 -- 172.21.15.81:0/2269305094 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44400e070 con 0x7fa44c0cf980 2024-05-08T23:31:50.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.220+0000 7fa44affd700 1 -- 172.21.15.81:0/2269305094 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44400b9c0 con 0x7fa44c0cf980 2024-05-08T23:31:50.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.220+0000 7fa44affd700 1 -- 172.21.15.81:0/2269305094 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44400bd70 con 0x7fa44c0cf980 2024-05-08T23:31:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.220+0000 7fa4516b9700 1 -- 172.21.15.81:0/2269305094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa44c0cf980 msgr2=0x7fa44c15e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.220+0000 7fa4516b9700 1 --2- 172.21.15.81:0/2269305094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa44c0cf980 0x7fa44c15e2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa44c08c4a0 tx=0x7fa44400b330 comp rx=0 tx=0).stop 2024-05-08T23:31:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.220+0000 7fa4516b9700 1 -- 172.21.15.81:0/2269305094 shutdown_connections 2024-05-08T23:31:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.220+0000 7fa4516b9700 1 --2- 172.21.15.81:0/2269305094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c160de0 0x7fa44c161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.220+0000 7fa4516b9700 1 --2- 172.21.15.81:0/2269305094 >> v2:172.21.15.117:3300/0 conn(0x7fa44c0cddf0 0x7fa44c0cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.220+0000 7fa4516b9700 1 --2- 172.21.15.81:0/2269305094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa44c0cf980 0x7fa44c15e2f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.220+0000 7fa4516b9700 1 -- 172.21.15.81:0/2269305094 >> 172.21.15.81:0/2269305094 conn(0x7fa44c09c370 msgr2=0x7fa44c0d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.220+0000 7fa4516b9700 1 -- 172.21.15.81:0/2269305094 shutdown_connections 2024-05-08T23:31:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.220+0000 7fa4516b9700 1 -- 172.21.15.81:0/2269305094 wait complete. 2024-05-08T23:31:50.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.220+0000 7fa4516b9700 1 Processor -- start 2024-05-08T23:31:50.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.220+0000 7fa4516b9700 1 -- start start 2024-05-08T23:31:50.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa4516b9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44c0cddf0 0x7fa44c0c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa4516b9700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c0cf980 0x7fa44c150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa4516b9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa44c160de0 0x7fa44c1f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa4516b9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c14f1e0 con 0x7fa44c160de0 2024-05-08T23:31:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa4516b9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c14f4c0 con 0x7fa44c0cddf0 2024-05-08T23:31:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa4516b9700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c14f7a0 con 0x7fa44c0cf980 2024-05-08T23:31:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa450eb8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa44c160de0 0x7fa44c1f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa450eb8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa44c160de0 0x7fa44c1f3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58396/0 (socket says 172.21.15.81:58396) 2024-05-08T23:31:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa450eb8700 1 -- 172.21.15.81:0/2431436082 learned_addr learned my addr 172.21.15.81:0/2431436082 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa44bfff700 1 --2- 172.21.15.81:0/2431436082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44c0cddf0 0x7fa44c0c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa44b7fe700 1 --2- 172.21.15.81:0/2431436082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c0cf980 0x7fa44c150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa450eb8700 1 -- 172.21.15.81:0/2431436082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c0cf980 msgr2=0x7fa44c150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa450eb8700 1 --2- 172.21.15.81:0/2431436082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c0cf980 0x7fa44c150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa450eb8700 1 -- 172.21.15.81:0/2431436082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44c0cddf0 msgr2=0x7fa44c0c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa450eb8700 1 --2- 172.21.15.81:0/2431436082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44c0cddf0 0x7fa44c0c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa450eb8700 1 -- 172.21.15.81:0/2431436082 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa444009b30 con 0x7fa44c160de0 2024-05-08T23:31:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa450eb8700 1 --2- 172.21.15.81:0/2431436082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa44c160de0 0x7fa44c1f3470 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00c7c0 tx=0x7fa43c00cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa448ff9700 1 -- 172.21.15.81:0/2431436082 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c011860 con 0x7fa44c160de0 2024-05-08T23:31:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa448ff9700 1 -- 172.21.15.81:0/2431436082 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43c0119c0 con 0x7fa44c160de0 2024-05-08T23:31:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa4516b9700 1 -- 172.21.15.81:0/2431436082 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44c14fa80 con 0x7fa44c160de0 2024-05-08T23:31:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa448ff9700 1 -- 172.21.15.81:0/2431436082 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c011d70 con 0x7fa44c160de0 2024-05-08T23:31:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa4516b9700 1 -- 172.21.15.81:0/2431436082 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44c0c6cd0 con 0x7fa44c160de0 2024-05-08T23:31:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.224+0000 7fa4516b9700 1 -- 172.21.15.81:0/2431436082 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44c14ffa0 con 0x7fa44c160de0 2024-05-08T23:31:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.228+0000 7fa448ff9700 1 -- 172.21.15.81:0/2431436082 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa43c021350 con 0x7fa44c160de0 2024-05-08T23:31:50.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.228+0000 7fa448ff9700 1 --2- 172.21.15.81:0/2431436082 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa438066730 0x7fa438068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.228+0000 7fa448ff9700 1 -- 172.21.15.81:0/2431436082 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fa43c084130 con 0x7fa44c160de0 2024-05-08T23:31:50.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.228+0000 7fa44bfff700 1 --2- 172.21.15.81:0/2431436082 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa438066730 0x7fa438068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.228+0000 7fa44bfff700 1 --2- 172.21.15.81:0/2431436082 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa438066730 0x7fa438068bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa44400b370 tx=0x7fa444009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:50.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.236+0000 7fa448ff9700 1 -- 172.21.15.81:0/2431436082 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa44c14ffa0 con 0x7fa44c160de0 2024-05-08T23:31:50.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.416+0000 7fa4516b9700 1 -- 172.21.15.81:0/2431436082 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa44c14ffa0 con 0x7fa44c160de0 2024-05-08T23:31:50.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.416+0000 7fa448ff9700 1 -- 172.21.15.81:0/2431436082 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa44c14ffa0 con 0x7fa44c160de0 2024-05-08T23:31:50.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.420+0000 7fa4516b9700 1 -- 172.21.15.81:0/2431436082 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa438066730 msgr2=0x7fa438068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.420+0000 7fa4516b9700 1 --2- 172.21.15.81:0/2431436082 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa438066730 0x7fa438068bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa44400b370 tx=0x7fa444009000 comp rx=0 tx=0).stop 2024-05-08T23:31:50.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.420+0000 7fa4516b9700 1 -- 172.21.15.81:0/2431436082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa44c160de0 msgr2=0x7fa44c1f3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.420+0000 7fa4516b9700 1 --2- 172.21.15.81:0/2431436082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa44c160de0 0x7fa44c1f3470 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00c7c0 tx=0x7fa43c00cad0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.420+0000 7fa4516b9700 1 -- 172.21.15.81:0/2431436082 shutdown_connections 2024-05-08T23:31:50.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.420+0000 7fa4516b9700 1 --2- 172.21.15.81:0/2431436082 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa438066730 0x7fa438068bf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.420+0000 7fa4516b9700 1 --2- 172.21.15.81:0/2431436082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa44c160de0 0x7fa44c1f3470 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.420+0000 7fa4516b9700 1 --2- 172.21.15.81:0/2431436082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa44c0cf980 0x7fa44c150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.422 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.420+0000 7fa4516b9700 1 --2- 172.21.15.81:0/2431436082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44c0cddf0 0x7fa44c0c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.422 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.420+0000 7fa4516b9700 1 -- 172.21.15.81:0/2431436082 >> 172.21.15.81:0/2431436082 conn(0x7fa44c09c370 msgr2=0x7fa44c15d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:50.422 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.420+0000 7fa4516b9700 1 -- 172.21.15.81:0/2431436082 shutdown_connections 2024-05-08T23:31:50.422 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.420+0000 7fa4516b9700 1 -- 172.21.15.81:0/2431436082 wait complete. 2024-05-08T23:31:50.422 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:31:50.432 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:31:50.432 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:31:50.440 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:50.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.544+0000 7f013cd0b700 1 Processor -- start 2024-05-08T23:31:50.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.544+0000 7f013cd0b700 1 -- start start 2024-05-08T23:31:50.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.544+0000 7f013cd0b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f01380b3c20 0x7f01381624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.544+0000 7f013cd0b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0138071e70 0x7f01380b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.544+0000 7f013cd0b700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0138158960 con 0x7f0138165070 2024-05-08T23:31:50.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.544+0000 7f013cd0b700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01380b1f60 con 0x7f01380b3c20 2024-05-08T23:31:50.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.544+0000 7f013cd0b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0138071120 con 0x7f0138071e70 2024-05-08T23:31:50.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.544+0000 7f01377fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f01380b3c20 0x7f01381624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.544+0000 7f01377fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f01380b3c20 0x7f01381624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58400/0 (socket says 172.21.15.81:58400) 2024-05-08T23:31:50.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.544+0000 7f01377fe700 1 -- 172.21.15.81:0/375270653 learned_addr learned my addr 172.21.15.81:0/375270653 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:50.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.544+0000 7f0136ffd700 1 --2- 172.21.15.81:0/375270653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0138071e70 0x7f01380b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.544+0000 7f01377fe700 1 -- 172.21.15.81:0/375270653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0138071e70 msgr2=0x7f01380b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:50.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.544+0000 7f01377fe700 1 --2- 172.21.15.81:0/375270653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0138071e70 0x7f01380b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.544+0000 7f01377fe700 1 -- 172.21.15.81:0/375270653 >> v1:172.21.15.117:6789/0 conn(0x7f0138165070 legacy=0x7f0138165500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.548+0000 7f01377fe700 1 -- 172.21.15.81:0/375270653 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01380b4540 con 0x7f01380b3c20 2024-05-08T23:31:50.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.548+0000 7f01377fe700 1 --2- 172.21.15.81:0/375270653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01380b3c20 0x7f01381624b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f012c009910 tx=0x7f012c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff253f73d05550f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:50.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.548+0000 7f01367fc700 1 -- 172.21.15.81:0/375270653 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c011830 con 0x7f01380b3c20 2024-05-08T23:31:50.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.548+0000 7f01367fc700 1 -- 172.21.15.81:0/375270653 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c011990 con 0x7f01380b3c20 2024-05-08T23:31:50.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.548+0000 7f01367fc700 1 -- 172.21.15.81:0/375270653 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c011d40 con 0x7f01380b3c20 2024-05-08T23:31:50.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.548+0000 7f013cd0b700 1 -- 172.21.15.81:0/375270653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01380b3c20 msgr2=0x7f01381624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.548+0000 7f013cd0b700 1 --2- 172.21.15.81:0/375270653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01380b3c20 0x7f01381624b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f012c009910 tx=0x7f012c009c20 comp rx=0 tx=0).stop 2024-05-08T23:31:50.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.548+0000 7f013cd0b700 1 -- 172.21.15.81:0/375270653 shutdown_connections 2024-05-08T23:31:50.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.548+0000 7f013cd0b700 1 --2- 172.21.15.81:0/375270653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0138071e70 0x7f01380b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.548+0000 7f013cd0b700 1 --2- 172.21.15.81:0/375270653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01380b3c20 0x7f01381624b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.548+0000 7f013cd0b700 1 -- 172.21.15.81:0/375270653 >> 172.21.15.81:0/375270653 conn(0x7f013806c330 msgr2=0x7f013806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:50.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.548+0000 7f013cd0b700 1 -- 172.21.15.81:0/375270653 shutdown_connections 2024-05-08T23:31:50.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.548+0000 7f013cd0b700 1 -- 172.21.15.81:0/375270653 wait complete. 2024-05-08T23:31:50.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f013cd0b700 1 Processor -- start 2024-05-08T23:31:50.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f013cd0b700 1 -- start start 2024-05-08T23:31:50.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f013cd0b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0138071e70 0x7f0138149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f013cd0b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01380b3c20 0x7f0138165500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f013cd0b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0138165070 0x7f0138165a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f013cd0b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013815aa30 con 0x7f0138165070 2024-05-08T23:31:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f013cd0b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0138159c70 con 0x7f01380b3c20 2024-05-08T23:31:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f013cd0b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0138159f50 con 0x7f0138071e70 2024-05-08T23:31:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f0137fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0138165070 0x7f0138165a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f0137fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0138165070 0x7f0138165a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58412/0 (socket says 172.21.15.81:58412) 2024-05-08T23:31:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f0137fff700 1 -- 172.21.15.81:0/1868219762 learned_addr learned my addr 172.21.15.81:0/1868219762 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f0136ffd700 1 --2- 172.21.15.81:0/1868219762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01380b3c20 0x7f0138165500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f01377fe700 1 --2- 172.21.15.81:0/1868219762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0138071e70 0x7f0138149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f0137fff700 1 -- 172.21.15.81:0/1868219762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0138071e70 msgr2=0x7f0138149fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f0137fff700 1 --2- 172.21.15.81:0/1868219762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0138071e70 0x7f0138149fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f0137fff700 1 -- 172.21.15.81:0/1868219762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01380b3c20 msgr2=0x7f0138165500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f0137fff700 1 --2- 172.21.15.81:0/1868219762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01380b3c20 0x7f0138165500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.552+0000 7f0137fff700 1 -- 172.21.15.81:0/1868219762 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c009570 con 0x7f0138165070 2024-05-08T23:31:50.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.556+0000 7f01377fe700 1 --2- 172.21.15.81:0/1868219762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0138071e70 0x7f0138149fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:50.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.556+0000 7f0137fff700 1 --2- 172.21.15.81:0/1868219762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0138165070 0x7f0138165a40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f013000cc50 tx=0x7f013000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:50.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.556+0000 7f011ffff700 1 -- 172.21.15.81:0/1868219762 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130003c20 con 0x7f0138165070 2024-05-08T23:31:50.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.556+0000 7f011ffff700 1 -- 172.21.15.81:0/1868219762 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0130003d80 con 0x7f0138165070 2024-05-08T23:31:50.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.556+0000 7f013cd0b700 1 -- 172.21.15.81:0/1868219762 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f013815a230 con 0x7f0138165070 2024-05-08T23:31:50.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.556+0000 7f011ffff700 1 -- 172.21.15.81:0/1868219762 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013000ab60 con 0x7f0138165070 2024-05-08T23:31:50.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.556+0000 7f013cd0b700 1 -- 172.21.15.81:0/1868219762 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f013814a4f0 con 0x7f0138165070 2024-05-08T23:31:50.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.556+0000 7f013cd0b700 1 -- 172.21.15.81:0/1868219762 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01380684b0 con 0x7f0138165070 2024-05-08T23:31:50.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.556+0000 7f011ffff700 1 -- 172.21.15.81:0/1868219762 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f013000acc0 con 0x7f0138165070 2024-05-08T23:31:50.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.556+0000 7f011ffff700 1 --2- 172.21.15.81:0/1868219762 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0124066730 0x7f0124068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.556+0000 7f011ffff700 1 -- 172.21.15.81:0/1868219762 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f0130084250 con 0x7f0138165070 2024-05-08T23:31:50.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.560+0000 7f01377fe700 1 --2- 172.21.15.81:0/1868219762 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0124066730 0x7f0124068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.560+0000 7f01377fe700 1 --2- 172.21.15.81:0/1868219762 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0124066730 0x7f0124068bf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f01380cb7d0 tx=0x7f012c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:50.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.564+0000 7f011ffff700 1 -- 172.21.15.81:0/1868219762 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f01300514c0 con 0x7f0138165070 2024-05-08T23:31:50.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.724+0000 7f013cd0b700 1 -- 172.21.15.81:0/1868219762 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0138071c50 con 0x7f0138165070 2024-05-08T23:31:50.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.728+0000 7f011ffff700 1 -- 172.21.15.81:0/1868219762 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+17880 (secure 0 0 0) 0x7f0130054ae0 con 0x7f0138165070 2024-05-08T23:31:50.728 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:31:50.728 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":39,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:48.310750+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:31:35.620423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:31:36.878524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-05-08T23:31:46.337484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:50.730 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.728+0000 7f013cd0b700 1 -- 172.21.15.81:0/1868219762 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0124066730 msgr2=0x7f0124068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.730 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.728+0000 7f013cd0b700 1 --2- 172.21.15.81:0/1868219762 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0124066730 0x7f0124068bf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f01380cb7d0 tx=0x7f012c008030 comp rx=0 tx=0).stop 2024-05-08T23:31:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.728+0000 7f013cd0b700 1 -- 172.21.15.81:0/1868219762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0138165070 msgr2=0x7f0138165a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.728+0000 7f013cd0b700 1 --2- 172.21.15.81:0/1868219762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0138165070 0x7f0138165a40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f013000cc50 tx=0x7f013000cf60 comp rx=0 tx=0).stop 2024-05-08T23:31:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.728+0000 7f013cd0b700 1 -- 172.21.15.81:0/1868219762 shutdown_connections 2024-05-08T23:31:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.728+0000 7f013cd0b700 1 --2- 172.21.15.81:0/1868219762 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0124066730 0x7f0124068bf0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.728+0000 7f013cd0b700 1 --2- 172.21.15.81:0/1868219762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0138165070 0x7f0138165a40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.728+0000 7f013cd0b700 1 --2- 172.21.15.81:0/1868219762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01380b3c20 0x7f0138165500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.728+0000 7f013cd0b700 1 --2- 172.21.15.81:0/1868219762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0138071e70 0x7f0138149fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.728+0000 7f013cd0b700 1 -- 172.21.15.81:0/1868219762 >> 172.21.15.81:0/1868219762 conn(0x7f013806c330 msgr2=0x7f01380afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.728+0000 7f013cd0b700 1 -- 172.21.15.81:0/1868219762 shutdown_connections 2024-05-08T23:31:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.728+0000 7f013cd0b700 1 -- 172.21.15.81:0/1868219762 wait complete. 2024-05-08T23:31:50.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:31:35.620423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:31:36.878524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-05-08T23:31:46.337484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:31:50.742 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:50.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.852+0000 7f5f21a10700 1 Processor -- start 2024-05-08T23:31:50.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.852+0000 7f5f21a10700 1 -- start start 2024-05-08T23:31:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.852+0000 7f5f21a10700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5f1c15b030 0x7f5f1c0ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.852+0000 7f5f21a10700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c0acb50 con 0x7f5f1c0b2fe0 2024-05-08T23:31:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.852+0000 7f5f21a10700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c0b1870 con 0x7f5f1c15abd0 2024-05-08T23:31:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.852+0000 7f5f21a10700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1c0b2080 con 0x7f5f1c15b030 2024-05-08T23:31:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.852+0000 7f5f20a0e700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5f1c0b2fe0 0x7f5f1c0b33c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:48130/0 (socket says 172.21.15.81:48130) 2024-05-08T23:31:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.852+0000 7f5f2120f700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5f1c15b030 0x7f5f1c0ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.852+0000 7f5f20a0e700 1 -- 172.21.15.81:0/3178596508 learned_addr learned my addr 172.21.15.81:0/3178596508 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.856+0000 7f5f2120f700 1 -- 172.21.15.81:0/3178596508 >> v1:172.21.15.117:6789/0 conn(0x7f5f1c15abd0 legacy=0x7f5f1c0acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.856+0000 7f5f2120f700 1 -- 172.21.15.81:0/3178596508 >> v1:172.21.15.81:6789/0 conn(0x7f5f1c0b2fe0 legacy=0x7f5f1c0b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.856+0000 7f5f2120f700 1 -- 172.21.15.81:0/3178596508 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c0b2360 con 0x7f5f1c15b030 2024-05-08T23:31:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.856+0000 7f5f2120f700 1 --2- 172.21.15.81:0/3178596508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1c15b030 0x7f5f1c0ad6d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5f14009d90 tx=0x7f5f1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc3c641bd0e3909 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.856+0000 7f5f137fe700 1 -- 172.21.15.81:0/3178596508 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400e070 con 0x7f5f1c15b030 2024-05-08T23:31:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.856+0000 7f5f137fe700 1 -- 172.21.15.81:0/3178596508 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1400bbd0 con 0x7f5f1c15b030 2024-05-08T23:31:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.856+0000 7f5f137fe700 1 -- 172.21.15.81:0/3178596508 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14012430 con 0x7f5f1c15b030 2024-05-08T23:31:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.856+0000 7f5f21a10700 1 -- 172.21.15.81:0/3178596508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1c15b030 msgr2=0x7f5f1c0ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.856+0000 7f5f21a10700 1 --2- 172.21.15.81:0/3178596508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1c15b030 0x7f5f1c0ad6d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5f14009d90 tx=0x7f5f1400b330 comp rx=0 tx=0).stop 2024-05-08T23:31:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.856+0000 7f5f21a10700 1 -- 172.21.15.81:0/3178596508 shutdown_connections 2024-05-08T23:31:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.856+0000 7f5f21a10700 1 --2- 172.21.15.81:0/3178596508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1c15b030 0x7f5f1c0ad6d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.856+0000 7f5f21a10700 1 -- 172.21.15.81:0/3178596508 >> 172.21.15.81:0/3178596508 conn(0x7f5f1c0b47c0 msgr2=0x7f5f1c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.856+0000 7f5f21a10700 1 -- 172.21.15.81:0/3178596508 shutdown_connections 2024-05-08T23:31:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.856+0000 7f5f21a10700 1 -- 172.21.15.81:0/3178596508 wait complete. 2024-05-08T23:31:50.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f21a10700 1 Processor -- start 2024-05-08T23:31:50.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f21a10700 1 -- start start 2024-05-08T23:31:50.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f21a10700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f1c0b2fe0 0x7f5f1c0961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f21a10700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f1c15abd0 0x7f5f1c0991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f21a10700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1c15b030 0x7f5f1c06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f21a10700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c09b3a0 con 0x7f5f1c15b030 2024-05-08T23:31:50.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f21a10700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c09b680 con 0x7f5f1c15abd0 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f21a10700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c09b960 con 0x7f5f1c0b2fe0 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f2120f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1c15b030 0x7f5f1c06d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f2120f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1c15b030 0x7f5f1c06d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58440/0 (socket says 172.21.15.81:58440) 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f2120f700 1 -- 172.21.15.81:0/475168721 learned_addr learned my addr 172.21.15.81:0/475168721 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f13fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f1c15abd0 0x7f5f1c0991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f20a0e700 1 --2- 172.21.15.81:0/475168721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f1c0b2fe0 0x7f5f1c0961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f2120f700 1 -- 172.21.15.81:0/475168721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f1c0b2fe0 msgr2=0x7f5f1c0961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f2120f700 1 --2- 172.21.15.81:0/475168721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f1c0b2fe0 0x7f5f1c0961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f2120f700 1 -- 172.21.15.81:0/475168721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f1c15abd0 msgr2=0x7f5f1c0991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f2120f700 1 --2- 172.21.15.81:0/475168721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f1c15abd0 0x7f5f1c0991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f2120f700 1 -- 172.21.15.81:0/475168721 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f14009a70 con 0x7f5f1c15b030 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.860+0000 7f5f20a0e700 1 --2- 172.21.15.81:0/475168721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f1c0b2fe0 0x7f5f1c0961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.864+0000 7f5f2120f700 1 --2- 172.21.15.81:0/475168721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1c15b030 0x7f5f1c06d5f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5f14014040 tx=0x7f5f14004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.864+0000 7f5f117fa700 1 -- 172.21.15.81:0/475168721 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14012dd0 con 0x7f5f1c15b030 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.864+0000 7f5f117fa700 1 -- 172.21.15.81:0/475168721 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f14020070 con 0x7f5f1c15b030 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.864+0000 7f5f21a10700 1 -- 172.21.15.81:0/475168721 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c09bc40 con 0x7f5f1c15b030 2024-05-08T23:31:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.864+0000 7f5f117fa700 1 -- 172.21.15.81:0/475168721 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14004280 con 0x7f5f1c15b030 2024-05-08T23:31:50.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.864+0000 7f5f21a10700 1 -- 172.21.15.81:0/475168721 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c09afe0 con 0x7f5f1c15b030 2024-05-08T23:31:50.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.864+0000 7f5f21a10700 1 -- 172.21.15.81:0/475168721 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c0684b0 con 0x7f5f1c15b030 2024-05-08T23:31:50.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.864+0000 7f5f117fa700 1 -- 172.21.15.81:0/475168721 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5f140043e0 con 0x7f5f1c15b030 2024-05-08T23:31:50.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.864+0000 7f5f117fa700 1 --2- 172.21.15.81:0/475168721 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5f08066730 0x7f5f08068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:50.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.864+0000 7f5f20a0e700 1 --2- 172.21.15.81:0/475168721 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5f08066730 0x7f5f08068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:50.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.864+0000 7f5f117fa700 1 -- 172.21.15.81:0/475168721 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f5f14022070 con 0x7f5f1c15b030 2024-05-08T23:31:50.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.868+0000 7f5f20a0e700 1 --2- 172.21.15.81:0/475168721 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5f08066730 0x7f5f08068bf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f18007fd0 tx=0x7f5f18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:50.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:50.872+0000 7f5f117fa700 1 -- 172.21.15.81:0/475168721 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5f14053310 con 0x7f5f1c15b030 2024-05-08T23:31:51.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.032+0000 7f5f21a10700 1 -- 172.21.15.81:0/475168721 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f1c09b4e0 con 0x7f5f1c15b030 2024-05-08T23:31:51.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.032+0000 7f5f117fa700 1 -- 172.21.15.81:0/475168721 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f1402c020 con 0x7f5f1c15b030 2024-05-08T23:31:51.032 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:31:51.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.032+0000 7f5f21a10700 1 -- 172.21.15.81:0/475168721 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5f08066730 msgr2=0x7f5f08068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.032+0000 7f5f21a10700 1 --2- 172.21.15.81:0/475168721 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5f08066730 0x7f5f08068bf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f18007fd0 tx=0x7f5f18009040 comp rx=0 tx=0).stop 2024-05-08T23:31:51.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.032+0000 7f5f21a10700 1 -- 172.21.15.81:0/475168721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1c15b030 msgr2=0x7f5f1c06d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.032+0000 7f5f21a10700 1 --2- 172.21.15.81:0/475168721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1c15b030 0x7f5f1c06d5f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5f14014040 tx=0x7f5f14004af0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.032+0000 7f5f21a10700 1 -- 172.21.15.81:0/475168721 shutdown_connections 2024-05-08T23:31:51.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.032+0000 7f5f21a10700 1 --2- 172.21.15.81:0/475168721 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5f08066730 0x7f5f08068bf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.032+0000 7f5f21a10700 1 --2- 172.21.15.81:0/475168721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1c15b030 0x7f5f1c06d5f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.032+0000 7f5f21a10700 1 --2- 172.21.15.81:0/475168721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f1c15abd0 0x7f5f1c0991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.032+0000 7f5f21a10700 1 --2- 172.21.15.81:0/475168721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f1c0b2fe0 0x7f5f1c0961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.032+0000 7f5f21a10700 1 -- 172.21.15.81:0/475168721 >> 172.21.15.81:0/475168721 conn(0x7f5f1c0b47c0 msgr2=0x7f5f1c14b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:51.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.032+0000 7f5f21a10700 1 -- 172.21.15.81:0/475168721 shutdown_connections 2024-05-08T23:31:51.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.032+0000 7f5f21a10700 1 -- 172.21.15.81:0/475168721 wait complete. 2024-05-08T23:31:51.046 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:51.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.156+0000 7fe603fff700 1 Processor -- start 2024-05-08T23:31:51.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.156+0000 7fe603fff700 1 -- start start 2024-05-08T23:31:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe603fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe5fc0e3b40 0x7fe5fc0e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe603fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe5fc006150 0x7fe5fc0e4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe603fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5fc0e49a0 con 0x7fe5fc0e0250 2024-05-08T23:31:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe603fff700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5fc0540b0 con 0x7fe5fc006150 2024-05-08T23:31:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe603fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5fc0532f0 con 0x7fe5fc0e3b40 2024-05-08T23:31:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe6037fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe5fc006150 0x7fe5fc0e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe6037fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe5fc006150 0x7fe5fc0e4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58448/0 (socket says 172.21.15.81:58448) 2024-05-08T23:31:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe6037fe700 1 -- 172.21.15.81:0/3320616146 learned_addr learned my addr 172.21.15.81:0/3320616146 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe602ffd700 1 --2- 172.21.15.81:0/3320616146 >> v2:172.21.15.117:3300/0 conn(0x7fe5fc0e3b40 0x7fe5fc0e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe6037fe700 1 -- 172.21.15.81:0/3320616146 >> v2:172.21.15.117:3300/0 conn(0x7fe5fc0e3b40 msgr2=0x7fe5fc0e3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe6037fe700 1 --2- 172.21.15.81:0/3320616146 >> v2:172.21.15.117:3300/0 conn(0x7fe5fc0e3b40 0x7fe5fc0e3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe6037fe700 1 -- 172.21.15.81:0/3320616146 >> v1:172.21.15.117:6789/0 conn(0x7fe5fc0e0250 legacy=0x7fe5fc038ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe6037fe700 1 -- 172.21.15.81:0/3320616146 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc0535d0 con 0x7fe5fc006150 2024-05-08T23:31:51.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe6037fe700 1 --2- 172.21.15.81:0/3320616146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5fc006150 0x7fe5fc0e4460 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe6040698a0 tx=0x7fe604068970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b35fbba5d8f25c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:51.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe601ffb700 1 -- 172.21.15.81:0/3320616146 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6040679b0 con 0x7fe5fc006150 2024-05-08T23:31:51.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe601ffb700 1 -- 172.21.15.81:0/3320616146 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe604067b10 con 0x7fe5fc006150 2024-05-08T23:31:51.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe601ffb700 1 -- 172.21.15.81:0/3320616146 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6040720c0 con 0x7fe5fc006150 2024-05-08T23:31:51.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe603fff700 1 -- 172.21.15.81:0/3320616146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5fc006150 msgr2=0x7fe5fc0e4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe603fff700 1 --2- 172.21.15.81:0/3320616146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5fc006150 0x7fe5fc0e4460 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe6040698a0 tx=0x7fe604068970 comp rx=0 tx=0).stop 2024-05-08T23:31:51.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe603fff700 1 -- 172.21.15.81:0/3320616146 shutdown_connections 2024-05-08T23:31:51.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe603fff700 1 --2- 172.21.15.81:0/3320616146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5fc006150 0x7fe5fc0e4460 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe603fff700 1 --2- 172.21.15.81:0/3320616146 >> v2:172.21.15.117:3300/0 conn(0x7fe5fc0e3b40 0x7fe5fc0e3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe603fff700 1 -- 172.21.15.81:0/3320616146 >> 172.21.15.81:0/3320616146 conn(0x7fe5fc054b70 msgr2=0x7fe5fc054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:51.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe603fff700 1 -- 172.21.15.81:0/3320616146 shutdown_connections 2024-05-08T23:31:51.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.160+0000 7fe603fff700 1 -- 172.21.15.81:0/3320616146 wait complete. 2024-05-08T23:31:51.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe603fff700 1 Processor -- start 2024-05-08T23:31:51.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe603fff700 1 -- start start 2024-05-08T23:31:51.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe603fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5fc006150 0x7fe5fc0e0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe603fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5fc0e3b40 0x7fe5fc0defb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe603fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5fc0de370 0x7fe5fc038e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe603fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc0d68d0 con 0x7fe5fc006150 2024-05-08T23:31:51.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe603fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc0df4f0 con 0x7fe5fc0e3b40 2024-05-08T23:31:51.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe603fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc0397d0 con 0x7fe5fc0de370 2024-05-08T23:31:51.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe602ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5fc006150 0x7fe5fc0e0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe602ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5fc006150 0x7fe5fc0e0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58464/0 (socket says 172.21.15.81:58464) 2024-05-08T23:31:51.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe602ffd700 1 -- 172.21.15.81:0/3477110634 learned_addr learned my addr 172.21.15.81:0/3477110634 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:51.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe6037fe700 1 --2- 172.21.15.81:0/3477110634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5fc0de370 0x7fe5fc038e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe6027fc700 1 --2- 172.21.15.81:0/3477110634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5fc0e3b40 0x7fe5fc0defb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe602ffd700 1 -- 172.21.15.81:0/3477110634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5fc0de370 msgr2=0x7fe5fc038e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe602ffd700 1 --2- 172.21.15.81:0/3477110634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5fc0de370 0x7fe5fc038e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe602ffd700 1 -- 172.21.15.81:0/3477110634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5fc0e3b40 msgr2=0x7fe5fc0defb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe602ffd700 1 --2- 172.21.15.81:0/3477110634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5fc0e3b40 0x7fe5fc0defb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe602ffd700 1 -- 172.21.15.81:0/3477110634 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60404f5c0 con 0x7fe5fc006150 2024-05-08T23:31:51.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe602ffd700 1 --2- 172.21.15.81:0/3477110634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5fc006150 0x7fe5fc0e0640 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400e9d0 tx=0x7fe5f400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:51.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.164+0000 7fe5fb7fe700 1 -- 172.21.15.81:0/3477110634 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4009720 con 0x7fe5fc006150 2024-05-08T23:31:51.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.168+0000 7fe5fb7fe700 1 -- 172.21.15.81:0/3477110634 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f400ee70 con 0x7fe5fc006150 2024-05-08T23:31:51.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.168+0000 7fe603fff700 1 -- 172.21.15.81:0/3477110634 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5fc039ab0 con 0x7fe5fc006150 2024-05-08T23:31:51.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.168+0000 7fe5fb7fe700 1 -- 172.21.15.81:0/3477110634 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f400ca50 con 0x7fe5fc006150 2024-05-08T23:31:51.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.168+0000 7fe603fff700 1 -- 172.21.15.81:0/3477110634 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5fc049580 con 0x7fe5fc006150 2024-05-08T23:31:51.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.168+0000 7fe603fff700 1 -- 172.21.15.81:0/3477110634 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5fc005440 con 0x7fe5fc006150 2024-05-08T23:31:51.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.168+0000 7fe5fb7fe700 1 -- 172.21.15.81:0/3477110634 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe5f400cbb0 con 0x7fe5fc006150 2024-05-08T23:31:51.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.172+0000 7fe5fb7fe700 1 --2- 172.21.15.81:0/3477110634 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe5e4066730 0x7fe5e4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.172+0000 7fe6027fc700 1 --2- 172.21.15.81:0/3477110634 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe5e4066730 0x7fe5e4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.172+0000 7fe5fb7fe700 1 -- 172.21.15.81:0/3477110634 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fe5f40842f0 con 0x7fe5fc006150 2024-05-08T23:31:51.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.172+0000 7fe6027fc700 1 --2- 172.21.15.81:0/3477110634 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe5e4066730 0x7fe5e4068bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc04a9a0 tx=0x7fe5ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:51.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.176+0000 7fe5fb7fe700 1 -- 172.21.15.81:0/3477110634 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe5f40514d0 con 0x7fe5fc006150 2024-05-08T23:31:51.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.348+0000 7fe603fff700 1 -- 172.21.15.81:0/3477110634 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe5fc054280 con 0x7fe5fc006150 2024-05-08T23:31:51.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.348+0000 7fe5fb7fe700 1 -- 172.21.15.81:0/3477110634 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe5f400ce60 con 0x7fe5fc006150 2024-05-08T23:31:51.350 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:51.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.352+0000 7fe603fff700 1 -- 172.21.15.81:0/3477110634 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe5e4066730 msgr2=0x7fe5e4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.352+0000 7fe603fff700 1 --2- 172.21.15.81:0/3477110634 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe5e4066730 0x7fe5e4068bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc04a9a0 tx=0x7fe5ec009040 comp rx=0 tx=0).stop 2024-05-08T23:31:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.352+0000 7fe603fff700 1 -- 172.21.15.81:0/3477110634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5fc006150 msgr2=0x7fe5fc0e0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.352+0000 7fe603fff700 1 --2- 172.21.15.81:0/3477110634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5fc006150 0x7fe5fc0e0640 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400e9d0 tx=0x7fe5f400ece0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.352+0000 7fe603fff700 1 -- 172.21.15.81:0/3477110634 shutdown_connections 2024-05-08T23:31:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.352+0000 7fe603fff700 1 --2- 172.21.15.81:0/3477110634 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe5e4066730 0x7fe5e4068bf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.352+0000 7fe603fff700 1 --2- 172.21.15.81:0/3477110634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5fc0de370 0x7fe5fc038e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.352+0000 7fe603fff700 1 --2- 172.21.15.81:0/3477110634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5fc0e3b40 0x7fe5fc0defb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.352+0000 7fe603fff700 1 --2- 172.21.15.81:0/3477110634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5fc006150 0x7fe5fc0e0640 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.352+0000 7fe603fff700 1 -- 172.21.15.81:0/3477110634 >> 172.21.15.81:0/3477110634 conn(0x7fe5fc054b70 msgr2=0x7fe5fc0daa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.352+0000 7fe603fff700 1 -- 172.21.15.81:0/3477110634 shutdown_connections 2024-05-08T23:31:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.352+0000 7fe603fff700 1 -- 172.21.15.81:0/3477110634 wait complete. 2024-05-08T23:31:51.365 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:51.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.476+0000 7fbf3efb7700 1 Processor -- start 2024-05-08T23:31:51.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.476+0000 7fbf3efb7700 1 -- start start 2024-05-08T23:31:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3efb7700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbf38071a50 0x7fbf380ad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3efb7700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf38070d20 con 0x7fbf38161470 2024-05-08T23:31:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3efb7700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf38071000 con 0x7fbf380b32e0 2024-05-08T23:31:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3efb7700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3806ca90 con 0x7fbf38071a50 2024-05-08T23:31:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3dfb5700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fbf38161470 0x7fbf3806c380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:48136/0 (socket says 172.21.15.81:48136) 2024-05-08T23:31:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3dfb5700 1 -- 172.21.15.81:0/3739537221 learned_addr learned my addr 172.21.15.81:0/3739537221 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3e7b6700 1 --2- 172.21.15.81:0/3739537221 >> v2:172.21.15.81:3300/0 conn(0x7fbf38071a50 0x7fbf380ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3cfb3700 1 -- 172.21.15.81:0/3739537221 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205363553 0 0) 0x7fbf38070d20 con 0x7fbf38161470 2024-05-08T23:31:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3cfb3700 1 -- 172.21.15.81:0/3739537221 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf240035d0 con 0x7fbf38161470 2024-05-08T23:31:51.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3cfb3700 1 -- 172.21.15.81:0/3739537221 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140033382 0 0) 0x7fbf38071000 con 0x7fbf380b32e0 2024-05-08T23:31:51.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3cfb3700 1 -- 172.21.15.81:0/3739537221 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf38070d20 con 0x7fbf380b32e0 2024-05-08T23:31:51.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3e7b6700 1 -- 172.21.15.81:0/3739537221 >> v1:172.21.15.117:6789/0 conn(0x7fbf380b32e0 legacy=0x7fbf38071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3cfb3700 1 -- 172.21.15.81:0/3739537221 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3038775390 0 0) 0x7fbf240035d0 con 0x7fbf38161470 2024-05-08T23:31:51.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3e7b6700 1 -- 172.21.15.81:0/3739537221 >> v1:172.21.15.81:6789/0 conn(0x7fbf38161470 legacy=0x7fbf3806c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3e7b6700 1 -- 172.21.15.81:0/3739537221 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf380ac890 con 0x7fbf38071a50 2024-05-08T23:31:51.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3e7b6700 1 --2- 172.21.15.81:0/3739537221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf38071a50 0x7fbf380ad370 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009d90 tx=0x7fbf3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e93f0e44b0a59a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:51.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3cfb3700 1 -- 172.21.15.81:0/3739537221 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3000e070 con 0x7fbf38071a50 2024-05-08T23:31:51.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3cfb3700 1 -- 172.21.15.81:0/3739537221 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf30003a40 con 0x7fbf38071a50 2024-05-08T23:31:51.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3cfb3700 1 -- 172.21.15.81:0/3739537221 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34001670 con 0x7fbf38071a50 2024-05-08T23:31:51.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3efb7700 1 -- 172.21.15.81:0/3739537221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf38071a50 msgr2=0x7fbf380ad370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3efb7700 1 --2- 172.21.15.81:0/3739537221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf38071a50 0x7fbf380ad370 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009d90 tx=0x7fbf3000b330 comp rx=0 tx=0).stop 2024-05-08T23:31:51.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3efb7700 1 -- 172.21.15.81:0/3739537221 shutdown_connections 2024-05-08T23:31:51.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3efb7700 1 --2- 172.21.15.81:0/3739537221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf38071a50 0x7fbf380ad370 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3efb7700 1 -- 172.21.15.81:0/3739537221 >> 172.21.15.81:0/3739537221 conn(0x7fbf380b0de0 msgr2=0x7fbf380b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:51.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3efb7700 1 -- 172.21.15.81:0/3739537221 shutdown_connections 2024-05-08T23:31:51.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.480+0000 7fbf3efb7700 1 -- 172.21.15.81:0/3739537221 wait complete. 2024-05-08T23:31:51.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.484+0000 7fbf3efb7700 1 Processor -- start 2024-05-08T23:31:51.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.484+0000 7fbf3efb7700 1 -- start start 2024-05-08T23:31:51.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3efb7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf38071a50 0x7fbf380ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3efb7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf380b32e0 0x7fbf38164140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3efb7700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf38161470 0x7fbf380cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3efb7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf38161ac0 con 0x7fbf380b32e0 2024-05-08T23:31:51.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3efb7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf38162640 con 0x7fbf38071a50 2024-05-08T23:31:51.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3efb7700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf380ce710 con 0x7fbf38161470 2024-05-08T23:31:51.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3d7b4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf380b32e0 0x7fbf38164140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3d7b4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf380b32e0 0x7fbf38164140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58484/0 (socket says 172.21.15.81:58484) 2024-05-08T23:31:51.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3d7b4700 1 -- 172.21.15.81:0/2625972075 learned_addr learned my addr 172.21.15.81:0/2625972075 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:51.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3dfb5700 1 --2- 172.21.15.81:0/2625972075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf38071a50 0x7fbf380ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3e7b6700 1 --2- 172.21.15.81:0/2625972075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf38161470 0x7fbf380cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3d7b4700 1 -- 172.21.15.81:0/2625972075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf38161470 msgr2=0x7fbf380cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3d7b4700 1 --2- 172.21.15.81:0/2625972075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf38161470 0x7fbf380cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3d7b4700 1 -- 172.21.15.81:0/2625972075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf38071a50 msgr2=0x7fbf380ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3d7b4700 1 --2- 172.21.15.81:0/2625972075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf38071a50 0x7fbf380ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3d7b4700 1 -- 172.21.15.81:0/2625972075 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf30009a70 con 0x7fbf380b32e0 2024-05-08T23:31:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3d7b4700 1 --2- 172.21.15.81:0/2625972075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf380b32e0 0x7fbf38164140 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800c760 tx=0x7fbf2800cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf2e7fc700 1 -- 172.21.15.81:0/2625972075 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf280052d0 con 0x7fbf380b32e0 2024-05-08T23:31:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf2e7fc700 1 -- 172.21.15.81:0/2625972075 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf28005430 con 0x7fbf380b32e0 2024-05-08T23:31:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3efb7700 1 -- 172.21.15.81:0/2625972075 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf38161d10 con 0x7fbf380b32e0 2024-05-08T23:31:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf2e7fc700 1 -- 172.21.15.81:0/2625972075 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf280055a0 con 0x7fbf380b32e0 2024-05-08T23:31:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3efb7700 1 -- 172.21.15.81:0/2625972075 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf381622d0 con 0x7fbf380b32e0 2024-05-08T23:31:51.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.488+0000 7fbf3efb7700 1 -- 172.21.15.81:0/2625972075 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf381627d0 con 0x7fbf380b32e0 2024-05-08T23:31:51.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.492+0000 7fbf2e7fc700 1 -- 172.21.15.81:0/2625972075 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbf280038e0 con 0x7fbf380b32e0 2024-05-08T23:31:51.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.492+0000 7fbf2e7fc700 1 --2- 172.21.15.81:0/2625972075 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbf1c066730 0x7fbf1c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.492+0000 7fbf2e7fc700 1 -- 172.21.15.81:0/2625972075 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fbf280837f0 con 0x7fbf380b32e0 2024-05-08T23:31:51.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.492+0000 7fbf3dfb5700 1 --2- 172.21.15.81:0/2625972075 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbf1c066730 0x7fbf1c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.492+0000 7fbf3dfb5700 1 --2- 172.21.15.81:0/2625972075 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbf1c066730 0x7fbf1c068bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbf3809cda0 tx=0x7fbf34001be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:51.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.496+0000 7fbf2e7fc700 1 -- 172.21.15.81:0/2625972075 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbf280509c0 con 0x7fbf380b32e0 2024-05-08T23:31:51.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.652+0000 7fbf3efb7700 1 -- 172.21.15.81:0/2625972075 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbf380684b0 con 0x7fbf380b32e0 2024-05-08T23:31:51.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.652+0000 7fbf2e7fc700 1 -- 172.21.15.81:0/2625972075 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbf28053fe0 con 0x7fbf380b32e0 2024-05-08T23:31:51.655 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:31:51.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.656+0000 7fbf3efb7700 1 -- 172.21.15.81:0/2625972075 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbf1c066730 msgr2=0x7fbf1c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.656+0000 7fbf3efb7700 1 --2- 172.21.15.81:0/2625972075 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbf1c066730 0x7fbf1c068bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbf3809cda0 tx=0x7fbf34001be0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.656+0000 7fbf3efb7700 1 -- 172.21.15.81:0/2625972075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf380b32e0 msgr2=0x7fbf38164140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.656+0000 7fbf3efb7700 1 --2- 172.21.15.81:0/2625972075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf380b32e0 0x7fbf38164140 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800c760 tx=0x7fbf2800cb20 comp rx=0 tx=0).stop 2024-05-08T23:31:51.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.660+0000 7fbf3efb7700 1 -- 172.21.15.81:0/2625972075 shutdown_connections 2024-05-08T23:31:51.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.660+0000 7fbf3efb7700 1 --2- 172.21.15.81:0/2625972075 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbf1c066730 0x7fbf1c068bf0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.660+0000 7fbf3efb7700 1 --2- 172.21.15.81:0/2625972075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf38161470 0x7fbf380cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.660+0000 7fbf3efb7700 1 --2- 172.21.15.81:0/2625972075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf380b32e0 0x7fbf38164140 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.660+0000 7fbf3efb7700 1 --2- 172.21.15.81:0/2625972075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf38071a50 0x7fbf380ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.660+0000 7fbf3efb7700 1 -- 172.21.15.81:0/2625972075 >> 172.21.15.81:0/2625972075 conn(0x7fbf380b0de0 msgr2=0x7fbf380b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:51.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.660+0000 7fbf3efb7700 1 -- 172.21.15.81:0/2625972075 shutdown_connections 2024-05-08T23:31:51.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.660+0000 7fbf3efb7700 1 -- 172.21.15.81:0/2625972075 wait complete. 2024-05-08T23:31:51.675 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-05-08T23:31:51.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.784+0000 7f5383fff700 1 Processor -- start 2024-05-08T23:31:51.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.784+0000 7f5383fff700 1 -- start start 2024-05-08T23:31:51.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5383fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5384157b30 0x7f538415e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5383fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53840abda0 0x7f53840ad820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5383fff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538415a350 con 0x7f53840add60 2024-05-08T23:31:51.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5383fff700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538415e980 con 0x7f5384157b30 2024-05-08T23:31:51.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5383fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538415b370 con 0x7f53840abda0 2024-05-08T23:31:51.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5382ffd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5384157b30 0x7f538415e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f53837fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f53840add60 0x7f53840b1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:48140/0 (socket says 172.21.15.81:48140) 2024-05-08T23:31:51.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f53837fe700 1 -- 172.21.15.81:0/1183241294 learned_addr learned my addr 172.21.15.81:0/1183241294 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:51.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f53827fc700 1 --2- 172.21.15.81:0/1183241294 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53840abda0 0x7f53840ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5382ffd700 1 -- 172.21.15.81:0/1183241294 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53840abda0 msgr2=0x7f53840ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5382ffd700 1 --2- 172.21.15.81:0/1183241294 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53840abda0 0x7f53840ad820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5382ffd700 1 -- 172.21.15.81:0/1183241294 >> v1:172.21.15.81:6789/0 conn(0x7f53840add60 legacy=0x7f53840b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5382ffd700 1 -- 172.21.15.81:0/1183241294 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538415a5b0 con 0x7f5384157b30 2024-05-08T23:31:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5381ffb700 1 -- 172.21.15.81:0/1183241294 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394421787 0 0) 0x7f538415a350 con 0x7f53840add60 2024-05-08T23:31:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5382ffd700 1 --2- 172.21.15.81:0/1183241294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5384157b30 0x7f538415e350 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5374009850 tx=0x7f5374009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b625b5fdf30a5d73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5381ffb700 1 -- 172.21.15.81:0/1183241294 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374011830 con 0x7f5384157b30 2024-05-08T23:31:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5381ffb700 1 -- 172.21.15.81:0/1183241294 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5374011990 con 0x7f5384157b30 2024-05-08T23:31:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5381ffb700 1 -- 172.21.15.81:0/1183241294 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374011d40 con 0x7f5384157b30 2024-05-08T23:31:51.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5383fff700 1 -- 172.21.15.81:0/1183241294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5384157b30 msgr2=0x7f538415e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5383fff700 1 --2- 172.21.15.81:0/1183241294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5384157b30 0x7f538415e350 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5374009850 tx=0x7f5374009b60 comp rx=0 tx=0).stop 2024-05-08T23:31:51.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5383fff700 1 -- 172.21.15.81:0/1183241294 shutdown_connections 2024-05-08T23:31:51.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5383fff700 1 --2- 172.21.15.81:0/1183241294 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53840abda0 0x7f53840ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5383fff700 1 --2- 172.21.15.81:0/1183241294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5384157b30 0x7f538415e350 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5383fff700 1 -- 172.21.15.81:0/1183241294 >> 172.21.15.81:0/1183241294 conn(0x7f53840b0de0 msgr2=0x7f53840b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:51.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5383fff700 1 -- 172.21.15.81:0/1183241294 shutdown_connections 2024-05-08T23:31:51.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.788+0000 7f5383fff700 1 -- 172.21.15.81:0/1183241294 wait complete. 2024-05-08T23:31:51.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f5383fff700 1 Processor -- start 2024-05-08T23:31:51.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f5383fff700 1 -- start start 2024-05-08T23:31:51.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f5383fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53840abda0 0x7f53840ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f5383fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53840add60 0x7f538409af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f5383fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f538409b470 0x7f5384149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f5383fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5384156c40 con 0x7f538409b470 2024-05-08T23:31:51.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f5383fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53840af8a0 con 0x7f53840abda0 2024-05-08T23:31:51.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f5383fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53840aed20 con 0x7f53840add60 2024-05-08T23:31:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f53837fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f538409b470 0x7f5384149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f53837fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f538409b470 0x7f5384149180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58490/0 (socket says 172.21.15.81:58490) 2024-05-08T23:31:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f53837fe700 1 -- 172.21.15.81:0/3551306825 learned_addr learned my addr 172.21.15.81:0/3551306825 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f53827fc700 1 --2- 172.21.15.81:0/3551306825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53840add60 0x7f538409af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f5382ffd700 1 --2- 172.21.15.81:0/3551306825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53840abda0 0x7f53840ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f53837fe700 1 -- 172.21.15.81:0/3551306825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53840add60 msgr2=0x7f538409af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f53837fe700 1 --2- 172.21.15.81:0/3551306825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53840add60 0x7f538409af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f53837fe700 1 -- 172.21.15.81:0/3551306825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53840abda0 msgr2=0x7f53840ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f53837fe700 1 --2- 172.21.15.81:0/3551306825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53840abda0 0x7f53840ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f53837fe700 1 -- 172.21.15.81:0/3551306825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53740094b0 con 0x7f538409b470 2024-05-08T23:31:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f53837fe700 1 --2- 172.21.15.81:0/3551306825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f538409b470 0x7f5384149180 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f537c00c460 tx=0x7f537c00c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:51.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.792+0000 7f537b7fe700 1 -- 172.21.15.81:0/3551306825 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c004ca0 con 0x7f538409b470 2024-05-08T23:31:51.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.796+0000 7f537b7fe700 1 -- 172.21.15.81:0/3551306825 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537c004e20 con 0x7f538409b470 2024-05-08T23:31:51.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.796+0000 7f5383fff700 1 -- 172.21.15.81:0/3551306825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53840af000 con 0x7f538409b470 2024-05-08T23:31:51.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.796+0000 7f537b7fe700 1 -- 172.21.15.81:0/3551306825 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c007310 con 0x7f538409b470 2024-05-08T23:31:51.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.796+0000 7f5383fff700 1 -- 172.21.15.81:0/3551306825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53840af5c0 con 0x7f538409b470 2024-05-08T23:31:51.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.796+0000 7f5383fff700 1 -- 172.21.15.81:0/3551306825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5384153970 con 0x7f538409b470 2024-05-08T23:31:51.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.796+0000 7f537b7fe700 1 -- 172.21.15.81:0/3551306825 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f537c007470 con 0x7f538409b470 2024-05-08T23:31:51.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.796+0000 7f537b7fe700 1 --2- 172.21.15.81:0/3551306825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5368066730 0x7f5368068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:51.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.800+0000 7f537b7fe700 1 -- 172.21.15.81:0/3551306825 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f537c083d80 con 0x7f538409b470 2024-05-08T23:31:51.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.800+0000 7f5382ffd700 1 --2- 172.21.15.81:0/3551306825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5368066730 0x7f5368068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:51.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.800+0000 7f5382ffd700 1 --2- 172.21.15.81:0/3551306825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5368066730 0x7f5368068bf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f538406faf0 tx=0x7f5374008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:51.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.804+0000 7f537b7fe700 1 -- 172.21.15.81:0/3551306825 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5384153970 con 0x7f538409b470 2024-05-08T23:31:51.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.972+0000 7f5383fff700 1 -- 172.21.15.81:0/3551306825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f5384153950 con 0x7f538409b470 2024-05-08T23:31:51.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.972+0000 7f537b7fe700 1 -- 172.21.15.81:0/3551306825 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v39) v1 ==== 97+0+11 (secure 0 0 0) 0x7f537c050ff0 con 0x7f538409b470 2024-05-08T23:31:51.974 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:31:51.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.976+0000 7f5383fff700 1 -- 172.21.15.81:0/3551306825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5368066730 msgr2=0x7f5368068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.976+0000 7f5383fff700 1 --2- 172.21.15.81:0/3551306825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5368066730 0x7f5368068bf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f538406faf0 tx=0x7f5374008030 comp rx=0 tx=0).stop 2024-05-08T23:31:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.976+0000 7f5383fff700 1 -- 172.21.15.81:0/3551306825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f538409b470 msgr2=0x7f5384149180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.976+0000 7f5383fff700 1 --2- 172.21.15.81:0/3551306825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f538409b470 0x7f5384149180 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f537c00c460 tx=0x7f537c00c770 comp rx=0 tx=0).stop 2024-05-08T23:31:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.976+0000 7f5383fff700 1 -- 172.21.15.81:0/3551306825 shutdown_connections 2024-05-08T23:31:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.976+0000 7f5383fff700 1 --2- 172.21.15.81:0/3551306825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5368066730 0x7f5368068bf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.976+0000 7f5383fff700 1 --2- 172.21.15.81:0/3551306825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f538409b470 0x7f5384149180 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.976+0000 7f5383fff700 1 --2- 172.21.15.81:0/3551306825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53840add60 0x7f538409af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.976+0000 7f5383fff700 1 --2- 172.21.15.81:0/3551306825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53840abda0 0x7f53840ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.976+0000 7f5383fff700 1 -- 172.21.15.81:0/3551306825 >> 172.21.15.81:0/3551306825 conn(0x7f53840b0de0 msgr2=0x7f53840b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.976+0000 7f5383fff700 1 -- 172.21.15.81:0/3551306825 shutdown_connections 2024-05-08T23:31:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:51.976+0000 7f5383fff700 1 -- 172.21.15.81:0/3551306825 wait complete. 2024-05-08T23:31:51.988 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:31:52.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.096+0000 7f79b7ba7700 1 Processor -- start 2024-05-08T23:31:52.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.096+0000 7f79b7ba7700 1 -- start start 2024-05-08T23:31:52.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b7ba7700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f79b0146b30 0x7f79b00b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:52.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b7ba7700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f79b00b3a10 0x7f79b0071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:52.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b7ba7700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b0160ea0 con 0x7f79b00b2c40 2024-05-08T23:31:52.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b7ba7700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b0146f10 con 0x7f79b00b3a10 2024-05-08T23:31:52.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b7ba7700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b0161160 con 0x7f79b0146b30 2024-05-08T23:31:52.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b6144700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f79b00b3a10 0x7f79b0071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:52.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b6144700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f79b00b3a10 0x7f79b0071110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58492/0 (socket says 172.21.15.81:58492) 2024-05-08T23:31:52.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b6144700 1 -- 172.21.15.81:0/4242678390 learned_addr learned my addr 172.21.15.81:0/4242678390 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:52.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b5943700 1 --2- 172.21.15.81:0/4242678390 >> v2:172.21.15.117:3300/0 conn(0x7f79b0146b30 0x7f79b00b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:52.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b6144700 1 -- 172.21.15.81:0/4242678390 >> v2:172.21.15.117:3300/0 conn(0x7f79b0146b30 msgr2=0x7f79b00b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:52.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b6144700 1 --2- 172.21.15.81:0/4242678390 >> v2:172.21.15.117:3300/0 conn(0x7f79b0146b30 0x7f79b00b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b6144700 1 -- 172.21.15.81:0/4242678390 >> v1:172.21.15.117:6789/0 conn(0x7f79b00b2c40 legacy=0x7f79b00b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:52.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b6144700 1 -- 172.21.15.81:0/4242678390 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b0161440 con 0x7f79b00b3a10 2024-05-08T23:31:52.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b6144700 1 --2- 172.21.15.81:0/4242678390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79b00b3a10 0x7f79b0071110 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f79b006fac0 tx=0x7f79a8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5265d9dd6feeada5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:52.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b4941700 1 -- 172.21.15.81:0/4242678390 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a8011830 con 0x7f79b00b3a10 2024-05-08T23:31:52.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b4941700 1 -- 172.21.15.81:0/4242678390 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a8011990 con 0x7f79b00b3a10 2024-05-08T23:31:52.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b4941700 1 -- 172.21.15.81:0/4242678390 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a8011d20 con 0x7f79b00b3a10 2024-05-08T23:31:52.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b7ba7700 1 -- 172.21.15.81:0/4242678390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79b00b3a10 msgr2=0x7f79b0071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:52.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b7ba7700 1 --2- 172.21.15.81:0/4242678390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79b00b3a10 0x7f79b0071110 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f79b006fac0 tx=0x7f79a8009bf0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b7ba7700 1 -- 172.21.15.81:0/4242678390 shutdown_connections 2024-05-08T23:31:52.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b7ba7700 1 --2- 172.21.15.81:0/4242678390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79b00b3a10 0x7f79b0071110 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b7ba7700 1 --2- 172.21.15.81:0/4242678390 >> v2:172.21.15.117:3300/0 conn(0x7f79b0146b30 0x7f79b00b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b7ba7700 1 -- 172.21.15.81:0/4242678390 >> 172.21.15.81:0/4242678390 conn(0x7f79b006a360 msgr2=0x7f79b006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:52.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b7ba7700 1 -- 172.21.15.81:0/4242678390 shutdown_connections 2024-05-08T23:31:52.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.100+0000 7f79b7ba7700 1 -- 172.21.15.81:0/4242678390 wait complete. 2024-05-08T23:31:52.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.104+0000 7f79b7ba7700 1 Processor -- start 2024-05-08T23:31:52.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.104+0000 7f79b7ba7700 1 -- start start 2024-05-08T23:31:52.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.104+0000 7f79b7ba7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b00b2c40 0x7f79b013f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:52.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.104+0000 7f79b7ba7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79b00b3a10 0x7f79b015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:52.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.104+0000 7f79b7ba7700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79b0146b30 0x7f79b0158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:52.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.104+0000 7f79b7ba7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01f72c0 con 0x7f79b00b3a10 2024-05-08T23:31:52.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.104+0000 7f79b7ba7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01f6410 con 0x7f79b00b2c40 2024-05-08T23:31:52.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.104+0000 7f79b7ba7700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0158270 con 0x7f79b0146b30 2024-05-08T23:31:52.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.104+0000 7f79b5142700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79b00b3a10 0x7f79b015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:52.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.104+0000 7f79b5142700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79b00b3a10 0x7f79b015a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58502/0 (socket says 172.21.15.81:58502) 2024-05-08T23:31:52.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.104+0000 7f79b5142700 1 -- 172.21.15.81:0/1720160733 learned_addr learned my addr 172.21.15.81:0/1720160733 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:52.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.108+0000 7f79b6144700 1 --2- 172.21.15.81:0/1720160733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79b0146b30 0x7f79b0158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:52.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.108+0000 7f79b5943700 1 --2- 172.21.15.81:0/1720160733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b00b2c40 0x7f79b013f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:52.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.108+0000 7f79b5142700 1 -- 172.21.15.81:0/1720160733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79b0146b30 msgr2=0x7f79b0158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:52.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.108+0000 7f79b5142700 1 --2- 172.21.15.81:0/1720160733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79b0146b30 0x7f79b0158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.108+0000 7f79b5142700 1 -- 172.21.15.81:0/1720160733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b00b2c40 msgr2=0x7f79b013f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:52.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.108+0000 7f79b5142700 1 --2- 172.21.15.81:0/1720160733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b00b2c40 0x7f79b013f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.108+0000 7f79b5142700 1 -- 172.21.15.81:0/1720160733 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a8009570 con 0x7f79b00b3a10 2024-05-08T23:31:52.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.108+0000 7f79b5943700 1 --2- 172.21.15.81:0/1720160733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b00b2c40 0x7f79b013f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:52.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.108+0000 7f79b5142700 1 --2- 172.21.15.81:0/1720160733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79b00b3a10 0x7f79b015a770 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f79a400cbd0 tx=0x7f79a400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:52.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.108+0000 7f79ae7fc700 1 -- 172.21.15.81:0/1720160733 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4003c20 con 0x7f79b00b3a10 2024-05-08T23:31:52.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.108+0000 7f79ae7fc700 1 -- 172.21.15.81:0/1720160733 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a4003d80 con 0x7f79b00b3a10 2024-05-08T23:31:52.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.108+0000 7f79b7ba7700 1 -- 172.21.15.81:0/1720160733 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b01f66b0 con 0x7f79b00b3a10 2024-05-08T23:31:52.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.108+0000 7f79ae7fc700 1 -- 172.21.15.81:0/1720160733 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a400ab40 con 0x7f79b00b3a10 2024-05-08T23:31:52.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.108+0000 7f79b7ba7700 1 -- 172.21.15.81:0/1720160733 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b01f6c70 con 0x7f79b00b3a10 2024-05-08T23:31:52.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.108+0000 7f79b7ba7700 1 -- 172.21.15.81:0/1720160733 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b00684b0 con 0x7f79b00b3a10 2024-05-08T23:31:52.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.112+0000 7f79ae7fc700 1 -- 172.21.15.81:0/1720160733 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f79a400aca0 con 0x7f79b00b3a10 2024-05-08T23:31:52.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.112+0000 7f79ae7fc700 1 --2- 172.21.15.81:0/1720160733 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f799c066730 0x7f799c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:52.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.112+0000 7f79ae7fc700 1 -- 172.21.15.81:0/1720160733 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f79a40842d0 con 0x7f79b00b3a10 2024-05-08T23:31:52.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.112+0000 7f79b5943700 1 --2- 172.21.15.81:0/1720160733 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f799c066730 0x7f799c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:52.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.112+0000 7f79b5943700 1 --2- 172.21.15.81:0/1720160733 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f799c066730 0x7f799c068bf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f79b01605d0 tx=0x7f79a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:52.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.120+0000 7f79ae7fc700 1 -- 172.21.15.81:0/1720160733 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f79a4051540 con 0x7f79b00b3a10 2024-05-08T23:31:52.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.304+0000 7f79b7ba7700 1 -- 172.21.15.81:0/1720160733 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f79b00b30e0 con 0x7f79b00b3a10 2024-05-08T23:31:52.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.308+0000 7f79ae7fc700 1 -- 172.21.15.81:0/1720160733 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) v1 ==== 80+0+0 (secure 0 0 0) 0x7f79a4054b60 con 0x7f79b00b3a10 2024-05-08T23:31:52.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.312+0000 7f79b7ba7700 1 -- 172.21.15.81:0/1720160733 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f799c066730 msgr2=0x7f799c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:52.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.312+0000 7f79b7ba7700 1 --2- 172.21.15.81:0/1720160733 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f799c066730 0x7f799c068bf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f79b01605d0 tx=0x7f79a0008040 comp rx=0 tx=0).stop 2024-05-08T23:31:52.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.312+0000 7f79b7ba7700 1 -- 172.21.15.81:0/1720160733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79b00b3a10 msgr2=0x7f79b015a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:52.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.312+0000 7f79b7ba7700 1 --2- 172.21.15.81:0/1720160733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79b00b3a10 0x7f79b015a770 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f79a400cbd0 tx=0x7f79a400cee0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.312+0000 7f79b7ba7700 1 -- 172.21.15.81:0/1720160733 shutdown_connections 2024-05-08T23:31:52.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.312+0000 7f79b7ba7700 1 --2- 172.21.15.81:0/1720160733 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f799c066730 0x7f799c068bf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.312+0000 7f79b7ba7700 1 --2- 172.21.15.81:0/1720160733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79b0146b30 0x7f79b0158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.312+0000 7f79b7ba7700 1 --2- 172.21.15.81:0/1720160733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79b00b3a10 0x7f79b015a770 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.312+0000 7f79b7ba7700 1 --2- 172.21.15.81:0/1720160733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b00b2c40 0x7f79b013f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.312+0000 7f79b7ba7700 1 -- 172.21.15.81:0/1720160733 >> 172.21.15.81:0/1720160733 conn(0x7f79b006a360 msgr2=0x7f79b0142c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:52.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.312+0000 7f79b7ba7700 1 -- 172.21.15.81:0/1720160733 shutdown_connections 2024-05-08T23:31:52.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:52.312+0000 7f79b7ba7700 1 -- 172.21.15.81:0/1720160733 wait complete. 2024-05-08T23:31:52.313 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:31:52.328 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2024-05-08T23:31:52.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:52.328 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T23:31:52.339 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[36330]: fuse finished with error 0 and tester_r 0 2024-05-08T23:31:52.341 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:31:58.443 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:31:58.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:58.443 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:31:58.451 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:31:58.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.572+0000 7f2155c2d700 1 Processor -- start 2024-05-08T23:31:58.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.572+0000 7f2155c2d700 1 -- start start 2024-05-08T23:31:58.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.572+0000 7f2155c2d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21500d1690 0x7f21500d37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:58.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.572+0000 7f2155c2d700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21500ce870 con 0x7f21500b2ea0 2024-05-08T23:31:58.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.572+0000 7f2155c2d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21500ceb50 con 0x7f21500d0f20 2024-05-08T23:31:58.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.572+0000 7f2155c2d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21500d3e50 con 0x7f21500d1690 2024-05-08T23:31:58.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.572+0000 7f215542c700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f21500b2ea0 0x7f21500b3330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47142/0 (socket says 172.21.15.81:47142) 2024-05-08T23:31:58.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.572+0000 7f215542c700 1 -- 172.21.15.81:0/3191315655 learned_addr learned my addr 172.21.15.81:0/3191315655 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:31:58.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.572+0000 7f2154c2b700 1 --2- 172.21.15.81:0/3191315655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21500d1690 0x7f21500d37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:58.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.572+0000 7f214f7fe700 1 -- 172.21.15.81:0/3191315655 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800167404 0 0) 0x7f21500ce870 con 0x7f21500b2ea0 2024-05-08T23:31:58.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.572+0000 7f214f7fe700 1 -- 172.21.15.81:0/3191315655 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f213c0035d0 con 0x7f21500b2ea0 2024-05-08T23:31:58.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.576+0000 7f214f7fe700 1 -- 172.21.15.81:0/3191315655 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996411241 0 0) 0x7f21500ceb50 con 0x7f21500d0f20 2024-05-08T23:31:58.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.576+0000 7f2154c2b700 1 -- 172.21.15.81:0/3191315655 >> v1:172.21.15.117:6789/0 conn(0x7f21500d0f20 legacy=0x7f21500d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:58.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.576+0000 7f2154c2b700 1 -- 172.21.15.81:0/3191315655 >> v1:172.21.15.81:6789/0 conn(0x7f21500b2ea0 legacy=0x7f21500b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:58.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.576+0000 7f2154c2b700 1 -- 172.21.15.81:0/3191315655 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21500cee30 con 0x7f21500d1690 2024-05-08T23:31:58.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.576+0000 7f2154c2b700 1 --2- 172.21.15.81:0/3191315655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21500d1690 0x7f21500d37c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2148009880 tx=0x7f2148009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42e4bcc062b61ab6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:58.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.576+0000 7f214f7fe700 1 -- 172.21.15.81:0/3191315655 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2148011830 con 0x7f21500d1690 2024-05-08T23:31:58.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.576+0000 7f214f7fe700 1 -- 172.21.15.81:0/3191315655 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2148011990 con 0x7f21500d1690 2024-05-08T23:31:58.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.576+0000 7f214f7fe700 1 -- 172.21.15.81:0/3191315655 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2148011d00 con 0x7f21500d1690 2024-05-08T23:31:58.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.576+0000 7f2155c2d700 1 -- 172.21.15.81:0/3191315655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21500d1690 msgr2=0x7f21500d37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:58.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.576+0000 7f2155c2d700 1 --2- 172.21.15.81:0/3191315655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21500d1690 0x7f21500d37c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2148009880 tx=0x7f2148009b90 comp rx=0 tx=0).stop 2024-05-08T23:31:58.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.576+0000 7f2155c2d700 1 -- 172.21.15.81:0/3191315655 shutdown_connections 2024-05-08T23:31:58.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.576+0000 7f2155c2d700 1 --2- 172.21.15.81:0/3191315655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21500d1690 0x7f21500d37c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:58.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.576+0000 7f2155c2d700 1 -- 172.21.15.81:0/3191315655 >> 172.21.15.81:0/3191315655 conn(0x7f215009c3c0 msgr2=0x7f21500d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:58.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.576+0000 7f2155c2d700 1 -- 172.21.15.81:0/3191315655 shutdown_connections 2024-05-08T23:31:58.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.576+0000 7f2155c2d700 1 -- 172.21.15.81:0/3191315655 wait complete. 2024-05-08T23:31:58.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.580+0000 7f2155c2d700 1 Processor -- start 2024-05-08T23:31:58.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.580+0000 7f2155c2d700 1 -- start start 2024-05-08T23:31:58.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.580+0000 7f2155c2d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21500b2ea0 0x7f21501525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:58.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.580+0000 7f2155c2d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21500d0f20 0x7f215015f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:58.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.580+0000 7f2155c2d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21500d1690 0x7f21500b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:58.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.580+0000 7f2155c2d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21500ca180 con 0x7f21500b2ea0 2024-05-08T23:31:58.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.580+0000 7f2155c2d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21500ca440 con 0x7f21500d1690 2024-05-08T23:31:58.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.580+0000 7f2155c2d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21500ca720 con 0x7f21500d0f20 2024-05-08T23:31:58.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.580+0000 7f2154c2b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21500b2ea0 0x7f21501525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:58.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.580+0000 7f2154c2b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21500b2ea0 0x7f21501525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53196/0 (socket says 172.21.15.81:53196) 2024-05-08T23:31:58.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.580+0000 7f2154c2b700 1 -- 172.21.15.81:0/3492357330 learned_addr learned my addr 172.21.15.81:0/3492357330 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:31:58.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.580+0000 7f214ffff700 1 --2- 172.21.15.81:0/3492357330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21500d0f20 0x7f215015f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:58.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.584+0000 7f215542c700 1 --2- 172.21.15.81:0/3492357330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21500d1690 0x7f21500b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:58.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.584+0000 7f2154c2b700 1 -- 172.21.15.81:0/3492357330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21500d0f20 msgr2=0x7f215015f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:58.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.584+0000 7f2154c2b700 1 --2- 172.21.15.81:0/3492357330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21500d0f20 0x7f215015f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:58.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.584+0000 7f2154c2b700 1 -- 172.21.15.81:0/3492357330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21500d1690 msgr2=0x7f21500b66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:58.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.584+0000 7f2154c2b700 1 --2- 172.21.15.81:0/3492357330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21500d1690 0x7f21500b66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:58.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.584+0000 7f2154c2b700 1 -- 172.21.15.81:0/3492357330 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21480094b0 con 0x7f21500b2ea0 2024-05-08T23:31:58.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.584+0000 7f2154c2b700 1 --2- 172.21.15.81:0/3492357330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21500b2ea0 0x7f21501525b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2148013040 tx=0x7f2148004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:58.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.584+0000 7f214d7fa700 1 -- 172.21.15.81:0/3492357330 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21480036c0 con 0x7f21500b2ea0 2024-05-08T23:31:58.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.584+0000 7f214d7fa700 1 -- 172.21.15.81:0/3492357330 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2148003820 con 0x7f21500b2ea0 2024-05-08T23:31:58.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.584+0000 7f2155c2d700 1 -- 172.21.15.81:0/3492357330 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21500caa00 con 0x7f21500b2ea0 2024-05-08T23:31:58.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.584+0000 7f214d7fa700 1 -- 172.21.15.81:0/3492357330 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2148003bd0 con 0x7f21500b2ea0 2024-05-08T23:31:58.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.584+0000 7f2155c2d700 1 -- 172.21.15.81:0/3492357330 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2150152af0 con 0x7f21500b2ea0 2024-05-08T23:31:58.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.584+0000 7f2155c2d700 1 -- 172.21.15.81:0/3492357330 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21500caec0 con 0x7f21500b2ea0 2024-05-08T23:31:58.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.584+0000 7f214d7fa700 1 -- 172.21.15.81:0/3492357330 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2148003d30 con 0x7f21500b2ea0 2024-05-08T23:31:58.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.588+0000 7f214d7fa700 1 --2- 172.21.15.81:0/3492357330 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2130066730 0x7f2130068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:58.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.588+0000 7f214d7fa700 1 -- 172.21.15.81:0/3492357330 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f2148085b00 con 0x7f21500b2ea0 2024-05-08T23:31:58.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.588+0000 7f214ffff700 1 --2- 172.21.15.81:0/3492357330 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2130066730 0x7f2130068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:58.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.588+0000 7f214ffff700 1 --2- 172.21.15.81:0/3492357330 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2130066730 0x7f2130068bf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f21500b2cd0 tx=0x7f2138009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:58.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.592+0000 7f214d7fa700 1 -- 172.21.15.81:0/3492357330 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2148087e60 con 0x7f21500b2ea0 2024-05-08T23:31:58.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.800+0000 7f2155c2d700 1 -- 172.21.15.81:0/3492357330 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f21500684b0 con 0x7f21500b2ea0 2024-05-08T23:31:58.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.800+0000 7f214d7fa700 1 -- 172.21.15.81:0/3492357330 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+82 (secure 0 0 0) 0x7f2148052df0 con 0x7f21500b2ea0 2024-05-08T23:31:58.804 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2024-05-08T23:31:58.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.804+0000 7f2155c2d700 1 -- 172.21.15.81:0/3492357330 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2130066730 msgr2=0x7f2130068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:58.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.804+0000 7f2155c2d700 1 --2- 172.21.15.81:0/3492357330 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2130066730 0x7f2130068bf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f21500b2cd0 tx=0x7f2138009040 comp rx=0 tx=0).stop 2024-05-08T23:31:58.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.804+0000 7f2155c2d700 1 -- 172.21.15.81:0/3492357330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21500b2ea0 msgr2=0x7f21501525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:58.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.804+0000 7f2155c2d700 1 --2- 172.21.15.81:0/3492357330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21500b2ea0 0x7f21501525b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2148013040 tx=0x7f2148004770 comp rx=0 tx=0).stop 2024-05-08T23:31:58.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.804+0000 7f2155c2d700 1 -- 172.21.15.81:0/3492357330 shutdown_connections 2024-05-08T23:31:58.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.804+0000 7f2155c2d700 1 --2- 172.21.15.81:0/3492357330 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2130066730 0x7f2130068bf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:58.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.804+0000 7f2155c2d700 1 --2- 172.21.15.81:0/3492357330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21500d1690 0x7f21500b66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:58.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.804+0000 7f2155c2d700 1 --2- 172.21.15.81:0/3492357330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21500d0f20 0x7f215015f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:58.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.804+0000 7f2155c2d700 1 --2- 172.21.15.81:0/3492357330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21500b2ea0 0x7f21501525b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:58.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.804+0000 7f2155c2d700 1 -- 172.21.15.81:0/3492357330 >> 172.21.15.81:0/3492357330 conn(0x7f215009c3c0 msgr2=0x7f21500d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:58.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.804+0000 7f2155c2d700 1 -- 172.21.15.81:0/3492357330 shutdown_connections 2024-05-08T23:31:58.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:31:58.804+0000 7f2155c2d700 1 -- 172.21.15.81:0/3492357330 wait complete. 2024-05-08T23:31:58.818 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:31:58.818 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:31:58.818 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2024-05-08T23:31:58.818 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:31:58.818 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:31:58.818 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:31:58.819 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:31:58.819 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:31:58.819 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:31:58.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:58.819 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2024-05-08T23:31:58.828 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:31:58.828 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:31:58.828 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:31:58.828 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:58.828 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-05-08T23:31:58.828 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:58.828 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:31:58.828 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:58.828 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:31:58.828 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:58.828 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:31:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:58.829 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:31:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T23:31:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42497sec preferred_lft 42497sec 2024-05-08T23:31:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2024-05-08T23:31:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:58.829 INFO:teuthology.orchestra.run.smithi081.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T23:31:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 02:42:22:62:7a:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T23:31:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:58.829 INFO:teuthology.orchestra.run.smithi081.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:31:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:2b:10:6b:08:e3 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:31:58.830 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:31:58.830 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:58.830 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::48fc:c7ff:fe41:6d94/64 scope link 2024-05-08T23:31:58.830 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:58.830 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:31:58.830 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:2b:10:6b:08:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:31:58.830 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::702b:10ff:fe6b:8e3/64 scope link 2024-05-08T23:31:58.830 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:58.830 INFO:teuthology.orchestra.run.smithi081.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:31:58.831 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:9d:1b:4e:6b:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:31:58.831 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::709d:1bff:fe4e:6bd5/64 scope link 2024-05-08T23:31:58.831 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:31:58.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:58.835 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:31:58.886 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-05-08T23:31:58.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:58.888 DEBUG:teuthology.orchestra.run.smithi081:> route 2024-05-08T23:31:58.894 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2024-05-08T23:31:58.895 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:31:58.895 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T23:31:58.895 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T23:31:58.895 INFO:teuthology.orchestra.run.smithi081.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T23:31:58.895 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T23:31:58.895 INFO:teuthology.orchestra.run.smithi081.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T23:31:58.895 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:31:58.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:58.895 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-05-08T23:31:58.896 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-05-08T23:31:58.896 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T23:31:58.896 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T23:31:58.896 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T23:31:58.896 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-05-08T23:31:58.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:58.933 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-05-08T23:31:58.941 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T23:31:58.941 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:31:58.942 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:31:58.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:58.942 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2024-05-08T23:31:58.957 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T23:31:58.957 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:31:58.957 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:31:58.964 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:31:58.965 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:31:58.965 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:31:58.971 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:31:58.988 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:31:58.988 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:31:59.047 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:31:59.048 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:31:59.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:59.048 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:31:59.053 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T23:31:59.053 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T23:31:59.055 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:31:59.070 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:31:59.071 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:31:59.085 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:31:59.086 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:31:59.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:31:59.086 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:31:59.140 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2024-05-08T23:31:59.140+0000 7f39ce1c2640 -1 init, newargv = 0x55e425f6af70 newargc=15 2024-05-08T23:31:59.140 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[36914]: starting ceph client 2024-05-08T23:31:59.149 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[36914]: starting fuse 2024-05-08T23:32:00.235 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:32:00.255 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:32:00.255 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:32:00.271 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:32:00.271 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:32:00.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:00.272 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:32:00.322 INFO:teuthology.orchestra.run.smithi081.stdout:52 2024-05-08T23:32:00.322 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> import re 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> import os 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2024-05-08T23:32:00.323 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:32:00.324 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2024-05-08T23:32:00.324 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T23:32:00.324 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T23:32:00.324 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T23:32:00.324 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2024-05-08T23:32:00.324 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2024-05-08T23:32:00.324 DEBUG:teuthology.orchestra.run.smithi081:> return f 2024-05-08T23:32:00.324 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T23:32:00.324 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:32:00.324 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2024-05-08T23:32:00.324 DEBUG:teuthology.orchestra.run.smithi081:> ' 2024-05-08T23:32:00.404 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.36914.asok 2024-05-08T23:32:00.590 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36914.asok 2024-05-08T23:32:00.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:00.590 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36914.asok status 2024-05-08T23:32:00.687 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:32:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2024-05-08T23:32:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T23:32:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:32:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2024-05-08T23:32:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2024-05-08T23:32:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T23:32:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "36914", 2024-05-08T23:32:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2024-05-08T23:32:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2024-05-08T23:32:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2024-05-08T23:32:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 15074, 2024-05-08T23:32:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2024-05-08T23:32:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2024-05-08T23:32:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2024-05-08T23:32:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 15074 2024-05-08T23:32:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:32:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:32:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:32:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3782091392 2024-05-08T23:32:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:32:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:32:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:32:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3782091392 2024-05-08T23:32:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.15074 192.168.0.1:0/3782091392", 2024-05-08T23:32:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/3782091392", 2024-05-08T23:32:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2024-05-08T23:32:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 15, 2024-05-08T23:32:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 39, 2024-05-08T23:32:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2024-05-08T23:32:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2024-05-08T23:32:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2024-05-08T23:32:00.689 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:32:00.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:00.697 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2024-05-08T23:32:00.697 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:00.702 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2024-05-08T23:32:00.703 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:00.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:00.703 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:00.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:00.713 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2024-05-08T23:32:00.741 INFO:teuthology.orchestra.run.smithi081.stderr:setfattr: layoutfile: Permission denied 2024-05-08T23:32:00.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:32:00.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:00.742 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2024-05-08T23:32:00.751 INFO:teuthology.orchestra.run.smithi081.stderr:setfattr: layoutdir: Permission denied 2024-05-08T23:32:00.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:32:00.752 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:32:00.752 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:00.803 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:00.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.920+0000 7f056c814700 1 Processor -- start 2024-05-08T23:32:00.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.920+0000 7f056c814700 1 -- start start 2024-05-08T23:32:00.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056c814700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0568071ab0 0x7f05680ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:00.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056c814700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0568071e90 0x7f056815a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:00.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056c814700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f056815aaa0 0x7f056813d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:00.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056c814700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05680cea50 con 0x7f0568071e90 2024-05-08T23:32:00.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056c814700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05680ce080 con 0x7f0568071ab0 2024-05-08T23:32:00.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056c814700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05680cd7c0 con 0x7f056815aaa0 2024-05-08T23:32:00.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056659c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0568071e90 0x7f056815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:00.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056659c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0568071e90 0x7f056815a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53210/0 (socket says 172.21.15.81:53210) 2024-05-08T23:32:00.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056659c700 1 -- 172.21.15.81:0/3562235451 learned_addr learned my addr 172.21.15.81:0/3562235451 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:00.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056759e700 1 --2- 172.21.15.81:0/3562235451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f056815aaa0 0x7f056813d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:00.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f0566d9d700 1 --2- 172.21.15.81:0/3562235451 >> v2:172.21.15.117:3300/0 conn(0x7f0568071ab0 0x7f05680ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:00.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056659c700 1 -- 172.21.15.81:0/3562235451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f056815aaa0 msgr2=0x7f056813d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:00.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056659c700 1 --2- 172.21.15.81:0/3562235451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f056815aaa0 0x7f056813d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:00.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056659c700 1 -- 172.21.15.81:0/3562235451 >> v2:172.21.15.117:3300/0 conn(0x7f0568071ab0 msgr2=0x7f05680ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:00.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056659c700 1 --2- 172.21.15.81:0/3562235451 >> v2:172.21.15.117:3300/0 conn(0x7f0568071ab0 0x7f05680ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:00.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056659c700 1 -- 172.21.15.81:0/3562235451 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05680cda30 con 0x7f0568071e90 2024-05-08T23:32:00.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056659c700 1 --2- 172.21.15.81:0/3562235451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0568071e90 0x7f056815a560 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f056806eb50 tx=0x7f0558009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79633e1edfa6ed5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:00.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f0565d9b700 1 -- 172.21.15.81:0/3562235451 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558011830 con 0x7f0568071e90 2024-05-08T23:32:00.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f0565d9b700 1 -- 172.21.15.81:0/3562235451 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0558011990 con 0x7f0568071e90 2024-05-08T23:32:00.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f0565d9b700 1 -- 172.21.15.81:0/3562235451 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558011d40 con 0x7f0568071e90 2024-05-08T23:32:00.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056c814700 1 -- 172.21.15.81:0/3562235451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0568071e90 msgr2=0x7f056815a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:00.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056c814700 1 --2- 172.21.15.81:0/3562235451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0568071e90 0x7f056815a560 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f056806eb50 tx=0x7f0558009c20 comp rx=0 tx=0).stop 2024-05-08T23:32:00.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056c814700 1 -- 172.21.15.81:0/3562235451 shutdown_connections 2024-05-08T23:32:00.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056c814700 1 --2- 172.21.15.81:0/3562235451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f056815aaa0 0x7f056813d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:00.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056c814700 1 --2- 172.21.15.81:0/3562235451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0568071e90 0x7f056815a560 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:00.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056c814700 1 --2- 172.21.15.81:0/3562235451 >> v2:172.21.15.117:3300/0 conn(0x7f0568071ab0 0x7f05680ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:00.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056c814700 1 -- 172.21.15.81:0/3562235451 >> 172.21.15.81:0/3562235451 conn(0x7f056806c330 msgr2=0x7f056806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:00.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056c814700 1 -- 172.21.15.81:0/3562235451 shutdown_connections 2024-05-08T23:32:00.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.924+0000 7f056c814700 1 -- 172.21.15.81:0/3562235451 wait complete. 2024-05-08T23:32:00.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f056c814700 1 Processor -- start 2024-05-08T23:32:00.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f056c814700 1 -- start start 2024-05-08T23:32:00.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f056c814700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0568071ab0 0x7f05680b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:00.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f056c814700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0568071e90 0x7f05680b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:00.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f056c814700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f056815aaa0 0x7f05681346f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:00.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f056c814700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05680b1ac0 con 0x7f0568071ab0 2024-05-08T23:32:00.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f056c814700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05680b2080 con 0x7f0568071e90 2024-05-08T23:32:00.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f056c814700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05680b3a80 con 0x7f056815aaa0 2024-05-08T23:32:00.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f0566d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0568071ab0 0x7f05680b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:00.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f0566d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0568071ab0 0x7f05680b2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53220/0 (socket says 172.21.15.81:53220) 2024-05-08T23:32:00.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f0566d9d700 1 -- 172.21.15.81:0/903249946 learned_addr learned my addr 172.21.15.81:0/903249946 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:00.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f056759e700 1 --2- 172.21.15.81:0/903249946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f056815aaa0 0x7f05681346f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:00.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f056659c700 1 --2- 172.21.15.81:0/903249946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0568071e90 0x7f05680b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:00.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f0566d9d700 1 -- 172.21.15.81:0/903249946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f056815aaa0 msgr2=0x7f05681346f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:00.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f0566d9d700 1 --2- 172.21.15.81:0/903249946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f056815aaa0 0x7f05681346f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:00.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f0566d9d700 1 -- 172.21.15.81:0/903249946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0568071e90 msgr2=0x7f05680b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:00.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f0566d9d700 1 --2- 172.21.15.81:0/903249946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0568071e90 0x7f05680b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:00.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f0566d9d700 1 -- 172.21.15.81:0/903249946 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0558009570 con 0x7f0568071ab0 2024-05-08T23:32:00.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f0566d9d700 1 --2- 172.21.15.81:0/903249946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0568071ab0 0x7f05680b2510 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f055000d830 tx=0x7f055000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:00.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f055f7fe700 1 -- 172.21.15.81:0/903249946 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05500125f0 con 0x7f0568071ab0 2024-05-08T23:32:00.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f056c814700 1 -- 172.21.15.81:0/903249946 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0568150b50 con 0x7f0568071ab0 2024-05-08T23:32:00.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f055f7fe700 1 -- 172.21.15.81:0/903249946 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0550012750 con 0x7f0568071ab0 2024-05-08T23:32:00.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f055f7fe700 1 -- 172.21.15.81:0/903249946 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550012ac0 con 0x7f0568071ab0 2024-05-08T23:32:00.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.928+0000 7f056c814700 1 -- 172.21.15.81:0/903249946 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05680b1d10 con 0x7f0568071ab0 2024-05-08T23:32:00.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.932+0000 7f056c814700 1 -- 172.21.15.81:0/903249946 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05680684b0 con 0x7f0568071ab0 2024-05-08T23:32:00.934 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.932+0000 7f055f7fe700 1 -- 172.21.15.81:0/903249946 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0550012c20 con 0x7f0568071ab0 2024-05-08T23:32:00.934 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.932+0000 7f055f7fe700 1 --2- 172.21.15.81:0/903249946 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0548066760 0x7f0548068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:00.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.932+0000 7f055f7fe700 1 -- 172.21.15.81:0/903249946 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f0550084470 con 0x7f0568071ab0 2024-05-08T23:32:00.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.932+0000 7f056659c700 1 --2- 172.21.15.81:0/903249946 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0548066760 0x7f0548068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:00.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.932+0000 7f056659c700 1 --2- 172.21.15.81:0/903249946 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0548066760 0x7f0548068c20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0558009c60 tx=0x7f0558008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:00.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:00.940+0000 7f055f7fe700 1 -- 172.21.15.81:0/903249946 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f05500515e0 con 0x7f0568071ab0 2024-05-08T23:32:01.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.116+0000 7f056c814700 1 -- 172.21.15.81:0/903249946 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05680ce5a0 con 0x7f0568071ab0 2024-05-08T23:32:01.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.116+0000 7f055f7fe700 1 -- 172.21.15.81:0/903249946 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+17880 (secure 0 0 0) 0x7f0550012ed0 con 0x7f0568071ab0 2024-05-08T23:32:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:01.119 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":39,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:48.310750+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:31:35.620423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:31:36.878524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-05-08T23:31:46.337484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:01.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.120+0000 7f056c814700 1 -- 172.21.15.81:0/903249946 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0548066760 msgr2=0x7f0548068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.120+0000 7f056c814700 1 --2- 172.21.15.81:0/903249946 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0548066760 0x7f0548068c20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0558009c60 tx=0x7f0558008030 comp rx=0 tx=0).stop 2024-05-08T23:32:01.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.120+0000 7f056c814700 1 -- 172.21.15.81:0/903249946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0568071ab0 msgr2=0x7f05680b2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.120+0000 7f056c814700 1 --2- 172.21.15.81:0/903249946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0568071ab0 0x7f05680b2510 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f055000d830 tx=0x7f055000db40 comp rx=0 tx=0).stop 2024-05-08T23:32:01.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.120+0000 7f056c814700 1 -- 172.21.15.81:0/903249946 shutdown_connections 2024-05-08T23:32:01.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.120+0000 7f056c814700 1 --2- 172.21.15.81:0/903249946 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0548066760 0x7f0548068c20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.120+0000 7f056c814700 1 --2- 172.21.15.81:0/903249946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f056815aaa0 0x7f05681346f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.120+0000 7f056c814700 1 --2- 172.21.15.81:0/903249946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0568071e90 0x7f05680b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.120+0000 7f056c814700 1 --2- 172.21.15.81:0/903249946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0568071ab0 0x7f05680b2510 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.120+0000 7f056c814700 1 -- 172.21.15.81:0/903249946 >> 172.21.15.81:0/903249946 conn(0x7f056806c330 msgr2=0x7f0568142690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:01.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.120+0000 7f056c814700 1 -- 172.21.15.81:0/903249946 shutdown_connections 2024-05-08T23:32:01.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.120+0000 7f056c814700 1 -- 172.21.15.81:0/903249946 wait complete. 2024-05-08T23:32:01.135 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:31:35.620423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:31:36.878524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-05-08T23:31:46.337484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:32:01.135 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:01.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5e04918700 1 Processor -- start 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5e04918700 1 -- start start 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5e04918700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5e0006d8c0 0x7f5e0015e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5e04918700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5e000b20f0 0x7f5e00071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5e04918700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e000b3620 0x7f5e00071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5e04918700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e0015e650 con 0x7f5e000b20f0 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5e04918700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e000721c0 con 0x7f5e0006d8c0 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5e04918700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e000b2580 con 0x7f5e000b3620 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dfe59c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5e000b20f0 0x7f5e00071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dfe59c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5e000b20f0 0x7f5e00071340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53224/0 (socket says 172.21.15.81:53224) 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dfe59c700 1 -- 172.21.15.81:0/784366766 learned_addr learned my addr 172.21.15.81:0/784366766 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dfed9d700 1 --2- 172.21.15.81:0/784366766 >> v2:172.21.15.117:3300/0 conn(0x7f5e0006d8c0 0x7f5e0015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dff59e700 1 --2- 172.21.15.81:0/784366766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e000b3620 0x7f5e00071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dfe59c700 1 -- 172.21.15.81:0/784366766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e000b3620 msgr2=0x7f5e00071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dfe59c700 1 --2- 172.21.15.81:0/784366766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e000b3620 0x7f5e00071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dfe59c700 1 -- 172.21.15.81:0/784366766 >> v2:172.21.15.117:3300/0 conn(0x7f5e0006d8c0 msgr2=0x7f5e0015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dfe59c700 1 --2- 172.21.15.81:0/784366766 >> v2:172.21.15.117:3300/0 conn(0x7f5e0006d8c0 0x7f5e0015e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dfe59c700 1 -- 172.21.15.81:0/784366766 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e000b2860 con 0x7f5e000b20f0 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dff59e700 1 --2- 172.21.15.81:0/784366766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e000b3620 0x7f5e00071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dfed9d700 1 --2- 172.21.15.81:0/784366766 >> v2:172.21.15.117:3300/0 conn(0x7f5e0006d8c0 0x7f5e0015e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dfe59c700 1 --2- 172.21.15.81:0/784366766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e000b20f0 0x7f5e00071340 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009910 tx=0x7f5df0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34e352d59d5735c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dfdd9b700 1 -- 172.21.15.81:0/784366766 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df000ba20 con 0x7f5e000b20f0 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dfdd9b700 1 -- 172.21.15.81:0/784366766 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df000bb80 con 0x7f5e000b20f0 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.252+0000 7f5dfdd9b700 1 -- 172.21.15.81:0/784366766 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0009e60 con 0x7f5e000b20f0 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.256+0000 7f5e04918700 1 -- 172.21.15.81:0/784366766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e000b20f0 msgr2=0x7f5e00071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.256+0000 7f5e04918700 1 --2- 172.21.15.81:0/784366766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e000b20f0 0x7f5e00071340 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009910 tx=0x7f5df0009c20 comp rx=0 tx=0).stop 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.256+0000 7f5e04918700 1 -- 172.21.15.81:0/784366766 shutdown_connections 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.256+0000 7f5e04918700 1 --2- 172.21.15.81:0/784366766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e000b3620 0x7f5e00071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.256+0000 7f5e04918700 1 --2- 172.21.15.81:0/784366766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e000b20f0 0x7f5e00071340 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.256+0000 7f5e04918700 1 --2- 172.21.15.81:0/784366766 >> v2:172.21.15.117:3300/0 conn(0x7f5e0006d8c0 0x7f5e0015e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.256+0000 7f5e04918700 1 -- 172.21.15.81:0/784366766 >> 172.21.15.81:0/784366766 conn(0x7f5e000b0de0 msgr2=0x7f5e000b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.256+0000 7f5e04918700 1 -- 172.21.15.81:0/784366766 shutdown_connections 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.256+0000 7f5e04918700 1 -- 172.21.15.81:0/784366766 wait complete. 2024-05-08T23:32:01.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.256+0000 7f5e04918700 1 Processor -- start 2024-05-08T23:32:01.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5e04918700 1 -- start start 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5e04918700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e0006d8c0 0x7f5e001f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5e04918700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e000b3620 0x7f5e0014a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5e04918700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e00146c80 0x7f5e00159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5e04918700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e00147630 con 0x7f5e000b3620 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5e04918700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0014a550 con 0x7f5e0006d8c0 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5e04918700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e00148d70 con 0x7f5e00146c80 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5dfe59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e000b3620 0x7f5e0014a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5dfe59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e000b3620 0x7f5e0014a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53228/0 (socket says 172.21.15.81:53228) 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5dfe59c700 1 -- 172.21.15.81:0/1451033801 learned_addr learned my addr 172.21.15.81:0/1451033801 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5dfed9d700 1 --2- 172.21.15.81:0/1451033801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e0006d8c0 0x7f5e001f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5dff59e700 1 --2- 172.21.15.81:0/1451033801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e00146c80 0x7f5e00159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5dfe59c700 1 -- 172.21.15.81:0/1451033801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e00146c80 msgr2=0x7f5e00159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5dfe59c700 1 --2- 172.21.15.81:0/1451033801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e00146c80 0x7f5e00159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5dfe59c700 1 -- 172.21.15.81:0/1451033801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e0006d8c0 msgr2=0x7f5e001f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5dfe59c700 1 --2- 172.21.15.81:0/1451033801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e0006d8c0 0x7f5e001f5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5dfe59c700 1 -- 172.21.15.81:0/1451033801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df0009570 con 0x7f5e000b3620 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5dfe59c700 1 --2- 172.21.15.81:0/1451033801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e000b3620 0x7f5e0014a010 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5df0013000 tx=0x7f5df0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5de77fe700 1 -- 172.21.15.81:0/1451033801 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0004450 con 0x7f5e000b3620 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5de77fe700 1 -- 172.21.15.81:0/1451033801 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df001f070 con 0x7f5e000b3620 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5e04918700 1 -- 172.21.15.81:0/1451033801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e001f6210 con 0x7f5e000b3620 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5de77fe700 1 -- 172.21.15.81:0/1451033801 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0003900 con 0x7f5e000b3620 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.260+0000 7f5e04918700 1 -- 172.21.15.81:0/1451033801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e001f67d0 con 0x7f5e000b3620 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.264+0000 7f5e04918700 1 -- 172.21.15.81:0/1451033801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e000684b0 con 0x7f5e000b3620 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.264+0000 7f5de77fe700 1 -- 172.21.15.81:0/1451033801 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5df0003a60 con 0x7f5e000b3620 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.264+0000 7f5de77fe700 1 --2- 172.21.15.81:0/1451033801 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5dec066760 0x7f5dec068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.264+0000 7f5de77fe700 1 -- 172.21.15.81:0/1451033801 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f5df0021070 con 0x7f5e000b3620 2024-05-08T23:32:01.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.264+0000 7f5dfed9d700 1 --2- 172.21.15.81:0/1451033801 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5dec066760 0x7f5dec068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.264+0000 7f5dfed9d700 1 --2- 172.21.15.81:0/1451033801 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5dec066760 0x7f5dec068c20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5df40045f0 tx=0x7f5df4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.272+0000 7f5de77fe700 1 -- 172.21.15.81:0/1451033801 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5df00528a0 con 0x7f5e000b3620 2024-05-08T23:32:01.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.440+0000 7f5e04918700 1 -- 172.21.15.81:0/1451033801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e000b0730 con 0x7f5e000b3620 2024-05-08T23:32:01.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.440+0000 7f5de77fe700 1 -- 172.21.15.81:0/1451033801 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5df00045d0 con 0x7f5e000b3620 2024-05-08T23:32:01.443 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:32:01.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.444+0000 7f5e04918700 1 -- 172.21.15.81:0/1451033801 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5dec066760 msgr2=0x7f5dec068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.444+0000 7f5e04918700 1 --2- 172.21.15.81:0/1451033801 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5dec066760 0x7f5dec068c20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5df40045f0 tx=0x7f5df4008040 comp rx=0 tx=0).stop 2024-05-08T23:32:01.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.444+0000 7f5e04918700 1 -- 172.21.15.81:0/1451033801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e000b3620 msgr2=0x7f5e0014a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.444+0000 7f5e04918700 1 --2- 172.21.15.81:0/1451033801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e000b3620 0x7f5e0014a010 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5df0013000 tx=0x7f5df0004770 comp rx=0 tx=0).stop 2024-05-08T23:32:01.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.444+0000 7f5e04918700 1 -- 172.21.15.81:0/1451033801 shutdown_connections 2024-05-08T23:32:01.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.444+0000 7f5e04918700 1 --2- 172.21.15.81:0/1451033801 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5dec066760 0x7f5dec068c20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.444+0000 7f5e04918700 1 --2- 172.21.15.81:0/1451033801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e00146c80 0x7f5e00159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.444+0000 7f5e04918700 1 --2- 172.21.15.81:0/1451033801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e000b3620 0x7f5e0014a010 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.444+0000 7f5e04918700 1 --2- 172.21.15.81:0/1451033801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e0006d8c0 0x7f5e001f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.444+0000 7f5e04918700 1 -- 172.21.15.81:0/1451033801 >> 172.21.15.81:0/1451033801 conn(0x7f5e000b0de0 msgr2=0x7f5e0006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:01.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.444+0000 7f5e04918700 1 -- 172.21.15.81:0/1451033801 shutdown_connections 2024-05-08T23:32:01.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.444+0000 7f5e04918700 1 -- 172.21.15.81:0/1451033801 wait complete. 2024-05-08T23:32:01.458 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:32:01.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.572+0000 7fcddb59e700 1 Processor -- start 2024-05-08T23:32:01.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.572+0000 7fcddb59e700 1 -- start start 2024-05-08T23:32:01.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.572+0000 7fcddb59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcddc0d1da0 0x7fcddc0cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.572+0000 7fcddb59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcddc0cfe80 0x7fcddc0b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.572+0000 7fcddb59e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcddc0d3d40 con 0x7fcddc160920 2024-05-08T23:32:01.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.572+0000 7fcddb59e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc0d02d0 con 0x7fcddc0d1da0 2024-05-08T23:32:01.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.572+0000 7fcddb59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc0d0d00 con 0x7fcddc0cfe80 2024-05-08T23:32:01.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.572+0000 7fcdda59c700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcddc160920 0x7fcddc0ce6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47148/0 (socket says 172.21.15.81:47148) 2024-05-08T23:32:01.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.572+0000 7fcdda59c700 1 -- 172.21.15.81:0/2337818054 learned_addr learned my addr 172.21.15.81:0/2337818054 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:01.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.572+0000 7fcdd9d9b700 1 --2- 172.21.15.81:0/2337818054 >> v2:172.21.15.81:3300/0 conn(0x7fcddc0d1da0 0x7fcddc0cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.572+0000 7fcddad9d700 1 --2- 172.21.15.81:0/2337818054 >> v2:172.21.15.117:3300/0 conn(0x7fcddc0cfe80 0x7fcddc0b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcdd959a700 1 -- 172.21.15.81:0/2337818054 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620447371 0 0) 0x7fcddc0d3d40 con 0x7fcddc160920 2024-05-08T23:32:01.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcdd959a700 1 -- 172.21.15.81:0/2337818054 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdc80035d0 con 0x7fcddc160920 2024-05-08T23:32:01.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcdd9d9b700 1 -- 172.21.15.81:0/2337818054 >> v2:172.21.15.117:3300/0 conn(0x7fcddc0cfe80 msgr2=0x7fcddc0b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcdd9d9b700 1 --2- 172.21.15.81:0/2337818054 >> v2:172.21.15.117:3300/0 conn(0x7fcddc0cfe80 0x7fcddc0b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcdd9d9b700 1 -- 172.21.15.81:0/2337818054 >> v1:172.21.15.81:6789/0 conn(0x7fcddc160920 legacy=0x7fcddc0ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcdd959a700 1 -- 172.21.15.81:0/2337818054 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3348138548 0 0) 0x7fcdc80035d0 con 0x7fcddc160920 2024-05-08T23:32:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcdd9d9b700 1 -- 172.21.15.81:0/2337818054 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc0d0fe0 con 0x7fcddc0d1da0 2024-05-08T23:32:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcdd9d9b700 1 --2- 172.21.15.81:0/2337818054 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcddc0d1da0 0x7fcddc0cf940 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0cd6e0 tx=0x7fcdc4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a985d53a7ccf4d7d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcdd959a700 1 -- 172.21.15.81:0/2337818054 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4011830 con 0x7fcddc0d1da0 2024-05-08T23:32:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcdd959a700 1 -- 172.21.15.81:0/2337818054 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc4011990 con 0x7fcddc0d1da0 2024-05-08T23:32:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcdd959a700 1 -- 172.21.15.81:0/2337818054 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4011d20 con 0x7fcddc0d1da0 2024-05-08T23:32:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcddb59e700 1 -- 172.21.15.81:0/2337818054 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcddc0d1da0 msgr2=0x7fcddc0cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcddb59e700 1 --2- 172.21.15.81:0/2337818054 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcddc0d1da0 0x7fcddc0cf940 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0cd6e0 tx=0x7fcdc4009b30 comp rx=0 tx=0).stop 2024-05-08T23:32:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcddb59e700 1 -- 172.21.15.81:0/2337818054 shutdown_connections 2024-05-08T23:32:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcddb59e700 1 --2- 172.21.15.81:0/2337818054 >> v2:172.21.15.117:3300/0 conn(0x7fcddc0cfe80 0x7fcddc0b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcddb59e700 1 --2- 172.21.15.81:0/2337818054 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcddc0d1da0 0x7fcddc0cf940 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcddb59e700 1 -- 172.21.15.81:0/2337818054 >> 172.21.15.81:0/2337818054 conn(0x7fcddc09c340 msgr2=0x7fcddc0d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:01.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcddb59e700 1 -- 172.21.15.81:0/2337818054 shutdown_connections 2024-05-08T23:32:01.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcddb59e700 1 -- 172.21.15.81:0/2337818054 wait complete. 2024-05-08T23:32:01.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcddb59e700 1 Processor -- start 2024-05-08T23:32:01.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.576+0000 7fcddb59e700 1 -- start start 2024-05-08T23:32:01.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddb59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcddc0cfe80 0x7fcddc0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddb59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcddc0d1da0 0x7fcddc0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddb59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcddc160920 0x7fcddc155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddb59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0b8fa0 con 0x7fcddc160920 2024-05-08T23:32:01.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddb59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0cd010 con 0x7fcddc0cfe80 2024-05-08T23:32:01.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddb59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0cba30 con 0x7fcddc0d1da0 2024-05-08T23:32:01.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcddc160920 0x7fcddc155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcddc160920 0x7fcddc155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53250/0 (socket says 172.21.15.81:53250) 2024-05-08T23:32:01.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddad9d700 1 -- 172.21.15.81:0/1271215310 learned_addr learned my addr 172.21.15.81:0/1271215310 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:01.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcdda59c700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcddc0cfe80 0x7fcddc0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcdd9d9b700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcddc0d1da0 0x7fcddc0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddad9d700 1 -- 172.21.15.81:0/1271215310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcddc0d1da0 msgr2=0x7fcddc0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddad9d700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcddc0d1da0 0x7fcddc0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddad9d700 1 -- 172.21.15.81:0/1271215310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcddc0cfe80 msgr2=0x7fcddc0b8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddad9d700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcddc0cfe80 0x7fcddc0b8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddad9d700 1 -- 172.21.15.81:0/1271215310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc40094b0 con 0x7fcddc160920 2024-05-08T23:32:01.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcdd9d9b700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcddc0d1da0 0x7fcddc0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:01.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcdda59c700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcddc0cfe80 0x7fcddc0b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:01.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddad9d700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcddc160920 0x7fcddc155b00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00efc0 tx=0x7fcdcc00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:01.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcdd2ffd700 1 -- 172.21.15.81:0/1271215310 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00cad0 con 0x7fcddc160920 2024-05-08T23:32:01.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcdd2ffd700 1 -- 172.21.15.81:0/1271215310 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc00f040 con 0x7fcddc160920 2024-05-08T23:32:01.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcdd2ffd700 1 -- 172.21.15.81:0/1271215310 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00ce70 con 0x7fcddc160920 2024-05-08T23:32:01.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddb59e700 1 -- 172.21.15.81:0/1271215310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcddc0cbd10 con 0x7fcddc160920 2024-05-08T23:32:01.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddb59e700 1 -- 172.21.15.81:0/1271215310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc0cc2d0 con 0x7fcddc160920 2024-05-08T23:32:01.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.580+0000 7fcddb59e700 1 -- 172.21.15.81:0/1271215310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcddc0d25a0 con 0x7fcddc160920 2024-05-08T23:32:01.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.584+0000 7fcdd2ffd700 1 -- 172.21.15.81:0/1271215310 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcdcc014480 con 0x7fcddc160920 2024-05-08T23:32:01.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.584+0000 7fcdd2ffd700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcdbc066730 0x7fcdbc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.584+0000 7fcdd2ffd700 1 -- 172.21.15.81:0/1271215310 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fcdcc084a80 con 0x7fcddc160920 2024-05-08T23:32:01.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.584+0000 7fcdda59c700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcdbc066730 0x7fcdbc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.584+0000 7fcdda59c700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcdbc066730 0x7fcdbc068bf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4007fd0 tx=0x7fcdd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:01.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.592+0000 7fcdd2ffd700 1 -- 172.21.15.81:0/1271215310 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcdcc051c70 con 0x7fcddc160920 2024-05-08T23:32:01.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.756+0000 7fcddb59e700 1 -- 172.21.15.81:0/1271215310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcddc0684b0 con 0x7fcddc160920 2024-05-08T23:32:01.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.756+0000 7fcdd2ffd700 1 -- 172.21.15.81:0/1271215310 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcdcc0180a0 con 0x7fcddc160920 2024-05-08T23:32:01.758 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:32:01.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.760+0000 7fcddb59e700 1 -- 172.21.15.81:0/1271215310 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcdbc066730 msgr2=0x7fcdbc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.760+0000 7fcddb59e700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcdbc066730 0x7fcdbc068bf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4007fd0 tx=0x7fcdd4009040 comp rx=0 tx=0).stop 2024-05-08T23:32:01.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.760+0000 7fcddb59e700 1 -- 172.21.15.81:0/1271215310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcddc160920 msgr2=0x7fcddc155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.760+0000 7fcddb59e700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcddc160920 0x7fcddc155b00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00efc0 tx=0x7fcdcc00c0c0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.760+0000 7fcddb59e700 1 -- 172.21.15.81:0/1271215310 shutdown_connections 2024-05-08T23:32:01.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.760+0000 7fcddb59e700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcdbc066730 0x7fcdbc068bf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.760+0000 7fcddb59e700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcddc160920 0x7fcddc155b00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.760+0000 7fcddb59e700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcddc0d1da0 0x7fcddc0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.760+0000 7fcddb59e700 1 --2- 172.21.15.81:0/1271215310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcddc0cfe80 0x7fcddc0b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.760+0000 7fcddb59e700 1 -- 172.21.15.81:0/1271215310 >> 172.21.15.81:0/1271215310 conn(0x7fcddc09c340 msgr2=0x7fcddc06afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:01.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.760+0000 7fcddb59e700 1 -- 172.21.15.81:0/1271215310 shutdown_connections 2024-05-08T23:32:01.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.760+0000 7fcddb59e700 1 -- 172.21.15.81:0/1271215310 wait complete. 2024-05-08T23:32:01.779 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:32:01.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.900+0000 7f3b371bc700 1 Processor -- start 2024-05-08T23:32:01.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.900+0000 7f3b371bc700 1 -- start start 2024-05-08T23:32:01.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.900+0000 7f3b371bc700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b3015ead0 0x7f3b300cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.900+0000 7f3b371bc700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3006c7d0 con 0x7f3b3015ae10 2024-05-08T23:32:01.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.900+0000 7f3b371bc700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b300acb30 con 0x7f3b300ce4f0 2024-05-08T23:32:01.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.900+0000 7f3b371bc700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b300ce9a0 con 0x7f3b3015ead0 2024-05-08T23:32:01.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.900+0000 7f3b359b9700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b3015ead0 0x7f3b300cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.900+0000 7f3b359b9700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b3015ead0 0x7f3b300cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53262/0 (socket says 172.21.15.81:53262) 2024-05-08T23:32:01.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.900+0000 7f3b359b9700 1 -- 172.21.15.81:0/4211432720 learned_addr learned my addr 172.21.15.81:0/4211432720 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:01.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b359b9700 1 -- 172.21.15.81:0/4211432720 >> v1:172.21.15.117:6789/0 conn(0x7f3b300ce4f0 legacy=0x7f3b3006cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b359b9700 1 -- 172.21.15.81:0/4211432720 >> v1:172.21.15.81:6789/0 conn(0x7f3b3015ae10 legacy=0x7f3b300ac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b351b8700 1 -- 172.21.15.81:0/4211432720 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359185779 0 0) 0x7f3b3006c7d0 con 0x7f3b3015ae10 2024-05-08T23:32:01.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b359b9700 1 -- 172.21.15.81:0/4211432720 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3006c4d0 con 0x7f3b3015ead0 2024-05-08T23:32:01.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b359b9700 1 --2- 172.21.15.81:0/4211432720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b3015ead0 0x7f3b300cdfb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009940 tx=0x7f3b20009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fe54bbaae1c01e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:01.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b351b8700 1 -- 172.21.15.81:0/4211432720 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2000ba20 con 0x7f3b3015ead0 2024-05-08T23:32:01.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b351b8700 1 -- 172.21.15.81:0/4211432720 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b20009e90 con 0x7f3b3015ead0 2024-05-08T23:32:01.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b351b8700 1 -- 172.21.15.81:0/4211432720 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2000bda0 con 0x7f3b3015ead0 2024-05-08T23:32:01.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b371bc700 1 -- 172.21.15.81:0/4211432720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b3015ead0 msgr2=0x7f3b300cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b371bc700 1 --2- 172.21.15.81:0/4211432720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b3015ead0 0x7f3b300cdfb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009940 tx=0x7f3b20009c50 comp rx=0 tx=0).stop 2024-05-08T23:32:01.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b371bc700 1 -- 172.21.15.81:0/4211432720 shutdown_connections 2024-05-08T23:32:01.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b371bc700 1 --2- 172.21.15.81:0/4211432720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b3015ead0 0x7f3b300cdfb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b371bc700 1 -- 172.21.15.81:0/4211432720 >> 172.21.15.81:0/4211432720 conn(0x7f3b300b0de0 msgr2=0x7f3b300b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:01.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b371bc700 1 -- 172.21.15.81:0/4211432720 shutdown_connections 2024-05-08T23:32:01.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b371bc700 1 -- 172.21.15.81:0/4211432720 wait complete. 2024-05-08T23:32:01.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.904+0000 7f3b371bc700 1 Processor -- start 2024-05-08T23:32:01.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b371bc700 1 -- start start 2024-05-08T23:32:01.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b371bc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b300ce4f0 0x7f3b30148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b371bc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b3015ae10 0x7f3b300ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b371bc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b3015ead0 0x7f3b300cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b371bc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30148ac0 con 0x7f3b3015ead0 2024-05-08T23:32:01.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b371bc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3015c230 con 0x7f3b3015ae10 2024-05-08T23:32:01.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b371bc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3015c510 con 0x7f3b300ce4f0 2024-05-08T23:32:01.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b369bb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b3015ead0 0x7f3b300cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b369bb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b3015ead0 0x7f3b300cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53276/0 (socket says 172.21.15.81:53276) 2024-05-08T23:32:01.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b369bb700 1 -- 172.21.15.81:0/2691790398 learned_addr learned my addr 172.21.15.81:0/2691790398 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:01.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b361ba700 1 --2- 172.21.15.81:0/2691790398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b300ce4f0 0x7f3b30148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b359b9700 1 --2- 172.21.15.81:0/2691790398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b3015ae10 0x7f3b300ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b369bb700 1 -- 172.21.15.81:0/2691790398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b300ce4f0 msgr2=0x7f3b30148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b369bb700 1 --2- 172.21.15.81:0/2691790398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b300ce4f0 0x7f3b30148580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b369bb700 1 -- 172.21.15.81:0/2691790398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b3015ae10 msgr2=0x7f3b300ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:01.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b369bb700 1 --2- 172.21.15.81:0/2691790398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b3015ae10 0x7f3b300ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:01.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b369bb700 1 -- 172.21.15.81:0/2691790398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b20009570 con 0x7f3b3015ead0 2024-05-08T23:32:01.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b369bb700 1 --2- 172.21.15.81:0/2691790398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b3015ead0 0x7f3b300cd130 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800cc50 tx=0x7f3b2800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:01.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b2effd700 1 -- 172.21.15.81:0/2691790398 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28003c20 con 0x7f3b3015ead0 2024-05-08T23:32:01.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b2effd700 1 -- 172.21.15.81:0/2691790398 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b28003d80 con 0x7f3b3015ead0 2024-05-08T23:32:01.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b371bc700 1 -- 172.21.15.81:0/2691790398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3015c7f0 con 0x7f3b3015ead0 2024-05-08T23:32:01.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b2effd700 1 -- 172.21.15.81:0/2691790398 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2800aab0 con 0x7f3b3015ead0 2024-05-08T23:32:01.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.908+0000 7f3b371bc700 1 -- 172.21.15.81:0/2691790398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3015cdb0 con 0x7f3b3015ead0 2024-05-08T23:32:01.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.912+0000 7f3b371bc700 1 -- 172.21.15.81:0/2691790398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b300684b0 con 0x7f3b3015ead0 2024-05-08T23:32:01.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.912+0000 7f3b2effd700 1 -- 172.21.15.81:0/2691790398 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3b2800ac10 con 0x7f3b3015ead0 2024-05-08T23:32:01.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.912+0000 7f3b2effd700 1 --2- 172.21.15.81:0/2691790398 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3b1c066760 0x7f3b1c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:01.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.912+0000 7f3b2effd700 1 -- 172.21.15.81:0/2691790398 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f3b280842d0 con 0x7f3b3015ead0 2024-05-08T23:32:01.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.912+0000 7f3b361ba700 1 --2- 172.21.15.81:0/2691790398 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3b1c066760 0x7f3b1c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:01.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.912+0000 7f3b361ba700 1 --2- 172.21.15.81:0/2691790398 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3b1c066760 0x7f3b1c068c20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3b24000f60 tx=0x7f3b24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:01.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:01.920+0000 7f3b2effd700 1 -- 172.21.15.81:0/2691790398 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3b280514b0 con 0x7f3b3015ead0 2024-05-08T23:32:02.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.084+0000 7f3b371bc700 1 -- 172.21.15.81:0/2691790398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b300b2660 con 0x7f3b3015ead0 2024-05-08T23:32:02.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.084+0000 7f3b2effd700 1 -- 172.21.15.81:0/2691790398 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3b28054ad0 con 0x7f3b3015ead0 2024-05-08T23:32:02.087 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:32:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.088+0000 7f3b371bc700 1 -- 172.21.15.81:0/2691790398 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3b1c066760 msgr2=0x7f3b1c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.088+0000 7f3b371bc700 1 --2- 172.21.15.81:0/2691790398 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3b1c066760 0x7f3b1c068c20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3b24000f60 tx=0x7f3b24009040 comp rx=0 tx=0).stop 2024-05-08T23:32:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.088+0000 7f3b371bc700 1 -- 172.21.15.81:0/2691790398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b3015ead0 msgr2=0x7f3b300cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.088+0000 7f3b371bc700 1 --2- 172.21.15.81:0/2691790398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b3015ead0 0x7f3b300cd130 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800cc50 tx=0x7f3b2800cf60 comp rx=0 tx=0).stop 2024-05-08T23:32:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.088+0000 7f3b371bc700 1 -- 172.21.15.81:0/2691790398 shutdown_connections 2024-05-08T23:32:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.088+0000 7f3b371bc700 1 --2- 172.21.15.81:0/2691790398 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3b1c066760 0x7f3b1c068c20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.088+0000 7f3b371bc700 1 --2- 172.21.15.81:0/2691790398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b3015ead0 0x7f3b300cd130 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.088+0000 7f3b371bc700 1 --2- 172.21.15.81:0/2691790398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b3015ae10 0x7f3b300ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.088+0000 7f3b371bc700 1 --2- 172.21.15.81:0/2691790398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b300ce4f0 0x7f3b30148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.088+0000 7f3b371bc700 1 -- 172.21.15.81:0/2691790398 >> 172.21.15.81:0/2691790398 conn(0x7f3b300b0de0 msgr2=0x7f3b300afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.088+0000 7f3b371bc700 1 -- 172.21.15.81:0/2691790398 shutdown_connections 2024-05-08T23:32:02.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.088+0000 7f3b371bc700 1 -- 172.21.15.81:0/2691790398 wait complete. 2024-05-08T23:32:02.102 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-05-08T23:32:02.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80e9e24700 1 Processor -- start 2024-05-08T23:32:02.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80e9e24700 1 -- start start 2024-05-08T23:32:02.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80e9e24700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80e406d8c0 0x7f80e415e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80e9e24700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f80e4071a50 0x7f80e4156d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80e9e24700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e40721c0 con 0x7f80e40b20f0 2024-05-08T23:32:02.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80e9e24700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e415e650 con 0x7f80e4071a50 2024-05-08T23:32:02.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80e9e24700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e40ad700 con 0x7f80e406d8c0 2024-05-08T23:32:02.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80dbfff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f80e40b20f0 0x7f80e4071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47156/0 (socket says 172.21.15.81:47156) 2024-05-08T23:32:02.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80dbfff700 1 -- 172.21.15.81:0/614402944 learned_addr learned my addr 172.21.15.81:0/614402944 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:02.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80e9623700 1 --2- 172.21.15.81:0/614402944 >> v2:172.21.15.117:3300/0 conn(0x7f80e4071a50 0x7f80e4156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80e8e22700 1 --2- 172.21.15.81:0/614402944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80e406d8c0 0x7f80e415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80db7fe700 1 -- 172.21.15.81:0/614402944 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179139057 0 0) 0x7f80e40721c0 con 0x7f80e40b20f0 2024-05-08T23:32:02.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80db7fe700 1 -- 172.21.15.81:0/614402944 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80cc0035d0 con 0x7f80e40b20f0 2024-05-08T23:32:02.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80e9623700 1 -- 172.21.15.81:0/614402944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80e406d8c0 msgr2=0x7f80e415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:02.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80e9623700 1 --2- 172.21.15.81:0/614402944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80e406d8c0 0x7f80e415e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80e9623700 1 -- 172.21.15.81:0/614402944 >> v1:172.21.15.81:6789/0 conn(0x7f80e40b20f0 legacy=0x7f80e4071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80e9623700 1 -- 172.21.15.81:0/614402944 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e40b2580 con 0x7f80e4071a50 2024-05-08T23:32:02.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80e9623700 1 --2- 172.21.15.81:0/614402944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e4071a50 0x7f80e4156d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009d90 tx=0x7f80dc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23063537cd076fc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:02.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80db7fe700 1 -- 172.21.15.81:0/614402944 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc00e070 con 0x7f80e4071a50 2024-05-08T23:32:02.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80db7fe700 1 -- 172.21.15.81:0/614402944 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc00bb20 con 0x7f80e4071a50 2024-05-08T23:32:02.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.224+0000 7f80db7fe700 1 -- 172.21.15.81:0/614402944 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc012430 con 0x7f80e4071a50 2024-05-08T23:32:02.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 -- 172.21.15.81:0/614402944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e4071a50 msgr2=0x7f80e4156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 --2- 172.21.15.81:0/614402944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e4071a50 0x7f80e4156d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009d90 tx=0x7f80dc00b330 comp rx=0 tx=0).stop 2024-05-08T23:32:02.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 -- 172.21.15.81:0/614402944 shutdown_connections 2024-05-08T23:32:02.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 --2- 172.21.15.81:0/614402944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e4071a50 0x7f80e4156d80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 --2- 172.21.15.81:0/614402944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80e406d8c0 0x7f80e415e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 -- 172.21.15.81:0/614402944 >> 172.21.15.81:0/614402944 conn(0x7f80e40b0de0 msgr2=0x7f80e40b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:02.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 -- 172.21.15.81:0/614402944 shutdown_connections 2024-05-08T23:32:02.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 -- 172.21.15.81:0/614402944 wait complete. 2024-05-08T23:32:02.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 Processor -- start 2024-05-08T23:32:02.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 -- start start 2024-05-08T23:32:02.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e406d8c0 0x7f80e4156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e4071a50 0x7f80e4099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80e41f3730 0x7f80e40cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e409a2c0 con 0x7f80e4071a50 2024-05-08T23:32:02.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e415e9d0 con 0x7f80e406d8c0 2024-05-08T23:32:02.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9e24700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e415ecb0 con 0x7f80e41f3730 2024-05-08T23:32:02.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80dbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e4071a50 0x7f80e4099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80dbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e4071a50 0x7f80e4099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53290/0 (socket says 172.21.15.81:53290) 2024-05-08T23:32:02.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80dbfff700 1 -- 172.21.15.81:0/4085896310 learned_addr learned my addr 172.21.15.81:0/4085896310 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:02.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e9623700 1 --2- 172.21.15.81:0/4085896310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80e41f3730 0x7f80e40cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e8e22700 1 --2- 172.21.15.81:0/4085896310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e406d8c0 0x7f80e4156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80dbfff700 1 -- 172.21.15.81:0/4085896310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80e41f3730 msgr2=0x7f80e40cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80dbfff700 1 --2- 172.21.15.81:0/4085896310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80e41f3730 0x7f80e40cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80dbfff700 1 -- 172.21.15.81:0/4085896310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e406d8c0 msgr2=0x7f80e4156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80dbfff700 1 --2- 172.21.15.81:0/4085896310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e406d8c0 0x7f80e4156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80dbfff700 1 -- 172.21.15.81:0/4085896310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80dc009a70 con 0x7f80e4071a50 2024-05-08T23:32:02.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.228+0000 7f80e8e22700 1 --2- 172.21.15.81:0/4085896310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e406d8c0 0x7f80e4156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:02.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.232+0000 7f80dbfff700 1 --2- 172.21.15.81:0/4085896310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e4071a50 0x7f80e4099a70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c4d0 tx=0x7f80d400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:02.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.232+0000 7f80d97fa700 1 -- 172.21.15.81:0/4085896310 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d4005d90 con 0x7f80e4071a50 2024-05-08T23:32:02.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.232+0000 7f80d97fa700 1 -- 172.21.15.81:0/4085896310 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d40070d0 con 0x7f80e4071a50 2024-05-08T23:32:02.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.232+0000 7f80e9e24700 1 -- 172.21.15.81:0/4085896310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e415ef90 con 0x7f80e4071a50 2024-05-08T23:32:02.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.232+0000 7f80e9e24700 1 -- 172.21.15.81:0/4085896310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e4156840 con 0x7f80e4071a50 2024-05-08T23:32:02.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.232+0000 7f80d97fa700 1 -- 172.21.15.81:0/4085896310 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d4007440 con 0x7f80e4071a50 2024-05-08T23:32:02.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.232+0000 7f80d97fa700 1 -- 172.21.15.81:0/4085896310 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f80d40075a0 con 0x7f80e4071a50 2024-05-08T23:32:02.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.232+0000 7f80d97fa700 1 --2- 172.21.15.81:0/4085896310 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80d0066730 0x7f80d0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.232+0000 7f80e9e24700 1 -- 172.21.15.81:0/4085896310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e415f450 con 0x7f80e4071a50 2024-05-08T23:32:02.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.232+0000 7f80e8e22700 1 --2- 172.21.15.81:0/4085896310 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80d0066730 0x7f80d0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.232+0000 7f80d97fa700 1 -- 172.21.15.81:0/4085896310 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f80d4083690 con 0x7f80e4071a50 2024-05-08T23:32:02.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.232+0000 7f80e8e22700 1 --2- 172.21.15.81:0/4085896310 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80d0066730 0x7f80d0068bf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009c40 tx=0x7f80e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:02.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.240+0000 7f80d97fa700 1 -- 172.21.15.81:0/4085896310 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f80d4021030 con 0x7f80e4071a50 2024-05-08T23:32:02.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.412+0000 7f80e9e24700 1 -- 172.21.15.81:0/4085896310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f80e40684b0 con 0x7f80e4071a50 2024-05-08T23:32:02.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.412+0000 7f80d97fa700 1 -- 172.21.15.81:0/4085896310 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v39) v1 ==== 97+0+11 (secure 0 0 0) 0x7f80d4017020 con 0x7f80e4071a50 2024-05-08T23:32:02.413 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:32:02.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.412+0000 7f80e9e24700 1 -- 172.21.15.81:0/4085896310 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80d0066730 msgr2=0x7f80d0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.412+0000 7f80e9e24700 1 --2- 172.21.15.81:0/4085896310 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80d0066730 0x7f80d0068bf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009c40 tx=0x7f80e0008040 comp rx=0 tx=0).stop 2024-05-08T23:32:02.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.416+0000 7f80e9e24700 1 -- 172.21.15.81:0/4085896310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e4071a50 msgr2=0x7f80e4099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.416+0000 7f80e9e24700 1 --2- 172.21.15.81:0/4085896310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e4071a50 0x7f80e4099a70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c4d0 tx=0x7f80d400c7e0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.416+0000 7f80e9e24700 1 -- 172.21.15.81:0/4085896310 shutdown_connections 2024-05-08T23:32:02.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.416+0000 7f80e9e24700 1 --2- 172.21.15.81:0/4085896310 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f80d0066730 0x7f80d0068bf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.416+0000 7f80e9e24700 1 --2- 172.21.15.81:0/4085896310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80e41f3730 0x7f80e40cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.416+0000 7f80e9e24700 1 --2- 172.21.15.81:0/4085896310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e4071a50 0x7f80e4099a70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.416+0000 7f80e9e24700 1 --2- 172.21.15.81:0/4085896310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e406d8c0 0x7f80e4156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.416+0000 7f80e9e24700 1 -- 172.21.15.81:0/4085896310 >> 172.21.15.81:0/4085896310 conn(0x7f80e40b0de0 msgr2=0x7f80e4161510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:02.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.416+0000 7f80e9e24700 1 -- 172.21.15.81:0/4085896310 shutdown_connections 2024-05-08T23:32:02.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.416+0000 7f80e9e24700 1 -- 172.21.15.81:0/4085896310 wait complete. 2024-05-08T23:32:02.429 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:32:02.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47f2d8700 1 Processor -- start 2024-05-08T23:32:02.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47f2d8700 1 -- start start 2024-05-08T23:32:02.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47f2d8700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4780b3c20 0x7fb4781624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47f2d8700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb478070110 0x7fb4780705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47f2d8700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb478158960 con 0x7fb478071e70 2024-05-08T23:32:02.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47f2d8700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4780b1f60 con 0x7fb478070110 2024-05-08T23:32:02.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47f2d8700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780ae990 con 0x7fb4780b3c20 2024-05-08T23:32:02.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47e2d6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4780b3c20 0x7fb4781624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47e2d6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4780b3c20 0x7fb4781624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:47690/0 (socket says 172.21.15.81:47690) 2024-05-08T23:32:02.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47e2d6700 1 -- 172.21.15.81:0/1978623656 learned_addr learned my addr 172.21.15.81:0/1978623656 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:02.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47ead7700 1 --2- 172.21.15.81:0/1978623656 >> v2:172.21.15.117:3300/0 conn(0x7fb478070110 0x7fb4780705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47e2d6700 1 -- 172.21.15.81:0/1978623656 >> v2:172.21.15.117:3300/0 conn(0x7fb478070110 msgr2=0x7fb4780705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:02.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47e2d6700 1 --2- 172.21.15.81:0/1978623656 >> v2:172.21.15.117:3300/0 conn(0x7fb478070110 0x7fb4780705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47e2d6700 1 -- 172.21.15.81:0/1978623656 >> v1:172.21.15.117:6789/0 conn(0x7fb478071e70 legacy=0x7fb4780b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47e2d6700 1 -- 172.21.15.81:0/1978623656 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4780b2170 con 0x7fb4780b3c20 2024-05-08T23:32:02.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47d2d4700 1 -- 172.21.15.81:0/1978623656 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572578546 0 0) 0x7fb478158960 con 0x7fb478071e70 2024-05-08T23:32:02.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47e2d6700 1 --2- 172.21.15.81:0/1978623656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4780b3c20 0x7fb4781624b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb468009910 tx=0x7fb468009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=474fa1dbdcd96bb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:02.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47d2d4700 1 -- 172.21.15.81:0/1978623656 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468011830 con 0x7fb4780b3c20 2024-05-08T23:32:02.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47d2d4700 1 -- 172.21.15.81:0/1978623656 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb468011990 con 0x7fb4780b3c20 2024-05-08T23:32:02.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.540+0000 7fb47d2d4700 1 -- 172.21.15.81:0/1978623656 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468011d40 con 0x7fb4780b3c20 2024-05-08T23:32:02.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 -- 172.21.15.81:0/1978623656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4780b3c20 msgr2=0x7fb4781624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 --2- 172.21.15.81:0/1978623656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4780b3c20 0x7fb4781624b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb468009910 tx=0x7fb468009c20 comp rx=0 tx=0).stop 2024-05-08T23:32:02.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 -- 172.21.15.81:0/1978623656 shutdown_connections 2024-05-08T23:32:02.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 --2- 172.21.15.81:0/1978623656 >> v2:172.21.15.117:3300/0 conn(0x7fb478070110 0x7fb4780705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 --2- 172.21.15.81:0/1978623656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4780b3c20 0x7fb4781624b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 -- 172.21.15.81:0/1978623656 >> 172.21.15.81:0/1978623656 conn(0x7fb47806c330 msgr2=0x7fb47806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:02.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 -- 172.21.15.81:0/1978623656 shutdown_connections 2024-05-08T23:32:02.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 -- 172.21.15.81:0/1978623656 wait complete. 2024-05-08T23:32:02.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 Processor -- start 2024-05-08T23:32:02.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 -- start start 2024-05-08T23:32:02.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb478070110 0x7fb4781ea890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb478071e70 0x7fb478152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4780b3c20 0x7fb478151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4781e96d0 con 0x7fb4780b3c20 2024-05-08T23:32:02.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4781eadd0 con 0x7fb478071e70 2024-05-08T23:32:02.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47f2d8700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4781e8a50 con 0x7fb478070110 2024-05-08T23:32:02.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47ead7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4780b3c20 0x7fb478151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47e2d6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb478070110 0x7fb4781ea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47dad5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb478071e70 0x7fb478152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47ead7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4780b3c20 0x7fb478151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53306/0 (socket says 172.21.15.81:53306) 2024-05-08T23:32:02.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47e2d6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb478070110 0x7fb4781ea890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:47694/0 (socket says 172.21.15.81:47694) 2024-05-08T23:32:02.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.544+0000 7fb47ead7700 1 -- 172.21.15.81:0/3780676660 learned_addr learned my addr 172.21.15.81:0/3780676660 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:02.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb47ead7700 1 -- 172.21.15.81:0/3780676660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb478070110 msgr2=0x7fb4781ea890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb47ead7700 1 --2- 172.21.15.81:0/3780676660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb478070110 0x7fb4781ea890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb47ead7700 1 -- 172.21.15.81:0/3780676660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb478071e70 msgr2=0x7fb478152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb47ead7700 1 --2- 172.21.15.81:0/3780676660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb478071e70 0x7fb478152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb47ead7700 1 -- 172.21.15.81:0/3780676660 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468009570 con 0x7fb4780b3c20 2024-05-08T23:32:02.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb47ead7700 1 --2- 172.21.15.81:0/3780676660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4780b3c20 0x7fb478151140 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb470009d30 tx=0x7fb47000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:02.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb46effd700 1 -- 172.21.15.81:0/3780676660 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47000cc90 con 0x7fb4780b3c20 2024-05-08T23:32:02.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb46effd700 1 -- 172.21.15.81:0/3780676660 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47000f040 con 0x7fb4780b3c20 2024-05-08T23:32:02.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb46effd700 1 -- 172.21.15.81:0/3780676660 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb470009e80 con 0x7fb4780b3c20 2024-05-08T23:32:02.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb47f2d8700 1 -- 172.21.15.81:0/3780676660 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4781e8d30 con 0x7fb4780b3c20 2024-05-08T23:32:02.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb47f2d8700 1 -- 172.21.15.81:0/3780676660 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4781e92f0 con 0x7fb4780b3c20 2024-05-08T23:32:02.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb47f2d8700 1 -- 172.21.15.81:0/3780676660 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4781e9810 con 0x7fb4780b3c20 2024-05-08T23:32:02.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb46effd700 1 -- 172.21.15.81:0/3780676660 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb470014480 con 0x7fb4780b3c20 2024-05-08T23:32:02.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb46effd700 1 --2- 172.21.15.81:0/3780676660 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb45c066730 0x7fb45c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb47e2d6700 1 --2- 172.21.15.81:0/3780676660 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb45c066730 0x7fb45c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.548+0000 7fb46effd700 1 -- 172.21.15.81:0/3780676660 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fb4700849d0 con 0x7fb4780b3c20 2024-05-08T23:32:02.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.552+0000 7fb47e2d6700 1 --2- 172.21.15.81:0/3780676660 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb45c066730 0x7fb45c068bf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb468000c00 tx=0x7fb468008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:02.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.556+0000 7fb46effd700 1 -- 172.21.15.81:0/3780676660 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb4781e9810 con 0x7fb4780b3c20 2024-05-08T23:32:02.740 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.740+0000 7fb47f2d8700 1 -- 172.21.15.81:0/3780676660 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb4781e9810 con 0x7fb4780b3c20 2024-05-08T23:32:02.741 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.740+0000 7fb46effd700 1 -- 172.21.15.81:0/3780676660 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb4781e9810 con 0x7fb4780b3c20 2024-05-08T23:32:02.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.740+0000 7fb47f2d8700 1 -- 172.21.15.81:0/3780676660 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb45c066730 msgr2=0x7fb45c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.740+0000 7fb47f2d8700 1 --2- 172.21.15.81:0/3780676660 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb45c066730 0x7fb45c068bf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb468000c00 tx=0x7fb468008030 comp rx=0 tx=0).stop 2024-05-08T23:32:02.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.740+0000 7fb47f2d8700 1 -- 172.21.15.81:0/3780676660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4780b3c20 msgr2=0x7fb478151140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.740+0000 7fb47f2d8700 1 --2- 172.21.15.81:0/3780676660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4780b3c20 0x7fb478151140 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb470009d30 tx=0x7fb47000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.744+0000 7fb47f2d8700 1 -- 172.21.15.81:0/3780676660 shutdown_connections 2024-05-08T23:32:02.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.744+0000 7fb47f2d8700 1 --2- 172.21.15.81:0/3780676660 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb45c066730 0x7fb45c068bf0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.744+0000 7fb47f2d8700 1 --2- 172.21.15.81:0/3780676660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4780b3c20 0x7fb478151140 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.744+0000 7fb47f2d8700 1 --2- 172.21.15.81:0/3780676660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb478071e70 0x7fb478152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.744+0000 7fb47f2d8700 1 --2- 172.21.15.81:0/3780676660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb478070110 0x7fb4781ea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.744+0000 7fb47f2d8700 1 -- 172.21.15.81:0/3780676660 >> 172.21.15.81:0/3780676660 conn(0x7fb47806c330 msgr2=0x7fb4780aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:02.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.744+0000 7fb47f2d8700 1 -- 172.21.15.81:0/3780676660 shutdown_connections 2024-05-08T23:32:02.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.744+0000 7fb47f2d8700 1 -- 172.21.15.81:0/3780676660 wait complete. 2024-05-08T23:32:02.746 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:32:02.755 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:32:02.755 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:02.764 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:02.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.872+0000 7f1055da2700 1 Processor -- start 2024-05-08T23:32:02.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f1055da2700 1 -- start start 2024-05-08T23:32:02.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f1055da2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f105006d220 0x7f105015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f1055da2700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10500d0cd0 con 0x7f105006efa0 2024-05-08T23:32:02.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f1055da2700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105006d8b0 con 0x7f1050159f10 2024-05-08T23:32:02.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f1055da2700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105006d6d0 con 0x7f105006d220 2024-05-08T23:32:02.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f10555a1700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f105006d220 0x7f105015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f104ffff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f105006efa0 0x7f10500b2eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47166/0 (socket says 172.21.15.81:47166) 2024-05-08T23:32:02.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f104ffff700 1 -- 172.21.15.81:0/2625270168 learned_addr learned my addr 172.21.15.81:0/2625270168 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:02.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f10555a1700 1 -- 172.21.15.81:0/2625270168 >> v1:172.21.15.117:6789/0 conn(0x7f1050159f10 legacy=0x7f105006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f10555a1700 1 -- 172.21.15.81:0/2625270168 >> v1:172.21.15.81:6789/0 conn(0x7f105006efa0 legacy=0x7f10500b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f10555a1700 1 -- 172.21.15.81:0/2625270168 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105006db90 con 0x7f105006d220 2024-05-08T23:32:02.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f104f7fe700 1 -- 172.21.15.81:0/2625270168 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026731660 0 0) 0x7f10500d0cd0 con 0x7f105006efa0 2024-05-08T23:32:02.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f10555a1700 1 --2- 172.21.15.81:0/2625270168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f105006d220 0x7f105015a2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1048009850 tx=0x7f1048009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775bd033fc168db5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:02.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f104f7fe700 1 -- 172.21.15.81:0/2625270168 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048011830 con 0x7f105006d220 2024-05-08T23:32:02.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f104f7fe700 1 -- 172.21.15.81:0/2625270168 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1048011990 con 0x7f105006d220 2024-05-08T23:32:02.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f104f7fe700 1 -- 172.21.15.81:0/2625270168 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048011d20 con 0x7f105006d220 2024-05-08T23:32:02.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f1055da2700 1 -- 172.21.15.81:0/2625270168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f105006d220 msgr2=0x7f105015a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f1055da2700 1 --2- 172.21.15.81:0/2625270168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f105006d220 0x7f105015a2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1048009850 tx=0x7f1048009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:02.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f1055da2700 1 -- 172.21.15.81:0/2625270168 shutdown_connections 2024-05-08T23:32:02.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f1055da2700 1 --2- 172.21.15.81:0/2625270168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f105006d220 0x7f105015a2f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f1055da2700 1 -- 172.21.15.81:0/2625270168 >> 172.21.15.81:0/2625270168 conn(0x7f10500cfda0 msgr2=0x7f10500d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:02.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f1055da2700 1 -- 172.21.15.81:0/2625270168 shutdown_connections 2024-05-08T23:32:02.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.876+0000 7f1055da2700 1 -- 172.21.15.81:0/2625270168 wait complete. 2024-05-08T23:32:02.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1055da2700 1 Processor -- start 2024-05-08T23:32:02.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1055da2700 1 -- start start 2024-05-08T23:32:02.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1055da2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f105006d220 0x7f1050142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1055da2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f105006efa0 0x7f1050154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1055da2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1050159f10 0x7f1050154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1055da2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10501e6220 con 0x7f105006d220 2024-05-08T23:32:02.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1055da2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10501e55a0 con 0x7f1050159f10 2024-05-08T23:32:02.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1055da2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10501e5880 con 0x7f105006efa0 2024-05-08T23:32:02.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1054da0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f105006d220 0x7f1050142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1054da0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f105006d220 0x7f1050142730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53316/0 (socket says 172.21.15.81:53316) 2024-05-08T23:32:02.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1054da0700 1 -- 172.21.15.81:0/3975876481 learned_addr learned my addr 172.21.15.81:0/3975876481 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:02.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f10555a1700 1 --2- 172.21.15.81:0/3975876481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1050159f10 0x7f1050154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f104ffff700 1 --2- 172.21.15.81:0/3975876481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f105006efa0 0x7f1050154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1054da0700 1 -- 172.21.15.81:0/3975876481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f105006efa0 msgr2=0x7f1050154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1054da0700 1 --2- 172.21.15.81:0/3975876481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f105006efa0 0x7f1050154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1054da0700 1 -- 172.21.15.81:0/3975876481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1050159f10 msgr2=0x7f1050154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:02.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1054da0700 1 --2- 172.21.15.81:0/3975876481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1050159f10 0x7f1050154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:02.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1054da0700 1 -- 172.21.15.81:0/3975876481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10480094b0 con 0x7f105006d220 2024-05-08T23:32:02.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1054da0700 1 --2- 172.21.15.81:0/3975876481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f105006d220 0x7f1050142730 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f104400c500 tx=0x7f104400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:02.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f104d7fa700 1 -- 172.21.15.81:0/3975876481 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10440052d0 con 0x7f105006d220 2024-05-08T23:32:02.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f104d7fa700 1 -- 172.21.15.81:0/3975876481 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1044005430 con 0x7f105006d220 2024-05-08T23:32:02.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1055da2700 1 -- 172.21.15.81:0/3975876481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10501e5b60 con 0x7f105006d220 2024-05-08T23:32:02.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f104d7fa700 1 -- 172.21.15.81:0/3975876481 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10440055a0 con 0x7f105006d220 2024-05-08T23:32:02.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.880+0000 7f1055da2700 1 -- 172.21.15.81:0/3975876481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1050142c70 con 0x7f105006d220 2024-05-08T23:32:02.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.884+0000 7f1055da2700 1 -- 172.21.15.81:0/3975876481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1050159a70 con 0x7f105006d220 2024-05-08T23:32:02.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.884+0000 7f104d7fa700 1 -- 172.21.15.81:0/3975876481 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f10440038e0 con 0x7f105006d220 2024-05-08T23:32:02.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.884+0000 7f104d7fa700 1 --2- 172.21.15.81:0/3975876481 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f103c066730 0x7f103c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:02.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.884+0000 7f104d7fa700 1 -- 172.21.15.81:0/3975876481 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f1044083690 con 0x7f105006d220 2024-05-08T23:32:02.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.884+0000 7f104ffff700 1 --2- 172.21.15.81:0/3975876481 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f103c066730 0x7f103c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:02.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.884+0000 7f104ffff700 1 --2- 172.21.15.81:0/3975876481 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f103c066730 0x7f103c068bf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f10500cf780 tx=0x7f1040001d20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:02.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:02.892+0000 7f104d7fa700 1 -- 172.21.15.81:0/3975876481 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f10440507d0 con 0x7f105006d220 2024-05-08T23:32:03.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.064+0000 7f1055da2700 1 -- 172.21.15.81:0/3975876481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10500672a0 con 0x7f105006d220 2024-05-08T23:32:03.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.068+0000 7f104d7fa700 1 -- 172.21.15.81:0/3975876481 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+17880 (secure 0 0 0) 0x7f1044053df0 con 0x7f105006d220 2024-05-08T23:32:03.070 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:03.070 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":39,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:48.310750+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:31:35.620423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:31:36.878524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-05-08T23:31:46.337484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:03.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.072+0000 7f1055da2700 1 -- 172.21.15.81:0/3975876481 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f103c066730 msgr2=0x7f103c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.072+0000 7f1055da2700 1 --2- 172.21.15.81:0/3975876481 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f103c066730 0x7f103c068bf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f10500cf780 tx=0x7f1040001d20 comp rx=0 tx=0).stop 2024-05-08T23:32:03.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.072+0000 7f1055da2700 1 -- 172.21.15.81:0/3975876481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f105006d220 msgr2=0x7f1050142730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.072+0000 7f1055da2700 1 --2- 172.21.15.81:0/3975876481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f105006d220 0x7f1050142730 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f104400c500 tx=0x7f104400c810 comp rx=0 tx=0).stop 2024-05-08T23:32:03.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.072+0000 7f1055da2700 1 -- 172.21.15.81:0/3975876481 shutdown_connections 2024-05-08T23:32:03.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.072+0000 7f1055da2700 1 --2- 172.21.15.81:0/3975876481 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f103c066730 0x7f103c068bf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.072+0000 7f1055da2700 1 --2- 172.21.15.81:0/3975876481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1050159f10 0x7f1050154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.072+0000 7f1055da2700 1 --2- 172.21.15.81:0/3975876481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f105006efa0 0x7f1050154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.072+0000 7f1055da2700 1 --2- 172.21.15.81:0/3975876481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f105006d220 0x7f1050142730 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.072+0000 7f1055da2700 1 -- 172.21.15.81:0/3975876481 >> 172.21.15.81:0/3975876481 conn(0x7f10500cfda0 msgr2=0x7f105015d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:03.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.072+0000 7f1055da2700 1 -- 172.21.15.81:0/3975876481 shutdown_connections 2024-05-08T23:32:03.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.072+0000 7f1055da2700 1 -- 172.21.15.81:0/3975876481 wait complete. 2024-05-08T23:32:03.085 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:31:35.620423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:31:36.878524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-05-08T23:31:46.337484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:32:03.085 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:03.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.204+0000 7fb6c9bcb700 1 Processor -- start 2024-05-08T23:32:03.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.204+0000 7fb6c9bcb700 1 -- start start 2024-05-08T23:32:03.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.204+0000 7fb6c9bcb700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb6c415ad60 0x7fb6c415d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.204+0000 7fb6c9bcb700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c415b140 con 0x7fb6c40b36a0 2024-05-08T23:32:03.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.204+0000 7fb6c9bcb700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c406b2e0 con 0x7fb6c406d000 2024-05-08T23:32:03.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.204+0000 7fb6c9bcb700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c406a6d0 con 0x7fb6c415ad60 2024-05-08T23:32:03.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.204+0000 7fb6c93ca700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb6c40b36a0 0x7fb6c40ad3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47168/0 (socket says 172.21.15.81:47168) 2024-05-08T23:32:03.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.204+0000 7fb6c8bc9700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb6c415ad60 0x7fb6c415d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.204+0000 7fb6c93ca700 1 -- 172.21.15.81:0/3966479026 learned_addr learned my addr 172.21.15.81:0/3966479026 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:03.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.204+0000 7fb6c8bc9700 1 -- 172.21.15.81:0/3966479026 >> v1:172.21.15.117:6789/0 conn(0x7fb6c406d000 legacy=0x7fb6c406b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.204+0000 7fb6c8bc9700 1 -- 172.21.15.81:0/3966479026 >> v1:172.21.15.81:6789/0 conn(0x7fb6c40b36a0 legacy=0x7fb6c40ad3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.204+0000 7fb6c37fe700 1 -- 172.21.15.81:0/3966479026 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457996715 0 0) 0x7fb6c415b140 con 0x7fb6c40b36a0 2024-05-08T23:32:03.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.204+0000 7fb6c8bc9700 1 -- 172.21.15.81:0/3966479026 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c406a9b0 con 0x7fb6c415ad60 2024-05-08T23:32:03.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c8bc9700 1 --2- 172.21.15.81:0/3966479026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6c415ad60 0x7fb6c415d660 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009850 tx=0x7fb6b8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9e6cafeb69dbef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:03.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c37fe700 1 -- 172.21.15.81:0/3966479026 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8011830 con 0x7fb6c415ad60 2024-05-08T23:32:03.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c37fe700 1 -- 172.21.15.81:0/3966479026 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b8011990 con 0x7fb6c415ad60 2024-05-08T23:32:03.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c37fe700 1 -- 172.21.15.81:0/3966479026 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8011d20 con 0x7fb6c415ad60 2024-05-08T23:32:03.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c9bcb700 1 -- 172.21.15.81:0/3966479026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6c415ad60 msgr2=0x7fb6c415d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c9bcb700 1 --2- 172.21.15.81:0/3966479026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6c415ad60 0x7fb6c415d660 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009850 tx=0x7fb6b8009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:03.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c9bcb700 1 -- 172.21.15.81:0/3966479026 shutdown_connections 2024-05-08T23:32:03.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c9bcb700 1 --2- 172.21.15.81:0/3966479026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6c415ad60 0x7fb6c415d660 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c9bcb700 1 -- 172.21.15.81:0/3966479026 >> 172.21.15.81:0/3966479026 conn(0x7fb6c40b16c0 msgr2=0x7fb6c40b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:03.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c9bcb700 1 -- 172.21.15.81:0/3966479026 shutdown_connections 2024-05-08T23:32:03.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c9bcb700 1 -- 172.21.15.81:0/3966479026 wait complete. 2024-05-08T23:32:03.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c9bcb700 1 Processor -- start 2024-05-08T23:32:03.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c9bcb700 1 -- start start 2024-05-08T23:32:03.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c9bcb700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6c406d000 0x7fb6c4153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c9bcb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c40b36a0 0x7fb6c4156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c9bcb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6c415ad60 0x7fb6c40c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c9bcb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c415e170 con 0x7fb6c415ad60 2024-05-08T23:32:03.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c9bcb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c415e450 con 0x7fb6c40b36a0 2024-05-08T23:32:03.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c9bcb700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c415e730 con 0x7fb6c406d000 2024-05-08T23:32:03.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.208+0000 7fb6c93ca700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6c415ad60 0x7fb6c40c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c93ca700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6c415ad60 0x7fb6c40c45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53326/0 (socket says 172.21.15.81:53326) 2024-05-08T23:32:03.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c93ca700 1 -- 172.21.15.81:0/710573190 learned_addr learned my addr 172.21.15.81:0/710573190 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:03.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c8bc9700 1 --2- 172.21.15.81:0/710573190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6c406d000 0x7fb6c4153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c3fff700 1 --2- 172.21.15.81:0/710573190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c40b36a0 0x7fb6c4156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c8bc9700 1 -- 172.21.15.81:0/710573190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c40b36a0 msgr2=0x7fb6c4156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c8bc9700 1 --2- 172.21.15.81:0/710573190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c40b36a0 0x7fb6c4156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c8bc9700 1 -- 172.21.15.81:0/710573190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6c415ad60 msgr2=0x7fb6c40c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c8bc9700 1 --2- 172.21.15.81:0/710573190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6c415ad60 0x7fb6c40c45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c8bc9700 1 -- 172.21.15.81:0/710573190 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b80094b0 con 0x7fb6c406d000 2024-05-08T23:32:03.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c93ca700 1 --2- 172.21.15.81:0/710573190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6c415ad60 0x7fb6c40c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:03.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c8bc9700 1 --2- 172.21.15.81:0/710573190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6c406d000 0x7fb6c4153fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8016000 tx=0x7fb6b8004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:03.214 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c17fa700 1 -- 172.21.15.81:0/710573190 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8004550 con 0x7fb6c406d000 2024-05-08T23:32:03.214 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c17fa700 1 -- 172.21.15.81:0/710573190 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b801e070 con 0x7fb6c406d000 2024-05-08T23:32:03.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c9bcb700 1 -- 172.21.15.81:0/710573190 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c415ea10 con 0x7fb6c406d000 2024-05-08T23:32:03.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c17fa700 1 -- 172.21.15.81:0/710573190 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8003900 con 0x7fb6c406d000 2024-05-08T23:32:03.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c9bcb700 1 -- 172.21.15.81:0/710573190 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c4154520 con 0x7fb6c406d000 2024-05-08T23:32:03.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c9bcb700 1 -- 172.21.15.81:0/710573190 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c40684b0 con 0x7fb6c406d000 2024-05-08T23:32:03.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.212+0000 7fb6c17fa700 1 -- 172.21.15.81:0/710573190 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb6b8003a60 con 0x7fb6c406d000 2024-05-08T23:32:03.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.216+0000 7fb6c17fa700 1 --2- 172.21.15.81:0/710573190 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6b0066760 0x7fb6b0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.216+0000 7fb6c17fa700 1 -- 172.21.15.81:0/710573190 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fb6b8084fe0 con 0x7fb6c406d000 2024-05-08T23:32:03.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.216+0000 7fb6c3fff700 1 --2- 172.21.15.81:0/710573190 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6b0066760 0x7fb6b0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.216+0000 7fb6c3fff700 1 --2- 172.21.15.81:0/710573190 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6b0066760 0x7fb6b0068c20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4159110 tx=0x7fb6b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:03.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.228+0000 7fb6c17fa700 1 -- 172.21.15.81:0/710573190 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb6b8052170 con 0x7fb6c406d000 2024-05-08T23:32:03.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.396+0000 7fb6c9bcb700 1 -- 172.21.15.81:0/710573190 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6c40b4830 con 0x7fb6c406d000 2024-05-08T23:32:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.396+0000 7fb6c17fa700 1 -- 172.21.15.81:0/710573190 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6b8055790 con 0x7fb6c406d000 2024-05-08T23:32:03.399 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:32:03.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.400+0000 7fb6c9bcb700 1 -- 172.21.15.81:0/710573190 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6b0066760 msgr2=0x7fb6b0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.400+0000 7fb6c9bcb700 1 --2- 172.21.15.81:0/710573190 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6b0066760 0x7fb6b0068c20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4159110 tx=0x7fb6b4009040 comp rx=0 tx=0).stop 2024-05-08T23:32:03.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.400+0000 7fb6c9bcb700 1 -- 172.21.15.81:0/710573190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6c406d000 msgr2=0x7fb6c4153fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.400+0000 7fb6c9bcb700 1 --2- 172.21.15.81:0/710573190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6c406d000 0x7fb6c4153fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8016000 tx=0x7fb6b8004770 comp rx=0 tx=0).stop 2024-05-08T23:32:03.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.400+0000 7fb6c9bcb700 1 -- 172.21.15.81:0/710573190 shutdown_connections 2024-05-08T23:32:03.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.400+0000 7fb6c9bcb700 1 --2- 172.21.15.81:0/710573190 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6b0066760 0x7fb6b0068c20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.400+0000 7fb6c9bcb700 1 --2- 172.21.15.81:0/710573190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6c415ad60 0x7fb6c40c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.400+0000 7fb6c9bcb700 1 --2- 172.21.15.81:0/710573190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c40b36a0 0x7fb6c4156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.400+0000 7fb6c9bcb700 1 --2- 172.21.15.81:0/710573190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6c406d000 0x7fb6c4153fe0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.400+0000 7fb6c9bcb700 1 -- 172.21.15.81:0/710573190 >> 172.21.15.81:0/710573190 conn(0x7fb6c40b16c0 msgr2=0x7fb6c406b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:03.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.400+0000 7fb6c9bcb700 1 -- 172.21.15.81:0/710573190 shutdown_connections 2024-05-08T23:32:03.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.400+0000 7fb6c9bcb700 1 -- 172.21.15.81:0/710573190 wait complete. 2024-05-08T23:32:03.415 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:32:03.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.532+0000 7f1b2da91700 1 Processor -- start 2024-05-08T23:32:03.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.532+0000 7f1b2da91700 1 -- start start 2024-05-08T23:32:03.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b2da91700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1b280d1da0 0x7f1b280cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b2da91700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b280d3d40 con 0x7f1b28160920 2024-05-08T23:32:03.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b2da91700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b280d02d0 con 0x7f1b280cfe80 2024-05-08T23:32:03.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b2da91700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b280d0d00 con 0x7f1b280d1da0 2024-05-08T23:32:03.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b27fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1b280d1da0 0x7f1b280cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b2ca8f700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f1b28160920 0x7f1b280ce6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47172/0 (socket says 172.21.15.81:47172) 2024-05-08T23:32:03.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b2ca8f700 1 -- 172.21.15.81:0/300782072 learned_addr learned my addr 172.21.15.81:0/300782072 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:03.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b27fff700 1 -- 172.21.15.81:0/300782072 >> v1:172.21.15.117:6789/0 conn(0x7f1b280cfe80 legacy=0x7f1b280b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b27fff700 1 -- 172.21.15.81:0/300782072 >> v1:172.21.15.81:6789/0 conn(0x7f1b28160920 legacy=0x7f1b280ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b27fff700 1 -- 172.21.15.81:0/300782072 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b280d0fe0 con 0x7f1b280d1da0 2024-05-08T23:32:03.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b27fff700 1 --2- 172.21.15.81:0/300782072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b280d1da0 0x7f1b280cf940 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009850 tx=0x7f1b10009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bd333d63e9a141 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:03.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b277fe700 1 -- 172.21.15.81:0/300782072 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10011830 con 0x7f1b280d1da0 2024-05-08T23:32:03.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b277fe700 1 -- 172.21.15.81:0/300782072 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b10011990 con 0x7f1b280d1da0 2024-05-08T23:32:03.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b277fe700 1 -- 172.21.15.81:0/300782072 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10011d20 con 0x7f1b280d1da0 2024-05-08T23:32:03.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b2da91700 1 -- 172.21.15.81:0/300782072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b280d1da0 msgr2=0x7f1b280cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b2da91700 1 --2- 172.21.15.81:0/300782072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b280d1da0 0x7f1b280cf940 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009850 tx=0x7f1b10009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:03.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b2da91700 1 -- 172.21.15.81:0/300782072 shutdown_connections 2024-05-08T23:32:03.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b2da91700 1 --2- 172.21.15.81:0/300782072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b280d1da0 0x7f1b280cf940 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b2da91700 1 -- 172.21.15.81:0/300782072 >> 172.21.15.81:0/300782072 conn(0x7f1b2809c340 msgr2=0x7f1b280d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:03.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b2da91700 1 -- 172.21.15.81:0/300782072 shutdown_connections 2024-05-08T23:32:03.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.536+0000 7f1b2da91700 1 -- 172.21.15.81:0/300782072 wait complete. 2024-05-08T23:32:03.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.540+0000 7f1b2da91700 1 Processor -- start 2024-05-08T23:32:03.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.540+0000 7f1b2da91700 1 -- start start 2024-05-08T23:32:03.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.540+0000 7f1b2da91700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b280cfe80 0x7f1b280c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.540+0000 7f1b2da91700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b280d1da0 0x7f1b28159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.540+0000 7f1b2da91700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b28160920 0x7f1b280b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.540+0000 7f1b2da91700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b281f33b0 con 0x7f1b28160920 2024-05-08T23:32:03.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.540+0000 7f1b2da91700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b281e7060 con 0x7f1b280d1da0 2024-05-08T23:32:03.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.540+0000 7f1b2da91700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b280b8180 con 0x7f1b280cfe80 2024-05-08T23:32:03.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.540+0000 7f1b2d290700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b28160920 0x7f1b280b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.540+0000 7f1b2ca8f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b280cfe80 0x7f1b280c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.540+0000 7f1b2ca8f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b280cfe80 0x7f1b280c9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:47722/0 (socket says 172.21.15.81:47722) 2024-05-08T23:32:03.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.540+0000 7f1b2ca8f700 1 -- 172.21.15.81:0/1912157561 learned_addr learned my addr 172.21.15.81:0/1912157561 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:03.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.540+0000 7f1b27fff700 1 --2- 172.21.15.81:0/1912157561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b280d1da0 0x7f1b28159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.544+0000 7f1b2ca8f700 1 -- 172.21.15.81:0/1912157561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b280d1da0 msgr2=0x7f1b28159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.544+0000 7f1b2ca8f700 1 --2- 172.21.15.81:0/1912157561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b280d1da0 0x7f1b28159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.544+0000 7f1b2ca8f700 1 -- 172.21.15.81:0/1912157561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b28160920 msgr2=0x7f1b280b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.544+0000 7f1b2ca8f700 1 --2- 172.21.15.81:0/1912157561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b28160920 0x7f1b280b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.544+0000 7f1b2ca8f700 1 -- 172.21.15.81:0/1912157561 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b100094b0 con 0x7f1b280cfe80 2024-05-08T23:32:03.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.544+0000 7f1b2d290700 1 --2- 172.21.15.81:0/1912157561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b28160920 0x7f1b280b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:03.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.544+0000 7f1b2ca8f700 1 --2- 172.21.15.81:0/1912157561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b280cfe80 0x7f1b280c9a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800c4d0 tx=0x7f1b1800c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:03.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.544+0000 7f1b257fa700 1 -- 172.21.15.81:0/1912157561 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1800cea0 con 0x7f1b280cfe80 2024-05-08T23:32:03.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.544+0000 7f1b257fa700 1 -- 172.21.15.81:0/1912157561 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b18005d90 con 0x7f1b280cfe80 2024-05-08T23:32:03.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.544+0000 7f1b2da91700 1 -- 172.21.15.81:0/1912157561 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b281e72b0 con 0x7f1b280cfe80 2024-05-08T23:32:03.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.544+0000 7f1b257fa700 1 -- 172.21.15.81:0/1912157561 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b180072d0 con 0x7f1b280cfe80 2024-05-08T23:32:03.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.544+0000 7f1b2da91700 1 -- 172.21.15.81:0/1912157561 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b281e7870 con 0x7f1b280cfe80 2024-05-08T23:32:03.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.544+0000 7f1b2da91700 1 -- 172.21.15.81:0/1912157561 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b281e7a60 con 0x7f1b280cfe80 2024-05-08T23:32:03.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.544+0000 7f1b257fa700 1 -- 172.21.15.81:0/1912157561 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1b18007430 con 0x7f1b280cfe80 2024-05-08T23:32:03.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.544+0000 7f1b257fa700 1 --2- 172.21.15.81:0/1912157561 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1b08066730 0x7f1b08068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.548+0000 7f1b257fa700 1 -- 172.21.15.81:0/1912157561 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f1b18083e70 con 0x7f1b280cfe80 2024-05-08T23:32:03.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.548+0000 7f1b27fff700 1 --2- 172.21.15.81:0/1912157561 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1b08066730 0x7f1b08068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.548+0000 7f1b27fff700 1 --2- 172.21.15.81:0/1912157561 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1b08066730 0x7f1b08068bf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1b10014040 tx=0x7f1b1001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:03.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.556+0000 7f1b257fa700 1 -- 172.21.15.81:0/1912157561 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1b18051070 con 0x7f1b280cfe80 2024-05-08T23:32:03.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.724+0000 7f1b2da91700 1 -- 172.21.15.81:0/1912157561 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1b280684b0 con 0x7f1b280cfe80 2024-05-08T23:32:03.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.724+0000 7f1b257fa700 1 -- 172.21.15.81:0/1912157561 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1b180170a0 con 0x7f1b280cfe80 2024-05-08T23:32:03.726 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:32:03.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.728+0000 7f1b2da91700 1 -- 172.21.15.81:0/1912157561 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1b08066730 msgr2=0x7f1b08068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.728+0000 7f1b2da91700 1 --2- 172.21.15.81:0/1912157561 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1b08066730 0x7f1b08068bf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1b10014040 tx=0x7f1b1001c040 comp rx=0 tx=0).stop 2024-05-08T23:32:03.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.728+0000 7f1b2da91700 1 -- 172.21.15.81:0/1912157561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b280cfe80 msgr2=0x7f1b280c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.728+0000 7f1b2da91700 1 --2- 172.21.15.81:0/1912157561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b280cfe80 0x7f1b280c9a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800c4d0 tx=0x7f1b1800c7e0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.728+0000 7f1b2da91700 1 -- 172.21.15.81:0/1912157561 shutdown_connections 2024-05-08T23:32:03.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.728+0000 7f1b2da91700 1 --2- 172.21.15.81:0/1912157561 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1b08066730 0x7f1b08068bf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.728+0000 7f1b2da91700 1 --2- 172.21.15.81:0/1912157561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b28160920 0x7f1b280b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.728+0000 7f1b2da91700 1 --2- 172.21.15.81:0/1912157561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b280d1da0 0x7f1b28159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.728+0000 7f1b2da91700 1 --2- 172.21.15.81:0/1912157561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b280cfe80 0x7f1b280c9a00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.728+0000 7f1b2da91700 1 -- 172.21.15.81:0/1912157561 >> 172.21.15.81:0/1912157561 conn(0x7f1b2809c340 msgr2=0x7f1b280ce580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:03.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.728+0000 7f1b2da91700 1 -- 172.21.15.81:0/1912157561 shutdown_connections 2024-05-08T23:32:03.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.728+0000 7f1b2da91700 1 -- 172.21.15.81:0/1912157561 wait complete. 2024-05-08T23:32:03.741 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:32:03.853 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.852+0000 7febdaabc700 1 Processor -- start 2024-05-08T23:32:03.853 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.852+0000 7febdaabc700 1 -- start start 2024-05-08T23:32:03.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.852+0000 7febdaabc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7febd40d1cf0 0x7febd4167f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.852+0000 7febdaabc700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd40cebe0 con 0x7febd415d410 2024-05-08T23:32:03.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.852+0000 7febdaabc700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd40ceec0 con 0x7febd40ce620 2024-05-08T23:32:03.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.852+0000 7febdaabc700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd40b24f0 con 0x7febd40d1cf0 2024-05-08T23:32:03.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.852+0000 7febd9aba700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7febd415d410 0x7febd40d3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47188/0 (socket says 172.21.15.81:47188) 2024-05-08T23:32:03.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.852+0000 7febd9aba700 1 -- 172.21.15.81:0/1012210440 learned_addr learned my addr 172.21.15.81:0/1012210440 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:03.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.852+0000 7febda2bb700 1 --2- 172.21.15.81:0/1012210440 >> v2:172.21.15.117:3300/0 conn(0x7febd40d1cf0 0x7febd4167f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.852+0000 7febd8ab8700 1 -- 172.21.15.81:0/1012210440 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2200962260 0 0) 0x7febd40cebe0 con 0x7febd415d410 2024-05-08T23:32:03.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.852+0000 7febd8ab8700 1 -- 172.21.15.81:0/1012210440 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febc00035d0 con 0x7febd415d410 2024-05-08T23:32:03.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.852+0000 7febd8ab8700 1 -- 172.21.15.81:0/1012210440 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1180195580 0 0) 0x7febc00035d0 con 0x7febd415d410 2024-05-08T23:32:03.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.852+0000 7febda2bb700 1 -- 172.21.15.81:0/1012210440 >> v1:172.21.15.117:6789/0 conn(0x7febd40ce620 legacy=0x7febd40d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.852+0000 7febda2bb700 1 -- 172.21.15.81:0/1012210440 >> v1:172.21.15.81:6789/0 conn(0x7febd415d410 legacy=0x7febd40d3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.852+0000 7febda2bb700 1 -- 172.21.15.81:0/1012210440 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd40b27d0 con 0x7febd40d1cf0 2024-05-08T23:32:03.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.856+0000 7febda2bb700 1 --2- 172.21.15.81:0/1012210440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd40d1cf0 0x7febd4167f90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7febcc009d90 tx=0x7febcc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=932176ef278b108c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:03.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.856+0000 7febd8ab8700 1 -- 172.21.15.81:0/1012210440 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc00e070 con 0x7febd40d1cf0 2024-05-08T23:32:03.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.856+0000 7febd8ab8700 1 -- 172.21.15.81:0/1012210440 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc003a40 con 0x7febd40d1cf0 2024-05-08T23:32:03.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.856+0000 7febd8ab8700 1 -- 172.21.15.81:0/1012210440 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0001670 con 0x7febd40d1cf0 2024-05-08T23:32:03.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.856+0000 7febdaabc700 1 -- 172.21.15.81:0/1012210440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd40d1cf0 msgr2=0x7febd4167f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.856+0000 7febdaabc700 1 --2- 172.21.15.81:0/1012210440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd40d1cf0 0x7febd4167f90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7febcc009d90 tx=0x7febcc00b330 comp rx=0 tx=0).stop 2024-05-08T23:32:03.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.856+0000 7febdaabc700 1 -- 172.21.15.81:0/1012210440 shutdown_connections 2024-05-08T23:32:03.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.856+0000 7febdaabc700 1 --2- 172.21.15.81:0/1012210440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd40d1cf0 0x7febd4167f90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.856+0000 7febdaabc700 1 -- 172.21.15.81:0/1012210440 >> 172.21.15.81:0/1012210440 conn(0x7febd409c340 msgr2=0x7febd40d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:03.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.856+0000 7febdaabc700 1 -- 172.21.15.81:0/1012210440 shutdown_connections 2024-05-08T23:32:03.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.856+0000 7febdaabc700 1 -- 172.21.15.81:0/1012210440 wait complete. 2024-05-08T23:32:03.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.856+0000 7febdaabc700 1 Processor -- start 2024-05-08T23:32:03.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.856+0000 7febdaabc700 1 -- start start 2024-05-08T23:32:03.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febdaabc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febd40ce620 0x7febd40cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febdaabc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd40d1cf0 0x7febd4159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febdaabc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415d410 0x7febd40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febdaabc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40b4c40 con 0x7febd40ce620 2024-05-08T23:32:03.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febdaabc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40b4960 con 0x7febd40d1cf0 2024-05-08T23:32:03.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febdaabc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd40b7b50 con 0x7febd415d410 2024-05-08T23:32:03.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febd9aba700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febd40ce620 0x7febd40cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febd9aba700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febd40ce620 0x7febd40cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53340/0 (socket says 172.21.15.81:53340) 2024-05-08T23:32:03.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febd9aba700 1 -- 172.21.15.81:0/3098546293 learned_addr learned my addr 172.21.15.81:0/3098546293 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:03.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febda2bb700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415d410 0x7febd40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febd92b9700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd40d1cf0 0x7febd4159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febd9aba700 1 -- 172.21.15.81:0/3098546293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415d410 msgr2=0x7febd40b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febd9aba700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415d410 0x7febd40b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febd9aba700 1 -- 172.21.15.81:0/3098546293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd40d1cf0 msgr2=0x7febd4159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febd9aba700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd40d1cf0 0x7febd4159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febd9aba700 1 -- 172.21.15.81:0/3098546293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc009a70 con 0x7febd40ce620 2024-05-08T23:32:03.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febda2bb700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415d410 0x7febd40b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:03.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febd92b9700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd40d1cf0 0x7febd4159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:03.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febd9aba700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febd40ce620 0x7febd40cc920 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7febd000c540 tx=0x7febd000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:03.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febc67fc700 1 -- 172.21.15.81:0/3098546293 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0004ca0 con 0x7febd40ce620 2024-05-08T23:32:03.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febdaabc700 1 -- 172.21.15.81:0/3098546293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd40b7e30 con 0x7febd40ce620 2024-05-08T23:32:03.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febc67fc700 1 -- 172.21.15.81:0/3098546293 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd0004e00 con 0x7febd40ce620 2024-05-08T23:32:03.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febc67fc700 1 -- 172.21.15.81:0/3098546293 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0007310 con 0x7febd40ce620 2024-05-08T23:32:03.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febdaabc700 1 -- 172.21.15.81:0/3098546293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd4159bd0 con 0x7febd40ce620 2024-05-08T23:32:03.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febdaabc700 1 -- 172.21.15.81:0/3098546293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd40b8350 con 0x7febd40ce620 2024-05-08T23:32:03.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.860+0000 7febc67fc700 1 -- 172.21.15.81:0/3098546293 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7febd0007470 con 0x7febd40ce620 2024-05-08T23:32:03.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.864+0000 7febc67fc700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febb8066730 0x7febb8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.864+0000 7febd92b9700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febb8066730 0x7febb8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.864+0000 7febc67fc700 1 -- 172.21.15.81:0/3098546293 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7febd00840c0 con 0x7febd40ce620 2024-05-08T23:32:03.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.864+0000 7febd92b9700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febb8066730 0x7febb8068bf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7febd415bbe0 tx=0x7febc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:03.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:03.872+0000 7febc67fc700 1 -- 172.21.15.81:0/3098546293 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7febd00512b0 con 0x7febd40ce620 2024-05-08T23:32:04.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.039+0000 7febdaabc700 1 -- 172.21.15.81:0/3098546293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7febd415d7f0 con 0x7febd40ce620 2024-05-08T23:32:04.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.039+0000 7febc67fc700 1 -- 172.21.15.81:0/3098546293 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7febd0007720 con 0x7febd40ce620 2024-05-08T23:32:04.043 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:32:04.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.043+0000 7febdaabc700 1 -- 172.21.15.81:0/3098546293 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febb8066730 msgr2=0x7febb8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.043+0000 7febdaabc700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febb8066730 0x7febb8068bf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7febd415bbe0 tx=0x7febc8009040 comp rx=0 tx=0).stop 2024-05-08T23:32:04.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.043+0000 7febdaabc700 1 -- 172.21.15.81:0/3098546293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febd40ce620 msgr2=0x7febd40cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.043+0000 7febdaabc700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febd40ce620 0x7febd40cc920 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7febd000c540 tx=0x7febd000c900 comp rx=0 tx=0).stop 2024-05-08T23:32:04.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.043+0000 7febdaabc700 1 -- 172.21.15.81:0/3098546293 shutdown_connections 2024-05-08T23:32:04.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.043+0000 7febdaabc700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febb8066730 0x7febb8068bf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.043+0000 7febdaabc700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415d410 0x7febd40b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.047 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.043+0000 7febdaabc700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd40d1cf0 0x7febd4159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.048 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.043+0000 7febdaabc700 1 --2- 172.21.15.81:0/3098546293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febd40ce620 0x7febd40cc920 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.048 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.043+0000 7febdaabc700 1 -- 172.21.15.81:0/3098546293 >> 172.21.15.81:0/3098546293 conn(0x7febd409c340 msgr2=0x7febd40b8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:04.048 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.043+0000 7febdaabc700 1 -- 172.21.15.81:0/3098546293 shutdown_connections 2024-05-08T23:32:04.048 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.043+0000 7febdaabc700 1 -- 172.21.15.81:0/3098546293 wait complete. 2024-05-08T23:32:04.061 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-05-08T23:32:04.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.175+0000 7f0b2d7d5700 1 Processor -- start 2024-05-08T23:32:04.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.175+0000 7f0b2d7d5700 1 -- start start 2024-05-08T23:32:04.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2d7d5700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b200e1110 0x7f0b20054e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2d7d5700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0b2003cad0 0x7f0b20051b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2d7d5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0b20056870 0x7f0b200520a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2d7d5700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b200563e0 con 0x7f0b20056870 2024-05-08T23:32:04.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2d7d5700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b200e14f0 con 0x7f0b2003cad0 2024-05-08T23:32:04.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2d7d5700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b200557d0 con 0x7f0b200e1110 2024-05-08T23:32:04.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2cfd4700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0b20056870 0x7f0b200520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2cfd4700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0b20056870 0x7f0b200520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53356/0 (socket says 172.21.15.81:53356) 2024-05-08T23:32:04.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2cfd4700 1 -- 172.21.15.81:0/3041107001 learned_addr learned my addr 172.21.15.81:0/3041107001 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:04.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b27fff700 1 --2- 172.21.15.81:0/3041107001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b200e1110 0x7f0b20054e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b277fe700 1 --2- 172.21.15.81:0/3041107001 >> v2:172.21.15.117:3300/0 conn(0x7f0b2003cad0 0x7f0b20051b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2cfd4700 1 -- 172.21.15.81:0/3041107001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b200e1110 msgr2=0x7f0b20054e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:04.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2cfd4700 1 --2- 172.21.15.81:0/3041107001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b200e1110 0x7f0b20054e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2cfd4700 1 -- 172.21.15.81:0/3041107001 >> v2:172.21.15.117:3300/0 conn(0x7f0b2003cad0 msgr2=0x7f0b20051b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:04.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2cfd4700 1 --2- 172.21.15.81:0/3041107001 >> v2:172.21.15.117:3300/0 conn(0x7f0b2003cad0 0x7f0b20051b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2cfd4700 1 -- 172.21.15.81:0/3041107001 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b20055ab0 con 0x7f0b20056870 2024-05-08T23:32:04.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2cfd4700 1 --2- 172.21.15.81:0/3041107001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b20056870 0x7f0b200520a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0b280698a0 tx=0x7f0b28068970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704ee6c9d6e9cfd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:04.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b26ffd700 1 -- 172.21.15.81:0/3041107001 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b280679b0 con 0x7f0b20056870 2024-05-08T23:32:04.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b26ffd700 1 -- 172.21.15.81:0/3041107001 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b28067b10 con 0x7f0b20056870 2024-05-08T23:32:04.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b26ffd700 1 -- 172.21.15.81:0/3041107001 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28072130 con 0x7f0b20056870 2024-05-08T23:32:04.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2d7d5700 1 -- 172.21.15.81:0/3041107001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b20056870 msgr2=0x7f0b200520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2d7d5700 1 --2- 172.21.15.81:0/3041107001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b20056870 0x7f0b200520a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0b280698a0 tx=0x7f0b28068970 comp rx=0 tx=0).stop 2024-05-08T23:32:04.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2d7d5700 1 -- 172.21.15.81:0/3041107001 shutdown_connections 2024-05-08T23:32:04.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2d7d5700 1 --2- 172.21.15.81:0/3041107001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b20056870 0x7f0b200520a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2d7d5700 1 --2- 172.21.15.81:0/3041107001 >> v2:172.21.15.117:3300/0 conn(0x7f0b2003cad0 0x7f0b20051b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2d7d5700 1 --2- 172.21.15.81:0/3041107001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b200e1110 0x7f0b20054e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2d7d5700 1 -- 172.21.15.81:0/3041107001 >> 172.21.15.81:0/3041107001 conn(0x7f0b20056d90 msgr2=0x7f0b200571a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:04.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2d7d5700 1 -- 172.21.15.81:0/3041107001 shutdown_connections 2024-05-08T23:32:04.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.179+0000 7f0b2d7d5700 1 -- 172.21.15.81:0/3041107001 wait complete. 2024-05-08T23:32:04.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.183+0000 7f0b2d7d5700 1 Processor -- start 2024-05-08T23:32:04.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.183+0000 7f0b2d7d5700 1 -- start start 2024-05-08T23:32:04.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.183+0000 7f0b2d7d5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2003cad0 0x7f0b200d5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.183+0000 7f0b2d7d5700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b20056870 0x7f0b200367f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.183+0000 7f0b2d7d5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b200e1110 0x7f0b20036d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.183+0000 7f0b2d7d5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b200e4210 con 0x7f0b200e1110 2024-05-08T23:32:04.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.183+0000 7f0b2d7d5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b200e3600 con 0x7f0b2003cad0 2024-05-08T23:32:04.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.183+0000 7f0b2d7d5700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b200e38e0 con 0x7f0b20056870 2024-05-08T23:32:04.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b2cfd4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b200e1110 0x7f0b20036d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b2cfd4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b200e1110 0x7f0b20036d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53372/0 (socket says 172.21.15.81:53372) 2024-05-08T23:32:04.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b2cfd4700 1 -- 172.21.15.81:0/4265109551 learned_addr learned my addr 172.21.15.81:0/4265109551 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:04.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b277fe700 1 --2- 172.21.15.81:0/4265109551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b20056870 0x7f0b200367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b27fff700 1 --2- 172.21.15.81:0/4265109551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2003cad0 0x7f0b200d5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b2cfd4700 1 -- 172.21.15.81:0/4265109551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b20056870 msgr2=0x7f0b200367f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b2cfd4700 1 --2- 172.21.15.81:0/4265109551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b20056870 0x7f0b200367f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b2cfd4700 1 -- 172.21.15.81:0/4265109551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2003cad0 msgr2=0x7f0b200d5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b2cfd4700 1 --2- 172.21.15.81:0/4265109551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2003cad0 0x7f0b200d5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b2cfd4700 1 -- 172.21.15.81:0/4265109551 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2804f5c0 con 0x7f0b200e1110 2024-05-08T23:32:04.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b2cfd4700 1 --2- 172.21.15.81:0/4265109551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b200e1110 0x7f0b20036d30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0b280795b0 tx=0x7f0b28065ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:04.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b24ff9700 1 -- 172.21.15.81:0/4265109551 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b280679b0 con 0x7f0b200e1110 2024-05-08T23:32:04.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b24ff9700 1 -- 172.21.15.81:0/4265109551 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b28093460 con 0x7f0b200e1110 2024-05-08T23:32:04.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b2d7d5700 1 -- 172.21.15.81:0/4265109551 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b200e3bc0 con 0x7f0b200e1110 2024-05-08T23:32:04.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b24ff9700 1 -- 172.21.15.81:0/4265109551 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b280937d0 con 0x7f0b200e1110 2024-05-08T23:32:04.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b2d7d5700 1 -- 172.21.15.81:0/4265109551 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b200d55b0 con 0x7f0b200e1110 2024-05-08T23:32:04.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b2d7d5700 1 -- 172.21.15.81:0/4265109551 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b20007480 con 0x7f0b200e1110 2024-05-08T23:32:04.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.187+0000 7f0b24ff9700 1 -- 172.21.15.81:0/4265109551 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0b28095450 con 0x7f0b200e1110 2024-05-08T23:32:04.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.191+0000 7f0b24ff9700 1 --2- 172.21.15.81:0/4265109551 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0b0c066730 0x7f0b0c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.191+0000 7f0b24ff9700 1 -- 172.21.15.81:0/4265109551 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f0b280ffd00 con 0x7f0b200e1110 2024-05-08T23:32:04.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.191+0000 7f0b27fff700 1 --2- 172.21.15.81:0/4265109551 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0b0c066730 0x7f0b0c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.191+0000 7f0b27fff700 1 --2- 172.21.15.81:0/4265109551 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0b0c066730 0x7f0b0c068bf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0b200398a0 tx=0x7f0b14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:04.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.195+0000 7f0b24ff9700 1 -- 172.21.15.81:0/4265109551 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0b280ccf70 con 0x7f0b200e1110 2024-05-08T23:32:04.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.363+0000 7f0b2d7d5700 1 -- 172.21.15.81:0/4265109551 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f0b20005440 con 0x7f0b200e1110 2024-05-08T23:32:04.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.367+0000 7f0b24ff9700 1 -- 172.21.15.81:0/4265109551 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v39) v1 ==== 97+0+11 (secure 0 0 0) 0x7f0b280d0590 con 0x7f0b200e1110 2024-05-08T23:32:04.368 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:32:04.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.367+0000 7f0b2d7d5700 1 -- 172.21.15.81:0/4265109551 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0b0c066730 msgr2=0x7f0b0c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.367+0000 7f0b2d7d5700 1 --2- 172.21.15.81:0/4265109551 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0b0c066730 0x7f0b0c068bf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0b200398a0 tx=0x7f0b14008040 comp rx=0 tx=0).stop 2024-05-08T23:32:04.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.367+0000 7f0b2d7d5700 1 -- 172.21.15.81:0/4265109551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b200e1110 msgr2=0x7f0b20036d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.367+0000 7f0b2d7d5700 1 --2- 172.21.15.81:0/4265109551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b200e1110 0x7f0b20036d30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0b280795b0 tx=0x7f0b28065ea0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.367+0000 7f0b2d7d5700 1 -- 172.21.15.81:0/4265109551 shutdown_connections 2024-05-08T23:32:04.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.367+0000 7f0b2d7d5700 1 --2- 172.21.15.81:0/4265109551 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0b0c066730 0x7f0b0c068bf0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.367+0000 7f0b2d7d5700 1 --2- 172.21.15.81:0/4265109551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b200e1110 0x7f0b20036d30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.367+0000 7f0b2d7d5700 1 --2- 172.21.15.81:0/4265109551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b20056870 0x7f0b200367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.367+0000 7f0b2d7d5700 1 --2- 172.21.15.81:0/4265109551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2003cad0 0x7f0b200d5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.367+0000 7f0b2d7d5700 1 -- 172.21.15.81:0/4265109551 >> 172.21.15.81:0/4265109551 conn(0x7f0b20056d90 msgr2=0x7f0b20009e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:04.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.367+0000 7f0b2d7d5700 1 -- 172.21.15.81:0/4265109551 shutdown_connections 2024-05-08T23:32:04.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.367+0000 7f0b2d7d5700 1 -- 172.21.15.81:0/4265109551 wait complete. 2024-05-08T23:32:04.383 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:32:04.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.499+0000 7fbc67fff700 1 Processor -- start 2024-05-08T23:32:04.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.499+0000 7fbc67fff700 1 -- start start 2024-05-08T23:32:04.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.499+0000 7fbc67fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbc680ae9f0 0x7fbc680b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.499+0000 7fbc67fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbc6815a6a0 0x7fbc6806c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.499+0000 7fbc67fff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6815ab30 con 0x7fbc6806ca50 2024-05-08T23:32:04.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.499+0000 7fbc67fff700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc680b3420 con 0x7fbc6815a6a0 2024-05-08T23:32:04.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.499+0000 7fbc67fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc680b2b80 con 0x7fbc680ae9f0 2024-05-08T23:32:04.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc677fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fbc6806ca50 0x7fbc680b4140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47198/0 (socket says 172.21.15.81:47198) 2024-05-08T23:32:04.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc677fe700 1 -- 172.21.15.81:0/3231463393 learned_addr learned my addr 172.21.15.81:0/3231463393 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:04.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc667fc700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbc6815a6a0 0x7fbc6806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc66ffd700 1 --2- 172.21.15.81:0/3231463393 >> v2:172.21.15.117:3300/0 conn(0x7fbc680ae9f0 0x7fbc680b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc65ffb700 1 -- 172.21.15.81:0/3231463393 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130574179 0 0) 0x7fbc6815ab30 con 0x7fbc6806ca50 2024-05-08T23:32:04.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc65ffb700 1 -- 172.21.15.81:0/3231463393 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc500035d0 con 0x7fbc6806ca50 2024-05-08T23:32:04.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc667fc700 1 -- 172.21.15.81:0/3231463393 >> v2:172.21.15.117:3300/0 conn(0x7fbc680ae9f0 msgr2=0x7fbc680b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:04.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc667fc700 1 --2- 172.21.15.81:0/3231463393 >> v2:172.21.15.117:3300/0 conn(0x7fbc680ae9f0 0x7fbc680b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc667fc700 1 -- 172.21.15.81:0/3231463393 >> v1:172.21.15.81:6789/0 conn(0x7fbc6806ca50 legacy=0x7fbc680b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc667fc700 1 -- 172.21.15.81:0/3231463393 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc680b2e60 con 0x7fbc6815a6a0 2024-05-08T23:32:04.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc667fc700 1 --2- 172.21.15.81:0/3231463393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc6815a6a0 0x7fbc6806c510 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009850 tx=0x7fbc5c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376ec64cb57f756b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:04.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc65ffb700 1 -- 172.21.15.81:0/3231463393 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c011830 con 0x7fbc6815a6a0 2024-05-08T23:32:04.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc65ffb700 1 -- 172.21.15.81:0/3231463393 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5c011990 con 0x7fbc6815a6a0 2024-05-08T23:32:04.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc65ffb700 1 -- 172.21.15.81:0/3231463393 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c011d20 con 0x7fbc6815a6a0 2024-05-08T23:32:04.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc67fff700 1 -- 172.21.15.81:0/3231463393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc6815a6a0 msgr2=0x7fbc6806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc67fff700 1 --2- 172.21.15.81:0/3231463393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc6815a6a0 0x7fbc6806c510 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009850 tx=0x7fbc5c009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:04.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc67fff700 1 -- 172.21.15.81:0/3231463393 shutdown_connections 2024-05-08T23:32:04.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc67fff700 1 --2- 172.21.15.81:0/3231463393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc6815a6a0 0x7fbc6806c510 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc67fff700 1 --2- 172.21.15.81:0/3231463393 >> v2:172.21.15.117:3300/0 conn(0x7fbc680ae9f0 0x7fbc680b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc67fff700 1 -- 172.21.15.81:0/3231463393 >> 172.21.15.81:0/3231463393 conn(0x7fbc68070790 msgr2=0x7fbc68070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:04.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc67fff700 1 -- 172.21.15.81:0/3231463393 shutdown_connections 2024-05-08T23:32:04.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc67fff700 1 -- 172.21.15.81:0/3231463393 wait complete. 2024-05-08T23:32:04.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc67fff700 1 Processor -- start 2024-05-08T23:32:04.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.503+0000 7fbc67fff700 1 -- start start 2024-05-08T23:32:04.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc67fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc6806ca50 0x7fbc681464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc67fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc680ae9f0 0x7fbc6814f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc67fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc6815a6a0 0x7fbc680ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc67fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6815b1f0 con 0x7fbc680ae9f0 2024-05-08T23:32:04.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc67fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6815b770 con 0x7fbc6815a6a0 2024-05-08T23:32:04.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc67fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6814fb10 con 0x7fbc6806ca50 2024-05-08T23:32:04.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc667fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc680ae9f0 0x7fbc6814f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc667fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc680ae9f0 0x7fbc6814f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53400/0 (socket says 172.21.15.81:53400) 2024-05-08T23:32:04.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc667fc700 1 -- 172.21.15.81:0/2066747539 learned_addr learned my addr 172.21.15.81:0/2066747539 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:04.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc66ffd700 1 --2- 172.21.15.81:0/2066747539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc6806ca50 0x7fbc681464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc677fe700 1 --2- 172.21.15.81:0/2066747539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc6815a6a0 0x7fbc680ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc667fc700 1 -- 172.21.15.81:0/2066747539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc6806ca50 msgr2=0x7fbc681464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc667fc700 1 --2- 172.21.15.81:0/2066747539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc6806ca50 0x7fbc681464a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc667fc700 1 -- 172.21.15.81:0/2066747539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc6815a6a0 msgr2=0x7fbc680ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc667fc700 1 --2- 172.21.15.81:0/2066747539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc6815a6a0 0x7fbc680ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc667fc700 1 -- 172.21.15.81:0/2066747539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c0094b0 con 0x7fbc680ae9f0 2024-05-08T23:32:04.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc667fc700 1 --2- 172.21.15.81:0/2066747539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc680ae9f0 0x7fbc6814f050 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009ba0 tx=0x7fbc5c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:04.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc4f7fe700 1 -- 172.21.15.81:0/2066747539 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c0036c0 con 0x7fbc680ae9f0 2024-05-08T23:32:04.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc4f7fe700 1 -- 172.21.15.81:0/2066747539 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5c01e070 con 0x7fbc680ae9f0 2024-05-08T23:32:04.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc67fff700 1 -- 172.21.15.81:0/2066747539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6814fdf0 con 0x7fbc680ae9f0 2024-05-08T23:32:04.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc4f7fe700 1 -- 172.21.15.81:0/2066747539 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c003a60 con 0x7fbc680ae9f0 2024-05-08T23:32:04.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc67fff700 1 -- 172.21.15.81:0/2066747539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc681469e0 con 0x7fbc680ae9f0 2024-05-08T23:32:04.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.507+0000 7fbc67fff700 1 -- 172.21.15.81:0/2066747539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc680b56b0 con 0x7fbc680ae9f0 2024-05-08T23:32:04.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.511+0000 7fbc4f7fe700 1 -- 172.21.15.81:0/2066747539 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbc5c003bc0 con 0x7fbc680ae9f0 2024-05-08T23:32:04.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.511+0000 7fbc4f7fe700 1 --2- 172.21.15.81:0/2066747539 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbc54066730 0x7fbc54068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.511+0000 7fbc4f7fe700 1 -- 172.21.15.81:0/2066747539 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fbc5c085350 con 0x7fbc680ae9f0 2024-05-08T23:32:04.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.511+0000 7fbc66ffd700 1 --2- 172.21.15.81:0/2066747539 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbc54066730 0x7fbc54068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.511+0000 7fbc66ffd700 1 --2- 172.21.15.81:0/2066747539 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbc54066730 0x7fbc54068bf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009b80 tx=0x7fbc58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:04.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.519+0000 7fbc4f7fe700 1 -- 172.21.15.81:0/2066747539 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbc5c0525c0 con 0x7fbc680ae9f0 2024-05-08T23:32:04.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.699+0000 7fbc67fff700 1 -- 172.21.15.81:0/2066747539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbc6804e700 con 0x7fbc680ae9f0 2024-05-08T23:32:04.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.699+0000 7fbc4f7fe700 1 -- 172.21.15.81:0/2066747539 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbc5c055be0 con 0x7fbc680ae9f0 2024-05-08T23:32:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.703+0000 7fbc67fff700 1 -- 172.21.15.81:0/2066747539 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbc54066730 msgr2=0x7fbc54068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.703+0000 7fbc67fff700 1 --2- 172.21.15.81:0/2066747539 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbc54066730 0x7fbc54068bf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009b80 tx=0x7fbc58008040 comp rx=0 tx=0).stop 2024-05-08T23:32:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.703+0000 7fbc67fff700 1 -- 172.21.15.81:0/2066747539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc680ae9f0 msgr2=0x7fbc6814f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.703+0000 7fbc67fff700 1 --2- 172.21.15.81:0/2066747539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc680ae9f0 0x7fbc6814f050 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009ba0 tx=0x7fbc5c004770 comp rx=0 tx=0).stop 2024-05-08T23:32:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.703+0000 7fbc67fff700 1 -- 172.21.15.81:0/2066747539 shutdown_connections 2024-05-08T23:32:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.703+0000 7fbc67fff700 1 --2- 172.21.15.81:0/2066747539 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbc54066730 0x7fbc54068bf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.703+0000 7fbc67fff700 1 --2- 172.21.15.81:0/2066747539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc6815a6a0 0x7fbc680ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.703+0000 7fbc67fff700 1 --2- 172.21.15.81:0/2066747539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc680ae9f0 0x7fbc6814f050 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.703+0000 7fbc67fff700 1 --2- 172.21.15.81:0/2066747539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc6806ca50 0x7fbc681464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.703+0000 7fbc67fff700 1 -- 172.21.15.81:0/2066747539 >> 172.21.15.81:0/2066747539 conn(0x7fbc68070790 msgr2=0x7fbc681494f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.703+0000 7fbc67fff700 1 -- 172.21.15.81:0/2066747539 shutdown_connections 2024-05-08T23:32:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:04.703+0000 7fbc67fff700 1 -- 172.21.15.81:0/2066747539 wait complete. 2024-05-08T23:32:04.706 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:32:04.717 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2024-05-08T23:32:04.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:04.717 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:04.730 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[36914]: fuse finished with error 0 and tester_r 0 2024-05-08T23:32:04.731 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:32:06.121 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:32:06.123 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:32:10.833 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:32:10.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:10.833 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:32:10.842 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rwp' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2024-05-08T23:32:11.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbdfa4b700 1 Processor -- start 2024-05-08T23:32:11.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbdfa4b700 1 -- start start 2024-05-08T23:32:11.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbdfa4b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcbd80b3ca0 0x7fcbd806c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbdfa4b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcbd80b4790 0x7fcbd80cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbdfa4b700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd80cbfb0 con 0x7fcbd806b710 2024-05-08T23:32:11.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbdfa4b700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd806ad30 con 0x7fcbd80b4790 2024-05-08T23:32:11.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbdfa4b700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd80b4c00 con 0x7fcbd80b3ca0 2024-05-08T23:32:11.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbdcfe6700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcbd806b710 0x7fcbd80b4080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45744/0 (socket says 172.21.15.81:45744) 2024-05-08T23:32:11.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbdcfe6700 1 -- 172.21.15.81:0/332380446 learned_addr learned my addr 172.21.15.81:0/332380446 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:11.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbddfe8700 1 --2- 172.21.15.81:0/332380446 >> v2:172.21.15.81:3300/0 conn(0x7fcbd80b4790 0x7fcbd80cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbdd7e7700 1 --2- 172.21.15.81:0/332380446 >> v2:172.21.15.117:3300/0 conn(0x7fcbd80b3ca0 0x7fcbd806c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbd7fff700 1 -- 172.21.15.81:0/332380446 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707991262 0 0) 0x7fcbd80cbfb0 con 0x7fcbd806b710 2024-05-08T23:32:11.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbd7fff700 1 -- 172.21.15.81:0/332380446 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbc40035d0 con 0x7fcbd806b710 2024-05-08T23:32:11.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbddfe8700 1 -- 172.21.15.81:0/332380446 >> v2:172.21.15.117:3300/0 conn(0x7fcbd80b3ca0 msgr2=0x7fcbd806c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:11.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbddfe8700 1 --2- 172.21.15.81:0/332380446 >> v2:172.21.15.117:3300/0 conn(0x7fcbd80b3ca0 0x7fcbd806c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbd7fff700 1 -- 172.21.15.81:0/332380446 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 767311865 0 0) 0x7fcbc40035d0 con 0x7fcbd806b710 2024-05-08T23:32:11.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbddfe8700 1 -- 172.21.15.81:0/332380446 >> v1:172.21.15.81:6789/0 conn(0x7fcbd806b710 legacy=0x7fcbd80b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbddfe8700 1 -- 172.21.15.81:0/332380446 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd80cb9f0 con 0x7fcbd80b4790 2024-05-08T23:32:11.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbddfe8700 1 --2- 172.21.15.81:0/332380446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbd80b4790 0x7fcbd80cc4f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80d0250 tx=0x7fcbd0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd43a10a50f7bbdd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbd7fff700 1 -- 172.21.15.81:0/332380446 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0001670 con 0x7fcbd80b4790 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbd7fff700 1 -- 172.21.15.81:0/332380446 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd0004030 con 0x7fcbd80b4790 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.015+0000 7fcbd7fff700 1 -- 172.21.15.81:0/332380446 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd00043a0 con 0x7fcbd80b4790 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 -- 172.21.15.81:0/332380446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbd80b4790 msgr2=0x7fcbd80cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 --2- 172.21.15.81:0/332380446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbd80b4790 0x7fcbd80cc4f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80d0250 tx=0x7fcbd0009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 -- 172.21.15.81:0/332380446 shutdown_connections 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 --2- 172.21.15.81:0/332380446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbd80b4790 0x7fcbd80cc4f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 --2- 172.21.15.81:0/332380446 >> v2:172.21.15.117:3300/0 conn(0x7fcbd80b3ca0 0x7fcbd806c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 -- 172.21.15.81:0/332380446 >> 172.21.15.81:0/332380446 conn(0x7fcbd80cf650 msgr2=0x7fcbd80cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:11.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 -- 172.21.15.81:0/332380446 shutdown_connections 2024-05-08T23:32:11.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 -- 172.21.15.81:0/332380446 wait complete. 2024-05-08T23:32:11.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 Processor -- start 2024-05-08T23:32:11.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 -- start start 2024-05-08T23:32:11.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbd806b710 0x7fcbd81553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbd80b3ca0 0x7fcbd8159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbd80b4790 0x7fcbd8159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80cc690 con 0x7fcbd80b4790 2024-05-08T23:32:11.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd8155dd0 con 0x7fcbd80b3ca0 2024-05-08T23:32:11.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.019+0000 7fcbdfa4b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd81560b0 con 0x7fcbd806b710 2024-05-08T23:32:11.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbddfe8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbd80b4790 0x7fcbd8159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbddfe8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbd80b4790 0x7fcbd8159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39350/0 (socket says 172.21.15.81:39350) 2024-05-08T23:32:11.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbddfe8700 1 -- 172.21.15.81:0/2353359007 learned_addr learned my addr 172.21.15.81:0/2353359007 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:11.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbdd7e7700 1 --2- 172.21.15.81:0/2353359007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbd806b710 0x7fcbd81553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbdcfe6700 1 --2- 172.21.15.81:0/2353359007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbd80b3ca0 0x7fcbd8159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbddfe8700 1 -- 172.21.15.81:0/2353359007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbd806b710 msgr2=0x7fcbd81553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbddfe8700 1 --2- 172.21.15.81:0/2353359007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbd806b710 0x7fcbd81553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbddfe8700 1 -- 172.21.15.81:0/2353359007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbd80b3ca0 msgr2=0x7fcbd8159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbddfe8700 1 --2- 172.21.15.81:0/2353359007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbd80b3ca0 0x7fcbd8159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbddfe8700 1 -- 172.21.15.81:0/2353359007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd00094b0 con 0x7fcbd80b4790 2024-05-08T23:32:11.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbddfe8700 1 --2- 172.21.15.81:0/2353359007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbd80b4790 0x7fcbd8159f90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00097d0 tx=0x7fcbd0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:11.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbd5ffb700 1 -- 172.21.15.81:0/2353359007 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0003d30 con 0x7fcbd80b4790 2024-05-08T23:32:11.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbd5ffb700 1 -- 172.21.15.81:0/2353359007 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd001f070 con 0x7fcbd80b4790 2024-05-08T23:32:11.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbdfa4b700 1 -- 172.21.15.81:0/2353359007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd8156390 con 0x7fcbd80b4790 2024-05-08T23:32:11.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbd5ffb700 1 -- 172.21.15.81:0/2353359007 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0011870 con 0x7fcbd80b4790 2024-05-08T23:32:11.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbdfa4b700 1 -- 172.21.15.81:0/2353359007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd8156670 con 0x7fcbd80b4790 2024-05-08T23:32:11.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.023+0000 7fcbdfa4b700 1 -- 172.21.15.81:0/2353359007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd80ccb00 con 0x7fcbd80b4790 2024-05-08T23:32:11.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.027+0000 7fcbd5ffb700 1 -- 172.21.15.81:0/2353359007 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcbd0021030 con 0x7fcbd80b4790 2024-05-08T23:32:11.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.027+0000 7fcbd5ffb700 1 --2- 172.21.15.81:0/2353359007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcbb8066730 0x7fcbb8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.027+0000 7fcbdd7e7700 1 --2- 172.21.15.81:0/2353359007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcbb8066730 0x7fcbb8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.027+0000 7fcbd5ffb700 1 -- 172.21.15.81:0/2353359007 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fcbd00855e0 con 0x7fcbd80b4790 2024-05-08T23:32:11.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.027+0000 7fcbdd7e7700 1 --2- 172.21.15.81:0/2353359007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcbb8066730 0x7fcbb8068bf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80113e0 tx=0x7fcbc800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:11.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.031+0000 7fcbd5ffb700 1 -- 172.21.15.81:0/2353359007 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcbd80ccb00 con 0x7fcbd80b4790 2024-05-08T23:32:11.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.251+0000 7fcbdfa4b700 1 -- 172.21.15.81:0/2353359007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7fcbd80ccb00 con 0x7fcbd80b4790 2024-05-08T23:32:11.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.259+0000 7fcbd5ffb700 1 -- 172.21.15.81:0/2353359007 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v12) v1 ==== 211+0+0 (secure 0 0 0) 0x7fcbd80ccb00 con 0x7fcbd80b4790 2024-05-08T23:32:11.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.263+0000 7fcbdfa4b700 1 -- 172.21.15.81:0/2353359007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcbb8066730 msgr2=0x7fcbb8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.263+0000 7fcbdfa4b700 1 --2- 172.21.15.81:0/2353359007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcbb8066730 0x7fcbb8068bf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80113e0 tx=0x7fcbc800b040 comp rx=0 tx=0).stop 2024-05-08T23:32:11.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.263+0000 7fcbdfa4b700 1 -- 172.21.15.81:0/2353359007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbd80b4790 msgr2=0x7fcbd8159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.263+0000 7fcbdfa4b700 1 --2- 172.21.15.81:0/2353359007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbd80b4790 0x7fcbd8159f90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00097d0 tx=0x7fcbd0004770 comp rx=0 tx=0).stop 2024-05-08T23:32:11.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.263+0000 7fcbdfa4b700 1 -- 172.21.15.81:0/2353359007 shutdown_connections 2024-05-08T23:32:11.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.263+0000 7fcbdfa4b700 1 --2- 172.21.15.81:0/2353359007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcbb8066730 0x7fcbb8068bf0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.263+0000 7fcbdfa4b700 1 --2- 172.21.15.81:0/2353359007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbd80b4790 0x7fcbd8159f90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.263+0000 7fcbdfa4b700 1 --2- 172.21.15.81:0/2353359007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbd80b3ca0 0x7fcbd8159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.263+0000 7fcbdfa4b700 1 --2- 172.21.15.81:0/2353359007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbd806b710 0x7fcbd81553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.263+0000 7fcbdfa4b700 1 -- 172.21.15.81:0/2353359007 >> 172.21.15.81:0/2353359007 conn(0x7fcbd80cf650 msgr2=0x7fcbd806d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:11.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.263+0000 7fcbdfa4b700 1 -- 172.21.15.81:0/2353359007 shutdown_connections 2024-05-08T23:32:11.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.263+0000 7fcbdfa4b700 1 -- 172.21.15.81:0/2353359007 wait complete. 2024-05-08T23:32:11.268 INFO:teuthology.orchestra.run.smithi081.stderr:updated caps for client.0 2024-05-08T23:32:11.281 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:32:11.392 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cbaad7700 1 Processor -- start 2024-05-08T23:32:11.392 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cbaad7700 1 -- start start 2024-05-08T23:32:11.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cbaad7700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7cb40b3c20 0x7f7cb41624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cbaad7700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb4158960 con 0x7f7cb4071e70 2024-05-08T23:32:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cbaad7700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb40ae990 con 0x7f7cb4070110 2024-05-08T23:32:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cbaad7700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40b1f60 con 0x7f7cb40b3c20 2024-05-08T23:32:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cb9ad5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7cb40b3c20 0x7f7cb41624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cb92d4700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f7cb4071e70 0x7f7cb40b4000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45758/0 (socket says 172.21.15.81:45758) 2024-05-08T23:32:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cb92d4700 1 -- 172.21.15.81:0/3493857783 learned_addr learned my addr 172.21.15.81:0/3493857783 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cb9ad5700 1 -- 172.21.15.81:0/3493857783 >> v1:172.21.15.117:6789/0 conn(0x7f7cb4070110 legacy=0x7f7cb40705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cb9ad5700 1 -- 172.21.15.81:0/3493857783 >> v1:172.21.15.81:6789/0 conn(0x7f7cb4071e70 legacy=0x7f7cb40b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cb9ad5700 1 -- 172.21.15.81:0/3493857783 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40b2170 con 0x7f7cb40b3c20 2024-05-08T23:32:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cb8ad3700 1 -- 172.21.15.81:0/3493857783 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139385286 0 0) 0x7f7cb4158960 con 0x7f7cb4071e70 2024-05-08T23:32:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cb9ad5700 1 --2- 172.21.15.81:0/3493857783 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb40b3c20 0x7f7cb41624b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009d90 tx=0x7f7ca800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e887752e88226b98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cb8ad3700 1 -- 172.21.15.81:0/3493857783 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca800e070 con 0x7f7cb40b3c20 2024-05-08T23:32:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cb8ad3700 1 -- 172.21.15.81:0/3493857783 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca800bb90 con 0x7f7cb40b3c20 2024-05-08T23:32:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cb8ad3700 1 -- 172.21.15.81:0/3493857783 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8003a40 con 0x7f7cb40b3c20 2024-05-08T23:32:11.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cbaad7700 1 -- 172.21.15.81:0/3493857783 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb40b3c20 msgr2=0x7f7cb41624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cbaad7700 1 --2- 172.21.15.81:0/3493857783 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb40b3c20 0x7f7cb41624b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009d90 tx=0x7f7ca800b330 comp rx=0 tx=0).stop 2024-05-08T23:32:11.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cbaad7700 1 -- 172.21.15.81:0/3493857783 shutdown_connections 2024-05-08T23:32:11.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cbaad7700 1 --2- 172.21.15.81:0/3493857783 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb40b3c20 0x7f7cb41624b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cbaad7700 1 -- 172.21.15.81:0/3493857783 >> 172.21.15.81:0/3493857783 conn(0x7f7cb406c330 msgr2=0x7f7cb406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:11.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cbaad7700 1 -- 172.21.15.81:0/3493857783 shutdown_connections 2024-05-08T23:32:11.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.391+0000 7f7cbaad7700 1 -- 172.21.15.81:0/3493857783 wait complete. 2024-05-08T23:32:11.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cbaad7700 1 Processor -- start 2024-05-08T23:32:11.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cbaad7700 1 -- start start 2024-05-08T23:32:11.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cbaad7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb4070110 0x7f7cb41fa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cbaad7700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cb4071e70 0x7f7cb41fa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cbaad7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb40b3c20 0x7f7cb41fb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cbaad7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4163c60 con 0x7f7cb4070110 2024-05-08T23:32:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cbaad7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4163f00 con 0x7f7cb40b3c20 2024-05-08T23:32:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cbaad7700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb41641e0 con 0x7f7cb4071e70 2024-05-08T23:32:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cb9ad5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb4070110 0x7f7cb41fa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cb9ad5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb4070110 0x7f7cb41fa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39362/0 (socket says 172.21.15.81:39362) 2024-05-08T23:32:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cb9ad5700 1 -- 172.21.15.81:0/1728548669 learned_addr learned my addr 172.21.15.81:0/1728548669 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cba2d6700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb40b3c20 0x7f7cb41fb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cb92d4700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cb4071e70 0x7f7cb41fa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cb9ad5700 1 -- 172.21.15.81:0/1728548669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cb4071e70 msgr2=0x7f7cb41fa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cb9ad5700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cb4071e70 0x7f7cb41fa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cb9ad5700 1 -- 172.21.15.81:0/1728548669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb40b3c20 msgr2=0x7f7cb41fb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cb9ad5700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb40b3c20 0x7f7cb41fb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cb9ad5700 1 -- 172.21.15.81:0/1728548669 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca8009a70 con 0x7f7cb4070110 2024-05-08T23:32:11.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cba2d6700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb40b3c20 0x7f7cb41fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:11.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cb92d4700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cb4071e70 0x7f7cb41fa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:11.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cb9ad5700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb4070110 0x7f7cb41fa1b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7cb406b120 tx=0x7f7ca8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:11.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cb27fc700 1 -- 172.21.15.81:0/1728548669 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8003a40 con 0x7f7cb4070110 2024-05-08T23:32:11.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cb27fc700 1 -- 172.21.15.81:0/1728548669 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca8012630 con 0x7f7cb4070110 2024-05-08T23:32:11.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cbaad7700 1 -- 172.21.15.81:0/1728548669 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb41644c0 con 0x7f7cb4070110 2024-05-08T23:32:11.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cb27fc700 1 -- 172.21.15.81:0/1728548669 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80129a0 con 0x7f7cb4070110 2024-05-08T23:32:11.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.395+0000 7f7cbaad7700 1 -- 172.21.15.81:0/1728548669 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb4143650 con 0x7f7cb4070110 2024-05-08T23:32:11.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.399+0000 7f7cbaad7700 1 -- 172.21.15.81:0/1728548669 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb41646d0 con 0x7f7cb4070110 2024-05-08T23:32:11.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.399+0000 7f7cb27fc700 1 -- 172.21.15.81:0/1728548669 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7ca802f470 con 0x7f7cb4070110 2024-05-08T23:32:11.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.399+0000 7f7cb27fc700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7ca0066730 0x7f7ca0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.399+0000 7f7cb27fc700 1 -- 172.21.15.81:0/1728548669 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f7ca80870b0 con 0x7f7cb4070110 2024-05-08T23:32:11.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.399+0000 7f7cb92d4700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7ca0066730 0x7f7ca0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.399+0000 7f7cb92d4700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7ca0066730 0x7f7ca0068bf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7cac007fd0 tx=0x7f7cac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:11.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.407+0000 7f7cb27fc700 1 -- 172.21.15.81:0/1728548669 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7ca8057d90 con 0x7f7cb4070110 2024-05-08T23:32:11.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.611+0000 7f7cbaad7700 1 -- 172.21.15.81:0/1728548669 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7cb40cd130 con 0x7f7cb4070110 2024-05-08T23:32:11.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.615+0000 7f7cb27fc700 1 -- 172.21.15.81:0/1728548669 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+82 (secure 0 0 0) 0x7f7ca8057920 con 0x7f7cb4070110 2024-05-08T23:32:11.616 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2024-05-08T23:32:11.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.615+0000 7f7cbaad7700 1 -- 172.21.15.81:0/1728548669 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7ca0066730 msgr2=0x7f7ca0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.615+0000 7f7cbaad7700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7ca0066730 0x7f7ca0068bf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7cac007fd0 tx=0x7f7cac009040 comp rx=0 tx=0).stop 2024-05-08T23:32:11.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.615+0000 7f7cbaad7700 1 -- 172.21.15.81:0/1728548669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb4070110 msgr2=0x7f7cb41fa1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.615+0000 7f7cbaad7700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb4070110 0x7f7cb41fa1b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7cb406b120 tx=0x7f7ca8004af0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.615+0000 7f7cbaad7700 1 -- 172.21.15.81:0/1728548669 shutdown_connections 2024-05-08T23:32:11.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.615+0000 7f7cbaad7700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7ca0066730 0x7f7ca0068bf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.615+0000 7f7cbaad7700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb40b3c20 0x7f7cb41fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.615+0000 7f7cbaad7700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cb4071e70 0x7f7cb41fa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.615+0000 7f7cbaad7700 1 --2- 172.21.15.81:0/1728548669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7cb4070110 0x7f7cb41fa1b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.615+0000 7f7cbaad7700 1 -- 172.21.15.81:0/1728548669 >> 172.21.15.81:0/1728548669 conn(0x7f7cb406c330 msgr2=0x7f7cb40cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:11.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.615+0000 7f7cbaad7700 1 -- 172.21.15.81:0/1728548669 shutdown_connections 2024-05-08T23:32:11.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:11.615+0000 7f7cbaad7700 1 -- 172.21.15.81:0/1728548669 wait complete. 2024-05-08T23:32:11.632 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:32:11.632 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:32:11.632 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2024-05-08T23:32:11.632 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:32:11.632 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:11.632 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:32:11.632 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:32:11.632 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:32:11.632 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:32:11.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:11.632 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2024-05-08T23:32:11.640 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:32:11.640 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:32:11.641 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:32:11.641 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:11.641 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-05-08T23:32:11.641 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:11.641 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:32:11.641 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:11.641 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:32:11.641 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:11.641 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:32:11.641 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:11.641 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:32:11.641 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:11.641 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T23:32:11.641 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42484sec preferred_lft 42484sec 2024-05-08T23:32:11.642 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2024-05-08T23:32:11.642 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:11.642 INFO:teuthology.orchestra.run.smithi081.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T23:32:11.642 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 02:42:22:62:7a:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:11.642 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T23:32:11.642 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:11.642 INFO:teuthology.orchestra.run.smithi081.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:32:11.642 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:2b:10:6b:08:e3 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:11.642 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:32:11.642 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:11.642 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::48fc:c7ff:fe41:6d94/64 scope link 2024-05-08T23:32:11.642 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:11.642 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:32:11.642 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:2b:10:6b:08:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:32:11.642 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::702b:10ff:fe6b:8e3/64 scope link 2024-05-08T23:32:11.643 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:11.643 INFO:teuthology.orchestra.run.smithi081.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:32:11.643 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:9d:1b:4e:6b:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:32:11.643 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::709d:1bff:fe4e:6bd5/64 scope link 2024-05-08T23:32:11.643 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:11.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:11.643 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:32:11.656 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-05-08T23:32:11.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:11.658 DEBUG:teuthology.orchestra.run.smithi081:> route 2024-05-08T23:32:11.708 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2024-05-08T23:32:11.708 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:32:11.708 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T23:32:11.708 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T23:32:11.708 INFO:teuthology.orchestra.run.smithi081.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T23:32:11.708 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T23:32:11.708 INFO:teuthology.orchestra.run.smithi081.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T23:32:11.708 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:32:11.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:11.709 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-05-08T23:32:11.709 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-05-08T23:32:11.709 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T23:32:11.709 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T23:32:11.709 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T23:32:11.709 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-05-08T23:32:11.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:11.748 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-05-08T23:32:11.755 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T23:32:11.755 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:32:11.755 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:32:11.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:11.756 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2024-05-08T23:32:11.771 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T23:32:11.772 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:32:11.772 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:11.822 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:32:11.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:32:11.823 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:11.829 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:32:11.843 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:32:11.843 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:32:11.854 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:32:11.855 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:32:11.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:11.856 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:32:11.906 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T23:32:11.906 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T23:32:11.908 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:32:11.926 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:32:11.926 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:32:11.936 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:32:11.937 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:32:11.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:11.937 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:32:11.999 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2024-05-08T23:32:11.995+0000 7ff73a21c640 -1 init, newargv = 0x56255b56ea80 newargc=15 2024-05-08T23:32:11.999 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[37387]: starting ceph client 2024-05-08T23:32:12.007 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[37387]: starting fuse 2024-05-08T23:32:13.091 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:32:13.110 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:32:13.110 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:32:13.124 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:32:13.125 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:32:13.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:13.126 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:32:13.132 INFO:teuthology.orchestra.run.smithi081.stdout:52 2024-05-08T23:32:13.132 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T23:32:13.132 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:32:13.132 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2024-05-08T23:32:13.133 DEBUG:teuthology.orchestra.run.smithi081:> import re 2024-05-08T23:32:13.133 DEBUG:teuthology.orchestra.run.smithi081:> import os 2024-05-08T23:32:13.133 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2024-05-08T23:32:13.133 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:32:13.133 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2024-05-08T23:32:13.133 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T23:32:13.133 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2024-05-08T23:32:13.133 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T23:32:13.133 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:32:13.133 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2024-05-08T23:32:13.133 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2024-05-08T23:32:13.133 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2024-05-08T23:32:13.133 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2024-05-08T23:32:13.133 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:32:13.133 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2024-05-08T23:32:13.134 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T23:32:13.134 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T23:32:13.134 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T23:32:13.134 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2024-05-08T23:32:13.134 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2024-05-08T23:32:13.134 DEBUG:teuthology.orchestra.run.smithi081:> return f 2024-05-08T23:32:13.134 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T23:32:13.135 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:32:13.135 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2024-05-08T23:32:13.135 DEBUG:teuthology.orchestra.run.smithi081:> ' 2024-05-08T23:32:13.262 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.37387.asok 2024-05-08T23:32:13.448 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37387.asok 2024-05-08T23:32:13.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:13.448 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37387.asok status 2024-05-08T23:32:13.548 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:32:13.548 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2024-05-08T23:32:13.548 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T23:32:13.549 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:32:13.549 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2024-05-08T23:32:13.549 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2024-05-08T23:32:13.549 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T23:32:13.549 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "37387", 2024-05-08T23:32:13.549 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2024-05-08T23:32:13.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:13.549 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2024-05-08T23:32:13.549 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2024-05-08T23:32:13.549 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 5646, 2024-05-08T23:32:13.549 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2024-05-08T23:32:13.549 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2024-05-08T23:32:13.549 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2024-05-08T23:32:13.550 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 5646 2024-05-08T23:32:13.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:13.550 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:32:13.550 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:32:13.550 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:32:13.550 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 830438443 2024-05-08T23:32:13.550 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:13.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:13.550 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:32:13.550 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:32:13.550 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:32:13.551 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 830438443 2024-05-08T23:32:13.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:13.551 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.5646 192.168.0.1:0/830438443", 2024-05-08T23:32:13.551 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/830438443", 2024-05-08T23:32:13.551 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2024-05-08T23:32:13.551 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 15, 2024-05-08T23:32:13.551 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 39, 2024-05-08T23:32:13.551 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2024-05-08T23:32:13.551 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2024-05-08T23:32:13.551 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2024-05-08T23:32:13.551 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:32:13.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:13.559 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2024-05-08T23:32:13.559 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:13.565 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2024-05-08T23:32:13.565 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:13.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:13.565 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:13.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:13.577 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2024-05-08T23:32:13.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:13.589 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2024-05-08T23:32:13.605 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:32:13.605 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:13.612 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:13.780 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b382a700 1 Processor -- start 2024-05-08T23:32:13.780 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b382a700 1 -- start start 2024-05-08T23:32:13.781 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b382a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f59ac06ca60 0x7f59ac15a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.781 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b382a700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ac0b3380 con 0x7f59ac13d180 2024-05-08T23:32:13.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b382a700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ac0b3660 con 0x7f59ac13cda0 2024-05-08T23:32:13.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b382a700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ac15b6a0 con 0x7f59ac06ca60 2024-05-08T23:32:13.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b0dc5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f59ac06ca60 0x7f59ac15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b0dc5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f59ac06ca60 0x7f59ac15a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39374/0 (socket says 172.21.15.81:39374) 2024-05-08T23:32:13.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b0dc5700 1 -- 172.21.15.81:0/3732797171 learned_addr learned my addr 172.21.15.81:0/3732797171 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:13.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b1dc7700 1 --1- 172.21.15.81:0/3732797171 >> v1:172.21.15.81:6789/0 conn(0x7f59ac13d180 0x7f59ac15ae90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45770/0 (socket says 172.21.15.81:45770) 2024-05-08T23:32:13.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b0dc5700 1 -- 172.21.15.81:0/3732797171 >> v1:172.21.15.117:6789/0 conn(0x7f59ac13cda0 legacy=0x7f59ac06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:13.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b0dc5700 1 -- 172.21.15.81:0/3732797171 >> v1:172.21.15.81:6789/0 conn(0x7f59ac13d180 legacy=0x7f59ac15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b0dc5700 1 -- 172.21.15.81:0/3732797171 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac0b3940 con 0x7f59ac06ca60 2024-05-08T23:32:13.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b0dc5700 1 --2- 172.21.15.81:0/3732797171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ac06ca60 0x7f59ac15a950 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f599c00bbb0 tx=0x7f599c00bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc91747d90c4aa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:13.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59a3fff700 1 -- 172.21.15.81:0/3732797171 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c00dd50 con 0x7f59ac06ca60 2024-05-08T23:32:13.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59a3fff700 1 -- 172.21.15.81:0/3732797171 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599c004030 con 0x7f59ac06ca60 2024-05-08T23:32:13.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59a3fff700 1 -- 172.21.15.81:0/3732797171 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c0043e0 con 0x7f59ac06ca60 2024-05-08T23:32:13.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b382a700 1 -- 172.21.15.81:0/3732797171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ac06ca60 msgr2=0x7f59ac15a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b382a700 1 --2- 172.21.15.81:0/3732797171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ac06ca60 0x7f59ac15a950 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f599c00bbb0 tx=0x7f599c00bec0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b382a700 1 -- 172.21.15.81:0/3732797171 shutdown_connections 2024-05-08T23:32:13.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b382a700 1 --2- 172.21.15.81:0/3732797171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ac06ca60 0x7f59ac15a950 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b382a700 1 -- 172.21.15.81:0/3732797171 >> 172.21.15.81:0/3732797171 conn(0x7f59ac06a360 msgr2=0x7f59ac06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:13.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b382a700 1 -- 172.21.15.81:0/3732797171 shutdown_connections 2024-05-08T23:32:13.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.779+0000 7f59b382a700 1 -- 172.21.15.81:0/3732797171 wait complete. 2024-05-08T23:32:13.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.783+0000 7f59b382a700 1 Processor -- start 2024-05-08T23:32:13.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.783+0000 7f59b382a700 1 -- start start 2024-05-08T23:32:13.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.783+0000 7f59b382a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59ac06ca60 0x7f59ac14f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.783+0000 7f59b382a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ac13cda0 0x7f59ac14fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.783+0000 7f59b382a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59ac13d180 0x7f59ac0aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.783+0000 7f59b382a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac1f37c0 con 0x7f59ac13cda0 2024-05-08T23:32:13.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.783+0000 7f59b382a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac0b1e40 con 0x7f59ac06ca60 2024-05-08T23:32:13.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.783+0000 7f59b382a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac1f2bd0 con 0x7f59ac13d180 2024-05-08T23:32:13.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.783+0000 7f59b0dc5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ac13cda0 0x7f59ac14fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.783+0000 7f59b0dc5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ac13cda0 0x7f59ac14fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39382/0 (socket says 172.21.15.81:39382) 2024-05-08T23:32:13.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.783+0000 7f59b0dc5700 1 -- 172.21.15.81:0/1654646886 learned_addr learned my addr 172.21.15.81:0/1654646886 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:13.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.783+0000 7f59b15c6700 1 --2- 172.21.15.81:0/1654646886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59ac06ca60 0x7f59ac14f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.783+0000 7f59b1dc7700 1 --2- 172.21.15.81:0/1654646886 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59ac13d180 0x7f59ac0aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.787+0000 7f59b0dc5700 1 -- 172.21.15.81:0/1654646886 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59ac13d180 msgr2=0x7f59ac0aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.787+0000 7f59b0dc5700 1 --2- 172.21.15.81:0/1654646886 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59ac13d180 0x7f59ac0aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.787+0000 7f59b0dc5700 1 -- 172.21.15.81:0/1654646886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59ac06ca60 msgr2=0x7f59ac14f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.787+0000 7f59b0dc5700 1 --2- 172.21.15.81:0/1654646886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59ac06ca60 0x7f59ac14f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.787+0000 7f59b0dc5700 1 -- 172.21.15.81:0/1654646886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c00b810 con 0x7f59ac13cda0 2024-05-08T23:32:13.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.787+0000 7f59b0dc5700 1 --2- 172.21.15.81:0/1654646886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ac13cda0 0x7f59ac14fd40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f59ac074490 tx=0x7f599c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:13.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.787+0000 7f59a1ffb700 1 -- 172.21.15.81:0/1654646886 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c004590 con 0x7f59ac13cda0 2024-05-08T23:32:13.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.787+0000 7f59a1ffb700 1 -- 172.21.15.81:0/1654646886 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599c010470 con 0x7f59ac13cda0 2024-05-08T23:32:13.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.787+0000 7f59b382a700 1 -- 172.21.15.81:0/1654646886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ac1f2e90 con 0x7f59ac13cda0 2024-05-08T23:32:13.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.787+0000 7f59a1ffb700 1 -- 172.21.15.81:0/1654646886 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c010820 con 0x7f59ac13cda0 2024-05-08T23:32:13.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.787+0000 7f59b382a700 1 -- 172.21.15.81:0/1654646886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ac1f3450 con 0x7f59ac13cda0 2024-05-08T23:32:13.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.787+0000 7f59b382a700 1 -- 172.21.15.81:0/1654646886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ac1f35e0 con 0x7f59ac13cda0 2024-05-08T23:32:13.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.787+0000 7f59a1ffb700 1 -- 172.21.15.81:0/1654646886 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f599c0193f0 con 0x7f59ac13cda0 2024-05-08T23:32:13.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.791+0000 7f59a1ffb700 1 --2- 172.21.15.81:0/1654646886 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5990066730 0x7f5990068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.791+0000 7f59a1ffb700 1 -- 172.21.15.81:0/1654646886 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f599c0862d0 con 0x7f59ac13cda0 2024-05-08T23:32:13.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.791+0000 7f59b15c6700 1 --2- 172.21.15.81:0/1654646886 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5990066730 0x7f5990068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.791+0000 7f59b15c6700 1 --2- 172.21.15.81:0/1654646886 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5990066730 0x7f5990068bf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f59a8005150 tx=0x7f59a8007ee0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:13.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.795+0000 7f59a1ffb700 1 -- 172.21.15.81:0/1654646886 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f599c0534a0 con 0x7f59ac13cda0 2024-05-08T23:32:13.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.971+0000 7f59b382a700 1 -- 172.21.15.81:0/1654646886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59ac0684b0 con 0x7f59ac13cda0 2024-05-08T23:32:13.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.975+0000 7f59a1ffb700 1 -- 172.21.15.81:0/1654646886 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+17880 (secure 0 0 0) 0x7f599c056ac0 con 0x7f59ac13cda0 2024-05-08T23:32:13.976 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:13.976 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":39,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:48.310750+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:31:35.620423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:31:36.878524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-05-08T23:31:46.337484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:13.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.975+0000 7f59b382a700 1 -- 172.21.15.81:0/1654646886 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5990066730 msgr2=0x7f5990068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.975+0000 7f59b382a700 1 --2- 172.21.15.81:0/1654646886 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5990066730 0x7f5990068bf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f59a8005150 tx=0x7f59a8007ee0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.975+0000 7f59b382a700 1 -- 172.21.15.81:0/1654646886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ac13cda0 msgr2=0x7f59ac14fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.975+0000 7f59b382a700 1 --2- 172.21.15.81:0/1654646886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ac13cda0 0x7f59ac14fd40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f59ac074490 tx=0x7f599c004770 comp rx=0 tx=0).stop 2024-05-08T23:32:13.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.975+0000 7f59b382a700 1 -- 172.21.15.81:0/1654646886 shutdown_connections 2024-05-08T23:32:13.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.975+0000 7f59b382a700 1 --2- 172.21.15.81:0/1654646886 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5990066730 0x7f5990068bf0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.975+0000 7f59b382a700 1 --2- 172.21.15.81:0/1654646886 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59ac13d180 0x7f59ac0aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.975+0000 7f59b382a700 1 --2- 172.21.15.81:0/1654646886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ac13cda0 0x7f59ac14fd40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.975+0000 7f59b382a700 1 --2- 172.21.15.81:0/1654646886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59ac06ca60 0x7f59ac14f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.975+0000 7f59b382a700 1 -- 172.21.15.81:0/1654646886 >> 172.21.15.81:0/1654646886 conn(0x7f59ac06a360 msgr2=0x7f59ac15c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:13.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.975+0000 7f59b382a700 1 -- 172.21.15.81:0/1654646886 shutdown_connections 2024-05-08T23:32:13.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:13.975+0000 7f59b382a700 1 -- 172.21.15.81:0/1654646886 wait complete. 2024-05-08T23:32:13.992 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:31:35.620423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:31:36.878524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-05-08T23:31:46.337484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:32:13.992 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:14.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e989a0700 1 Processor -- start 2024-05-08T23:32:14.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e989a0700 1 -- start start 2024-05-08T23:32:14.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e989a0700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2e94160920 0x7f2e940ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e989a0700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2e940d1da0 0x7f2e940cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e989a0700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e940d3d40 con 0x7f2e940cec30 2024-05-08T23:32:14.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e989a0700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e940d0d00 con 0x7f2e94160920 2024-05-08T23:32:14.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e989a0700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e940d0fe0 con 0x7f2e940d1da0 2024-05-08T23:32:14.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e92d9d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2e94160920 0x7f2e940ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e92d9d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2e94160920 0x7f2e940ce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39384/0 (socket says 172.21.15.81:39384) 2024-05-08T23:32:14.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e92d9d700 1 -- 172.21.15.81:0/2745528181 learned_addr learned my addr 172.21.15.81:0/2745528181 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:14.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e9259c700 1 --2- 172.21.15.81:0/2745528181 >> v2:172.21.15.117:3300/0 conn(0x7f2e940d1da0 0x7f2e940cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e92d9d700 1 -- 172.21.15.81:0/2745528181 >> v2:172.21.15.117:3300/0 conn(0x7f2e940d1da0 msgr2=0x7f2e940cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:14.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e92d9d700 1 --2- 172.21.15.81:0/2745528181 >> v2:172.21.15.117:3300/0 conn(0x7f2e940d1da0 0x7f2e940cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e92d9d700 1 -- 172.21.15.81:0/2745528181 >> v1:172.21.15.117:6789/0 conn(0x7f2e940cec30 legacy=0x7f2e94161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e92d9d700 1 -- 172.21.15.81:0/2745528181 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e940d12c0 con 0x7f2e94160920 2024-05-08T23:32:14.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e92d9d700 1 --2- 172.21.15.81:0/2745528181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e94160920 0x7f2e940ce6f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2e940b90b0 tx=0x7f2e84009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69a9a9165cbeb0e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e91d9b700 1 -- 172.21.15.81:0/2745528181 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84011830 con 0x7f2e94160920 2024-05-08T23:32:14.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e91d9b700 1 -- 172.21.15.81:0/2745528181 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e84011990 con 0x7f2e94160920 2024-05-08T23:32:14.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.103+0000 7f2e91d9b700 1 -- 172.21.15.81:0/2745528181 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84011d20 con 0x7f2e94160920 2024-05-08T23:32:14.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 -- 172.21.15.81:0/2745528181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e94160920 msgr2=0x7f2e940ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 --2- 172.21.15.81:0/2745528181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e94160920 0x7f2e940ce6f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2e940b90b0 tx=0x7f2e84009bf0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 -- 172.21.15.81:0/2745528181 shutdown_connections 2024-05-08T23:32:14.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 --2- 172.21.15.81:0/2745528181 >> v2:172.21.15.117:3300/0 conn(0x7f2e940d1da0 0x7f2e940cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 --2- 172.21.15.81:0/2745528181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e94160920 0x7f2e940ce6f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 -- 172.21.15.81:0/2745528181 >> 172.21.15.81:0/2745528181 conn(0x7f2e9409c340 msgr2=0x7f2e940d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:14.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 -- 172.21.15.81:0/2745528181 shutdown_connections 2024-05-08T23:32:14.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 -- 172.21.15.81:0/2745528181 wait complete. 2024-05-08T23:32:14.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 Processor -- start 2024-05-08T23:32:14.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 -- start start 2024-05-08T23:32:14.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e940cec30 0x7f2e940b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e940d1da0 0x7f2e940b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e94160920 0x7f2e940b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e940b8700 con 0x7f2e940d1da0 2024-05-08T23:32:14.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e940b2df0 con 0x7f2e940cec30 2024-05-08T23:32:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e989a0700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e940b1d50 con 0x7f2e94160920 2024-05-08T23:32:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e9259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e940d1da0 0x7f2e940b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e9259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e940d1da0 0x7f2e940b81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39396/0 (socket says 172.21.15.81:39396) 2024-05-08T23:32:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e9259c700 1 -- 172.21.15.81:0/1145108159 learned_addr learned my addr 172.21.15.81:0/1145108159 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e92d9d700 1 --2- 172.21.15.81:0/1145108159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e940cec30 0x7f2e940b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e9359e700 1 --2- 172.21.15.81:0/1145108159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e94160920 0x7f2e940b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e9259c700 1 -- 172.21.15.81:0/1145108159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e94160920 msgr2=0x7f2e940b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e9259c700 1 --2- 172.21.15.81:0/1145108159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e94160920 0x7f2e940b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e9259c700 1 -- 172.21.15.81:0/1145108159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e940cec30 msgr2=0x7f2e940b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e9259c700 1 --2- 172.21.15.81:0/1145108159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e940cec30 0x7f2e940b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e9259c700 1 -- 172.21.15.81:0/1145108159 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e84009570 con 0x7f2e940d1da0 2024-05-08T23:32:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.107+0000 7f2e92d9d700 1 --2- 172.21.15.81:0/1145108159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e940cec30 0x7f2e940b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.111+0000 7f2e9259c700 1 --2- 172.21.15.81:0/1145108159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e940d1da0 0x7f2e940b81c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800e840 tx=0x7f2e8800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.111+0000 7f2e7b7fe700 1 -- 172.21.15.81:0/1145108159 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e880095a0 con 0x7f2e940d1da0 2024-05-08T23:32:14.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.111+0000 7f2e7b7fe700 1 -- 172.21.15.81:0/1145108159 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e88013680 con 0x7f2e940d1da0 2024-05-08T23:32:14.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.111+0000 7f2e989a0700 1 -- 172.21.15.81:0/1145108159 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e94160e80 con 0x7f2e940d1da0 2024-05-08T23:32:14.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.111+0000 7f2e7b7fe700 1 -- 172.21.15.81:0/1145108159 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e880139f0 con 0x7f2e940d1da0 2024-05-08T23:32:14.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.111+0000 7f2e989a0700 1 -- 172.21.15.81:0/1145108159 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e940b20e0 con 0x7f2e940d1da0 2024-05-08T23:32:14.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.111+0000 7f2e989a0700 1 -- 172.21.15.81:0/1145108159 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e940684b0 con 0x7f2e940d1da0 2024-05-08T23:32:14.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.111+0000 7f2e7b7fe700 1 -- 172.21.15.81:0/1145108159 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2e88013b50 con 0x7f2e940d1da0 2024-05-08T23:32:14.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.115+0000 7f2e7b7fe700 1 --2- 172.21.15.81:0/1145108159 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e80066760 0x7f2e80068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.115+0000 7f2e92d9d700 1 --2- 172.21.15.81:0/1145108159 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e80066760 0x7f2e80068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.115+0000 7f2e7b7fe700 1 -- 172.21.15.81:0/1145108159 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f2e880840d0 con 0x7f2e940d1da0 2024-05-08T23:32:14.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.115+0000 7f2e92d9d700 1 --2- 172.21.15.81:0/1145108159 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e80066760 0x7f2e80068c20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2e84000c00 tx=0x7f2e84008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.123+0000 7f2e7b7fe700 1 -- 172.21.15.81:0/1145108159 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2e88051340 con 0x7f2e940d1da0 2024-05-08T23:32:14.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.287+0000 7f2e989a0700 1 -- 172.21.15.81:0/1145108159 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e9406c9b0 con 0x7f2e940d1da0 2024-05-08T23:32:14.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.291+0000 7f2e7b7fe700 1 -- 172.21.15.81:0/1145108159 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e88054960 con 0x7f2e940d1da0 2024-05-08T23:32:14.292 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:32:14.294 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.291+0000 7f2e989a0700 1 -- 172.21.15.81:0/1145108159 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e80066760 msgr2=0x7f2e80068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.291+0000 7f2e989a0700 1 --2- 172.21.15.81:0/1145108159 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e80066760 0x7f2e80068c20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2e84000c00 tx=0x7f2e84008030 comp rx=0 tx=0).stop 2024-05-08T23:32:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.291+0000 7f2e989a0700 1 -- 172.21.15.81:0/1145108159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e940d1da0 msgr2=0x7f2e940b81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.291+0000 7f2e989a0700 1 --2- 172.21.15.81:0/1145108159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e940d1da0 0x7f2e940b81c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800e840 tx=0x7f2e8800eb50 comp rx=0 tx=0).stop 2024-05-08T23:32:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.291+0000 7f2e989a0700 1 -- 172.21.15.81:0/1145108159 shutdown_connections 2024-05-08T23:32:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.291+0000 7f2e989a0700 1 --2- 172.21.15.81:0/1145108159 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e80066760 0x7f2e80068c20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.291+0000 7f2e989a0700 1 --2- 172.21.15.81:0/1145108159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e94160920 0x7f2e940b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.291+0000 7f2e989a0700 1 --2- 172.21.15.81:0/1145108159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e940d1da0 0x7f2e940b81c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.291+0000 7f2e989a0700 1 --2- 172.21.15.81:0/1145108159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e940cec30 0x7f2e940b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.291+0000 7f2e989a0700 1 -- 172.21.15.81:0/1145108159 >> 172.21.15.81:0/1145108159 conn(0x7f2e9409c340 msgr2=0x7f2e940b6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:14.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.291+0000 7f2e989a0700 1 -- 172.21.15.81:0/1145108159 shutdown_connections 2024-05-08T23:32:14.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.291+0000 7f2e989a0700 1 -- 172.21.15.81:0/1145108159 wait complete. 2024-05-08T23:32:14.307 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:32:14.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.419+0000 7f0170874700 1 Processor -- start 2024-05-08T23:32:14.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.419+0000 7f0170874700 1 -- start start 2024-05-08T23:32:14.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.419+0000 7f0170874700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f016c15d5f0 0x7f016c0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.419+0000 7f0170874700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016c0b50b0 con 0x7f016c15d210 2024-05-08T23:32:14.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.419+0000 7f0170874700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016c0b5390 con 0x7f016c099a70 2024-05-08T23:32:14.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.419+0000 7f0170874700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016c0b5670 con 0x7f016c15d5f0 2024-05-08T23:32:14.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.419+0000 7f016a59c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f016c15d5f0 0x7f016c0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.419+0000 7f016ad9d700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f016c15d210 0x7f016c06c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45782/0 (socket says 172.21.15.81:45782) 2024-05-08T23:32:14.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.419+0000 7f016ad9d700 1 -- 172.21.15.81:0/2744071033 learned_addr learned my addr 172.21.15.81:0/2744071033 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:14.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.419+0000 7f016a59c700 1 -- 172.21.15.81:0/2744071033 >> v1:172.21.15.117:6789/0 conn(0x7f016c099a70 legacy=0x7f016c0addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.419+0000 7f016a59c700 1 -- 172.21.15.81:0/2744071033 >> v1:172.21.15.81:6789/0 conn(0x7f016c15d210 legacy=0x7f016c06c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.424 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.419+0000 7f016a59c700 1 -- 172.21.15.81:0/2744071033 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c0b5950 con 0x7f016c15d5f0 2024-05-08T23:32:14.424 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.419+0000 7f0169d9b700 1 -- 172.21.15.81:0/2744071033 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835233877 0 0) 0x7f016c0b50b0 con 0x7f016c15d210 2024-05-08T23:32:14.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.423+0000 7f016a59c700 1 --2- 172.21.15.81:0/2744071033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f016c15d5f0 0x7f016c0b49d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f015c009850 tx=0x7f015c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef7216fa98fd20b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.423+0000 7f0169d9b700 1 -- 172.21.15.81:0/2744071033 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c011830 con 0x7f016c15d5f0 2024-05-08T23:32:14.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.423+0000 7f0169d9b700 1 -- 172.21.15.81:0/2744071033 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015c011990 con 0x7f016c15d5f0 2024-05-08T23:32:14.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.423+0000 7f0169d9b700 1 -- 172.21.15.81:0/2744071033 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c011d20 con 0x7f016c15d5f0 2024-05-08T23:32:14.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.423+0000 7f0170874700 1 -- 172.21.15.81:0/2744071033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f016c15d5f0 msgr2=0x7f016c0b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.423+0000 7f0170874700 1 --2- 172.21.15.81:0/2744071033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f016c15d5f0 0x7f016c0b49d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f015c009850 tx=0x7f015c009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:14.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.423+0000 7f0170874700 1 -- 172.21.15.81:0/2744071033 shutdown_connections 2024-05-08T23:32:14.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.423+0000 7f0170874700 1 --2- 172.21.15.81:0/2744071033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f016c15d5f0 0x7f016c0b49d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.423+0000 7f0170874700 1 -- 172.21.15.81:0/2744071033 >> 172.21.15.81:0/2744071033 conn(0x7f016c0b6520 msgr2=0x7f016c0b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:14.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.423+0000 7f0170874700 1 -- 172.21.15.81:0/2744071033 shutdown_connections 2024-05-08T23:32:14.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.423+0000 7f0170874700 1 -- 172.21.15.81:0/2744071033 wait complete. 2024-05-08T23:32:14.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.423+0000 7f0170874700 1 Processor -- start 2024-05-08T23:32:14.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f0170874700 1 -- start start 2024-05-08T23:32:14.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f0170874700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f016c099a70 0x7f016c1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f0170874700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f016c15d210 0x7f016c14f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f0170874700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f016c15d5f0 0x7f016c154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f0170874700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c156b30 con 0x7f016c099a70 2024-05-08T23:32:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f0170874700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c156160 con 0x7f016c15d5f0 2024-05-08T23:32:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f0170874700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c155550 con 0x7f016c15d210 2024-05-08T23:32:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f016ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f016c099a70 0x7f016c1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f016ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f016c099a70 0x7f016c1565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39418/0 (socket says 172.21.15.81:39418) 2024-05-08T23:32:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f016ad9d700 1 -- 172.21.15.81:0/3708243424 learned_addr learned my addr 172.21.15.81:0/3708243424 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f016b59e700 1 --2- 172.21.15.81:0/3708243424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f016c15d5f0 0x7f016c154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f016a59c700 1 --2- 172.21.15.81:0/3708243424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f016c15d210 0x7f016c14f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f016ad9d700 1 -- 172.21.15.81:0/3708243424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f016c15d210 msgr2=0x7f016c14f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f016ad9d700 1 --2- 172.21.15.81:0/3708243424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f016c15d210 0x7f016c14f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f016ad9d700 1 -- 172.21.15.81:0/3708243424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f016c15d5f0 msgr2=0x7f016c154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f016ad9d700 1 --2- 172.21.15.81:0/3708243424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f016c15d5f0 0x7f016c154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f016ad9d700 1 -- 172.21.15.81:0/3708243424 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c0094b0 con 0x7f016c099a70 2024-05-08T23:32:14.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f016a59c700 1 --2- 172.21.15.81:0/3708243424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f016c15d210 0x7f016c14f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:14.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f016ad9d700 1 --2- 172.21.15.81:0/3708243424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f016c099a70 0x7f016c1565f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f016000c550 tx=0x7f016000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f01537fe700 1 -- 172.21.15.81:0/3708243424 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160004ca0 con 0x7f016c099a70 2024-05-08T23:32:14.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f01537fe700 1 -- 172.21.15.81:0/3708243424 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0160004e00 con 0x7f016c099a70 2024-05-08T23:32:14.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f01537fe700 1 -- 172.21.15.81:0/3708243424 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160007310 con 0x7f016c099a70 2024-05-08T23:32:14.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f0170874700 1 -- 172.21.15.81:0/3708243424 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016c155830 con 0x7f016c099a70 2024-05-08T23:32:14.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.427+0000 7f0170874700 1 -- 172.21.15.81:0/3708243424 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016c155df0 con 0x7f016c099a70 2024-05-08T23:32:14.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.431+0000 7f0170874700 1 -- 172.21.15.81:0/3708243424 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f016c0684b0 con 0x7f016c099a70 2024-05-08T23:32:14.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.431+0000 7f01537fe700 1 -- 172.21.15.81:0/3708243424 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0160007470 con 0x7f016c099a70 2024-05-08T23:32:14.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.431+0000 7f01537fe700 1 --2- 172.21.15.81:0/3708243424 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0158066730 0x7f0158068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.431+0000 7f01537fe700 1 -- 172.21.15.81:0/3708243424 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f0160083f90 con 0x7f016c099a70 2024-05-08T23:32:14.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.431+0000 7f016a59c700 1 --2- 172.21.15.81:0/3708243424 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0158066730 0x7f0158068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.431+0000 7f016a59c700 1 --2- 172.21.15.81:0/3708243424 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0158066730 0x7f0158068bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f015c0090b0 tx=0x7f015c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.439+0000 7f01537fe700 1 -- 172.21.15.81:0/3708243424 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f01600b5cf0 con 0x7f016c099a70 2024-05-08T23:32:14.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.611+0000 7f0170874700 1 -- 172.21.15.81:0/3708243424 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f016c09a950 con 0x7f016c099a70 2024-05-08T23:32:14.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.611+0000 7f01537fe700 1 -- 172.21.15.81:0/3708243424 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0160051170 con 0x7f016c099a70 2024-05-08T23:32:14.613 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:32:14.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.611+0000 7f0170874700 1 -- 172.21.15.81:0/3708243424 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0158066730 msgr2=0x7f0158068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.611+0000 7f0170874700 1 --2- 172.21.15.81:0/3708243424 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0158066730 0x7f0158068bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f015c0090b0 tx=0x7f015c01c040 comp rx=0 tx=0).stop 2024-05-08T23:32:14.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.611+0000 7f0170874700 1 -- 172.21.15.81:0/3708243424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f016c099a70 msgr2=0x7f016c1565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.611+0000 7f0170874700 1 --2- 172.21.15.81:0/3708243424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f016c099a70 0x7f016c1565f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f016000c550 tx=0x7f016000c860 comp rx=0 tx=0).stop 2024-05-08T23:32:14.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.615+0000 7f0170874700 1 -- 172.21.15.81:0/3708243424 shutdown_connections 2024-05-08T23:32:14.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.615+0000 7f0170874700 1 --2- 172.21.15.81:0/3708243424 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0158066730 0x7f0158068bf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.615+0000 7f0170874700 1 --2- 172.21.15.81:0/3708243424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f016c15d5f0 0x7f016c154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.615+0000 7f0170874700 1 --2- 172.21.15.81:0/3708243424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f016c15d210 0x7f016c14f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.615+0000 7f0170874700 1 --2- 172.21.15.81:0/3708243424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f016c099a70 0x7f016c1565f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.615+0000 7f0170874700 1 -- 172.21.15.81:0/3708243424 >> 172.21.15.81:0/3708243424 conn(0x7f016c0b6520 msgr2=0x7f016c0b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:14.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.615+0000 7f0170874700 1 -- 172.21.15.81:0/3708243424 shutdown_connections 2024-05-08T23:32:14.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.615+0000 7f0170874700 1 -- 172.21.15.81:0/3708243424 wait complete. 2024-05-08T23:32:14.628 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:32:14.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.747+0000 7f2e44888700 1 Processor -- start 2024-05-08T23:32:14.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.747+0000 7f2e44888700 1 -- start start 2024-05-08T23:32:14.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e44888700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e400b32e0 0x7f2e40071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e44888700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e40071cd0 con 0x7f2e40071880 2024-05-08T23:32:14.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e44888700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e40070d20 con 0x7f2e40161470 2024-05-08T23:32:14.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e44888700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4006ca90 con 0x7f2e400b32e0 2024-05-08T23:32:14.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3ffff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2e40071880 0x7f2e400ad370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45794/0 (socket says 172.21.15.81:45794) 2024-05-08T23:32:14.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3ffff700 1 -- 172.21.15.81:0/1819905331 learned_addr learned my addr 172.21.15.81:0/1819905331 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:14.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3effd700 1 --2- 172.21.15.81:0/1819905331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e400b32e0 0x7f2e40071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 -- 172.21.15.81:0/1819905331 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17031130 0 0) 0x7f2e38001490 con 0x7f2e40071880 2024-05-08T23:32:14.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 -- 172.21.15.81:0/1819905331 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e2c0035d0 con 0x7f2e40071880 2024-05-08T23:32:14.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 -- 172.21.15.81:0/1819905331 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1960445966 0 0) 0x7f2e2c0035d0 con 0x7f2e40071880 2024-05-08T23:32:14.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 -- 172.21.15.81:0/1819905331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e400b32e0 msgr2=0x7f2e40071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:14.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 --2- 172.21.15.81:0/1819905331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e400b32e0 0x7f2e40071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 -- 172.21.15.81:0/1819905331 >> v1:172.21.15.117:6789/0 conn(0x7f2e40161470 legacy=0x7f2e4006c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 -- 172.21.15.81:0/1819905331 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e40071000 con 0x7f2e40071880 2024-05-08T23:32:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 -- 172.21.15.81:0/1819905331 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f2e38001b80 con 0x7f2e40071880 2024-05-08T23:32:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 -- 172.21.15.81:0/1819905331 >> v1:172.21.15.81:6789/0 conn(0x7f2e40071880 legacy=0x7f2e400ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 --2- 172.21.15.81:0/1819905331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e2c003a50 0x7f2e2c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 --2- 172.21.15.81:0/1819905331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e400b32e0 0x7f2e2c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 --2- 172.21.15.81:0/1819905331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2c004db0 0x7f2e2c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 -- 172.21.15.81:0/1819905331 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4006ca90 con 0x7f2e2c004db0 2024-05-08T23:32:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 -- 172.21.15.81:0/1819905331 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c00bb10 con 0x7f2e2c003a50 2024-05-08T23:32:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 -- 172.21.15.81:0/1819905331 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c00bcd0 con 0x7f2e400b32e0 2024-05-08T23:32:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3ffff700 1 --2- 172.21.15.81:0/1819905331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2c004db0 0x7f2e2c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3f7fe700 1 --2- 172.21.15.81:0/1819905331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e2c003a50 0x7f2e2c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3effd700 1 --2- 172.21.15.81:0/1819905331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e400b32e0 0x7f2e2c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3ffff700 1 -- 172.21.15.81:0/1819905331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e400b32e0 msgr2=0x7f2e2c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3ffff700 1 --2- 172.21.15.81:0/1819905331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e400b32e0 0x7f2e2c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3ffff700 1 -- 172.21.15.81:0/1819905331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e2c003a50 msgr2=0x7f2e2c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3ffff700 1 --2- 172.21.15.81:0/1819905331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e2c003a50 0x7f2e2c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3ffff700 1 -- 172.21.15.81:0/1819905331 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c0035d0 con 0x7f2e2c004db0 2024-05-08T23:32:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3ffff700 1 --2- 172.21.15.81:0/1819905331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2c004db0 0x7f2e2c005230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800a7c0 tx=0x7f2e2c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25d6f3a5eefda8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 -- 172.21.15.81:0/1819905331 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380072d0 con 0x7f2e2c004db0 2024-05-08T23:32:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 -- 172.21.15.81:0/1819905331 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e38007450 con 0x7f2e2c004db0 2024-05-08T23:32:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3e7fc700 1 -- 172.21.15.81:0/1819905331 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38007800 con 0x7f2e2c004db0 2024-05-08T23:32:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e44888700 1 -- 172.21.15.81:0/1819905331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2c004db0 msgr2=0x7f2e2c005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e44888700 1 --2- 172.21.15.81:0/1819905331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2c004db0 0x7f2e2c005230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800a7c0 tx=0x7f2e2c0058a0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e3ffff700 1 -- 172.21.15.81:0/1819905331 reap_dead start 2024-05-08T23:32:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e44888700 1 -- 172.21.15.81:0/1819905331 shutdown_connections 2024-05-08T23:32:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e44888700 1 -- 172.21.15.81:0/1819905331 >> 172.21.15.81:0/1819905331 conn(0x7f2e400b0de0 msgr2=0x7f2e400b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e44888700 1 -- 172.21.15.81:0/1819905331 shutdown_connections 2024-05-08T23:32:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.751+0000 7f2e44888700 1 -- 172.21.15.81:0/1819905331 wait complete. 2024-05-08T23:32:14.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e44888700 1 Processor -- start 2024-05-08T23:32:14.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e44888700 1 -- start start 2024-05-08T23:32:14.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e44888700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e400cd9c0 0x7f2e400af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e44888700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e400afaf0 0x7f2e400ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e44888700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e400ad820 0x7f2e400cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e44888700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4009a410 con 0x7f2e400ad820 2024-05-08T23:32:14.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e44888700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4009a6f0 con 0x7f2e400afaf0 2024-05-08T23:32:14.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e44888700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4009a9d0 con 0x7f2e400cd9c0 2024-05-08T23:32:14.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e3ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e400ad820 0x7f2e400cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e3ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e400ad820 0x7f2e400cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39436/0 (socket says 172.21.15.81:39436) 2024-05-08T23:32:14.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e3ffff700 1 -- 172.21.15.81:0/3041350274 learned_addr learned my addr 172.21.15.81:0/3041350274 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:14.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e3effd700 1 --2- 172.21.15.81:0/3041350274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e400afaf0 0x7f2e400ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e3f7fe700 1 --2- 172.21.15.81:0/3041350274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e400cd9c0 0x7f2e400af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e3ffff700 1 -- 172.21.15.81:0/3041350274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e400cd9c0 msgr2=0x7f2e400af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e3ffff700 1 --2- 172.21.15.81:0/3041350274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e400cd9c0 0x7f2e400af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e3ffff700 1 -- 172.21.15.81:0/3041350274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e400afaf0 msgr2=0x7f2e400ad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e3ffff700 1 --2- 172.21.15.81:0/3041350274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e400afaf0 0x7f2e400ad2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e3ffff700 1 -- 172.21.15.81:0/3041350274 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4009acb0 con 0x7f2e400ad820 2024-05-08T23:32:14.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e3ffff700 1 --2- 172.21.15.81:0/3041350274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e400ad820 0x7f2e400cc510 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2e38011000 tx=0x7f2e38001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e37fff700 1 -- 172.21.15.81:0/3041350274 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380077c0 con 0x7f2e400ad820 2024-05-08T23:32:14.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e44888700 1 -- 172.21.15.81:0/3041350274 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e400aff70 con 0x7f2e400ad820 2024-05-08T23:32:14.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e37fff700 1 -- 172.21.15.81:0/3041350274 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e380072e0 con 0x7f2e400ad820 2024-05-08T23:32:14.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e44888700 1 -- 172.21.15.81:0/3041350274 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e400ae050 con 0x7f2e400ad820 2024-05-08T23:32:14.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.755+0000 7f2e37fff700 1 -- 172.21.15.81:0/3041350274 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380011c0 con 0x7f2e400ad820 2024-05-08T23:32:14.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.759+0000 7f2e44888700 1 -- 172.21.15.81:0/3041350274 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4009a550 con 0x7f2e400ad820 2024-05-08T23:32:14.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.759+0000 7f2e37fff700 1 -- 172.21.15.81:0/3041350274 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2e3801a070 con 0x7f2e400ad820 2024-05-08T23:32:14.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.759+0000 7f2e37fff700 1 --2- 172.21.15.81:0/3041350274 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e20066730 0x7f2e20068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.759+0000 7f2e37fff700 1 -- 172.21.15.81:0/3041350274 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f2e38085830 con 0x7f2e400ad820 2024-05-08T23:32:14.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.759+0000 7f2e3f7fe700 1 --2- 172.21.15.81:0/3041350274 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e20066730 0x7f2e20068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.759+0000 7f2e3f7fe700 1 --2- 172.21.15.81:0/3041350274 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e20066730 0x7f2e20068bf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2e30007fd0 tx=0x7f2e30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.767+0000 7f2e37fff700 1 -- 172.21.15.81:0/3041350274 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2e4009a550 con 0x7f2e400ad820 2024-05-08T23:32:14.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.939+0000 7f2e44888700 1 -- 172.21.15.81:0/3041350274 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2e4009a830 con 0x7f2e400ad820 2024-05-08T23:32:14.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.939+0000 7f2e37fff700 1 -- 172.21.15.81:0/3041350274 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2e4009a830 con 0x7f2e400ad820 2024-05-08T23:32:14.941 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:32:14.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.943+0000 7f2e44888700 1 -- 172.21.15.81:0/3041350274 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e20066730 msgr2=0x7f2e20068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.943+0000 7f2e44888700 1 --2- 172.21.15.81:0/3041350274 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e20066730 0x7f2e20068bf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2e30007fd0 tx=0x7f2e30009040 comp rx=0 tx=0).stop 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.943+0000 7f2e44888700 1 -- 172.21.15.81:0/3041350274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e400ad820 msgr2=0x7f2e400cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.943+0000 7f2e44888700 1 --2- 172.21.15.81:0/3041350274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e400ad820 0x7f2e400cc510 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2e38011000 tx=0x7f2e38001e90 comp rx=0 tx=0).stop 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.943+0000 7f2e44888700 1 -- 172.21.15.81:0/3041350274 shutdown_connections 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.943+0000 7f2e44888700 1 --2- 172.21.15.81:0/3041350274 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2e20066730 0x7f2e20068bf0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.943+0000 7f2e44888700 1 --2- 172.21.15.81:0/3041350274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e400ad820 0x7f2e400cc510 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.943+0000 7f2e44888700 1 --2- 172.21.15.81:0/3041350274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e400afaf0 0x7f2e400ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.943+0000 7f2e44888700 1 --2- 172.21.15.81:0/3041350274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e400cd9c0 0x7f2e400af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.943+0000 7f2e44888700 1 -- 172.21.15.81:0/3041350274 >> 172.21.15.81:0/3041350274 conn(0x7f2e400b0de0 msgr2=0x7f2e400b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.943+0000 7f2e44888700 1 -- 172.21.15.81:0/3041350274 shutdown_connections 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:14.943+0000 7f2e44888700 1 -- 172.21.15.81:0/3041350274 wait complete. 2024-05-08T23:32:14.962 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-05-08T23:32:15.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.075+0000 7ff0156ae700 1 Processor -- start 2024-05-08T23:32:15.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.075+0000 7ff0156ae700 1 -- start start 2024-05-08T23:32:15.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff0156ae700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff01006d8c0 0x7ff01015e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff0156ae700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0100721c0 con 0x7ff0100b20f0 2024-05-08T23:32:15.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff0156ae700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0100ad700 con 0x7ff010071a50 2024-05-08T23:32:15.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff0156ae700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01015e650 con 0x7ff01006d8c0 2024-05-08T23:32:15.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00f7fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff0100b20f0 0x7ff010071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45804/0 (socket says 172.21.15.81:45804) 2024-05-08T23:32:15.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00f7fe700 1 -- 172.21.15.81:0/1999352604 learned_addr learned my addr 172.21.15.81:0/1999352604 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:15.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00ffff700 1 --2- 172.21.15.81:0/1999352604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff01006d8c0 0x7ff01015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889217791 0 0) 0x7ff0100721c0 con 0x7ff0100b20f0 2024-05-08T23:32:15.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feffc0035d0 con 0x7ff0100b20f0 2024-05-08T23:32:15.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051758231 0 0) 0x7ff0100ad700 con 0x7ff010071a50 2024-05-08T23:32:15.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0100721c0 con 0x7ff010071a50 2024-05-08T23:32:15.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2572657260 0 0) 0x7feffc0035d0 con 0x7ff0100b20f0 2024-05-08T23:32:15.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff01006d8c0 msgr2=0x7ff01015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:15.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 --2- 172.21.15.81:0/1999352604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff01006d8c0 0x7ff01015e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 >> v1:172.21.15.117:6789/0 conn(0x7ff010071a50 legacy=0x7ff010156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0100b2580 con 0x7ff0100b20f0 2024-05-08T23:32:15.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7ff000001670 con 0x7ff0100b20f0 2024-05-08T23:32:15.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 >> v1:172.21.15.81:6789/0 conn(0x7ff0100b20f0 legacy=0x7ff010071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 --2- 172.21.15.81:0/1999352604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feffc003be0 0x7feffc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 --2- 172.21.15.81:0/1999352604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff01006d8c0 0x7feffc004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 --2- 172.21.15.81:0/1999352604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feffc004db0 0x7feffc005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01015e650 con 0x7feffc003be0 2024-05-08T23:32:15.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc00bb30 con 0x7feffc004db0 2024-05-08T23:32:15.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feffc00bcf0 con 0x7ff01006d8c0 2024-05-08T23:32:15.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00ffff700 1 --2- 172.21.15.81:0/1999352604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feffc003be0 0x7feffc003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00f7fe700 1 --2- 172.21.15.81:0/1999352604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff01006d8c0 0x7feffc004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff014ead700 1 --2- 172.21.15.81:0/1999352604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feffc004db0 0x7feffc005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00ffff700 1 -- 172.21.15.81:0/1999352604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff01006d8c0 msgr2=0x7feffc004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:15.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00ffff700 1 --2- 172.21.15.81:0/1999352604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff01006d8c0 0x7feffc004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00ffff700 1 -- 172.21.15.81:0/1999352604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feffc004db0 msgr2=0x7feffc005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:15.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00ffff700 1 --2- 172.21.15.81:0/1999352604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feffc004db0 0x7feffc005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00ffff700 1 -- 172.21.15.81:0/1999352604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc0035d0 con 0x7feffc003be0 2024-05-08T23:32:15.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff014ead700 1 --2- 172.21.15.81:0/1999352604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feffc004db0 0x7feffc005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:15.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00f7fe700 1 --2- 172.21.15.81:0/1999352604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff01006d8c0 0x7feffc004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:15.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00ffff700 1 --2- 172.21.15.81:0/1999352604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feffc003be0 0x7feffc003fe0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7feff8009b70 tx=0x7feffc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4bdba7f355cd79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:15.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff800e6c0 con 0x7feffc003be0 2024-05-08T23:32:15.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff800e820 con 0x7feffc003be0 2024-05-08T23:32:15.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff00effd700 1 -- 172.21.15.81:0/1999352604 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff800eb90 con 0x7feffc003be0 2024-05-08T23:32:15.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff0156ae700 1 -- 172.21.15.81:0/1999352604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feffc003be0 msgr2=0x7feffc003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff0156ae700 1 --2- 172.21.15.81:0/1999352604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feffc003be0 0x7feffc003fe0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7feff8009b70 tx=0x7feffc0058a0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff014ead700 1 -- 172.21.15.81:0/1999352604 reap_dead start 2024-05-08T23:32:15.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff0156ae700 1 -- 172.21.15.81:0/1999352604 shutdown_connections 2024-05-08T23:32:15.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff0156ae700 1 -- 172.21.15.81:0/1999352604 >> 172.21.15.81:0/1999352604 conn(0x7ff0100b0de0 msgr2=0x7ff0100b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:15.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff0156ae700 1 -- 172.21.15.81:0/1999352604 shutdown_connections 2024-05-08T23:32:15.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.079+0000 7ff0156ae700 1 -- 172.21.15.81:0/1999352604 wait complete. 2024-05-08T23:32:15.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff0156ae700 1 Processor -- start 2024-05-08T23:32:15.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff0156ae700 1 -- start start 2024-05-08T23:32:15.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff0156ae700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff010148450 0x7ff01009a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff0156ae700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff010099b70 0x7ff01009aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff0156ae700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0101f57b0 0x7ff0101f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff0156ae700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01015ff60 con 0x7ff010099b70 2024-05-08T23:32:15.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff0156ae700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01015f2e0 con 0x7ff010148450 2024-05-08T23:32:15.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff0156ae700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01015f5c0 con 0x7ff0101f57b0 2024-05-08T23:32:15.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff00f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff010099b70 0x7ff01009aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff00f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff010099b70 0x7ff01009aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39450/0 (socket says 172.21.15.81:39450) 2024-05-08T23:32:15.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff00f7fe700 1 -- 172.21.15.81:0/3404764769 learned_addr learned my addr 172.21.15.81:0/3404764769 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:15.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff014ead700 1 --2- 172.21.15.81:0/3404764769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0101f57b0 0x7ff0101f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff00ffff700 1 --2- 172.21.15.81:0/3404764769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff010148450 0x7ff01009a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff00f7fe700 1 -- 172.21.15.81:0/3404764769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0101f57b0 msgr2=0x7ff0101f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff00f7fe700 1 --2- 172.21.15.81:0/3404764769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0101f57b0 0x7ff0101f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff00f7fe700 1 -- 172.21.15.81:0/3404764769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff010148450 msgr2=0x7ff01009a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff00f7fe700 1 --2- 172.21.15.81:0/3404764769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff010148450 0x7ff01009a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff00f7fe700 1 -- 172.21.15.81:0/3404764769 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01015f8a0 con 0x7ff010099b70 2024-05-08T23:32:15.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff00ffff700 1 --2- 172.21.15.81:0/3404764769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff010148450 0x7ff01009a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:15.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.083+0000 7ff00f7fe700 1 --2- 172.21.15.81:0/3404764769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff010099b70 0x7ff01009aca0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff0000048f0 tx=0x7ff00000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:15.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.087+0000 7ff00cff9700 1 -- 172.21.15.81:0/3404764769 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00000ee70 con 0x7ff010099b70 2024-05-08T23:32:15.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.087+0000 7ff00cff9700 1 -- 172.21.15.81:0/3404764769 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00000c7e0 con 0x7ff010099b70 2024-05-08T23:32:15.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.087+0000 7ff0156ae700 1 -- 172.21.15.81:0/3404764769 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01015fb80 con 0x7ff010099b70 2024-05-08T23:32:15.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.087+0000 7ff00cff9700 1 -- 172.21.15.81:0/3404764769 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00000cb90 con 0x7ff010099b70 2024-05-08T23:32:15.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.087+0000 7ff0156ae700 1 -- 172.21.15.81:0/3404764769 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff010148a10 con 0x7ff010099b70 2024-05-08T23:32:15.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.087+0000 7ff0156ae700 1 -- 172.21.15.81:0/3404764769 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0101600a0 con 0x7ff010099b70 2024-05-08T23:32:15.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.087+0000 7ff00cff9700 1 -- 172.21.15.81:0/3404764769 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff00000ccf0 con 0x7ff010099b70 2024-05-08T23:32:15.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.087+0000 7ff00cff9700 1 --2- 172.21.15.81:0/3404764769 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7feff0066730 0x7feff0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.087+0000 7ff00cff9700 1 -- 172.21.15.81:0/3404764769 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7ff000084250 con 0x7ff010099b70 2024-05-08T23:32:15.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.087+0000 7ff00ffff700 1 --2- 172.21.15.81:0/3404764769 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7feff0066730 0x7feff0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.087+0000 7ff00ffff700 1 --2- 172.21.15.81:0/3404764769 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7feff0066730 0x7feff0068bf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff0100b19f0 tx=0x7feff8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:15.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.095+0000 7ff00cff9700 1 -- 172.21.15.81:0/3404764769 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff0101600a0 con 0x7ff010099b70 2024-05-08T23:32:15.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.263+0000 7ff0156ae700 1 -- 172.21.15.81:0/3404764769 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7ff0101600a0 con 0x7ff010099b70 2024-05-08T23:32:15.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.263+0000 7ff00cff9700 1 -- 172.21.15.81:0/3404764769 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v39) v1 ==== 97+0+11 (secure 0 0 0) 0x7ff0101600a0 con 0x7ff010099b70 2024-05-08T23:32:15.265 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:32:15.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.267+0000 7ff0156ae700 1 -- 172.21.15.81:0/3404764769 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7feff0066730 msgr2=0x7feff0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.267+0000 7ff0156ae700 1 --2- 172.21.15.81:0/3404764769 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7feff0066730 0x7feff0068bf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff0100b19f0 tx=0x7feff8008030 comp rx=0 tx=0).stop 2024-05-08T23:32:15.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.267+0000 7ff0156ae700 1 -- 172.21.15.81:0/3404764769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff010099b70 msgr2=0x7ff01009aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.267+0000 7ff0156ae700 1 --2- 172.21.15.81:0/3404764769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff010099b70 0x7ff01009aca0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff0000048f0 tx=0x7ff00000ed50 comp rx=0 tx=0).stop 2024-05-08T23:32:15.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.267+0000 7ff0156ae700 1 -- 172.21.15.81:0/3404764769 shutdown_connections 2024-05-08T23:32:15.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.267+0000 7ff0156ae700 1 --2- 172.21.15.81:0/3404764769 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7feff0066730 0x7feff0068bf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.267+0000 7ff0156ae700 1 --2- 172.21.15.81:0/3404764769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0101f57b0 0x7ff0101f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.267+0000 7ff0156ae700 1 --2- 172.21.15.81:0/3404764769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff010099b70 0x7ff01009aca0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.267+0000 7ff0156ae700 1 --2- 172.21.15.81:0/3404764769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff010148450 0x7ff01009a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.267+0000 7ff0156ae700 1 -- 172.21.15.81:0/3404764769 >> 172.21.15.81:0/3404764769 conn(0x7ff0100b0de0 msgr2=0x7ff0100b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:15.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.267+0000 7ff0156ae700 1 -- 172.21.15.81:0/3404764769 shutdown_connections 2024-05-08T23:32:15.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.267+0000 7ff0156ae700 1 -- 172.21.15.81:0/3404764769 wait complete. 2024-05-08T23:32:15.287 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:32:15.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7372d51700 1 Processor -- start 2024-05-08T23:32:15.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7372d51700 1 -- start start 2024-05-08T23:32:15.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7372d51700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f736c06b470 0x7f736c0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7372d51700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f736c0d0420 con 0x7f736c06c1b0 2024-05-08T23:32:15.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7372d51700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f736c0d0180 con 0x7f736c1606b0 2024-05-08T23:32:15.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7372d51700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f736c0b04a0 con 0x7f736c06b470 2024-05-08T23:32:15.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f737154e700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f736c06c1b0 0x7f736c0b0f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45816/0 (socket says 172.21.15.81:45816) 2024-05-08T23:32:15.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f737154e700 1 -- 172.21.15.81:0/3187378704 learned_addr learned my addr 172.21.15.81:0/3187378704 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:15.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7372550700 1 --2- 172.21.15.81:0/3187378704 >> v2:172.21.15.117:3300/0 conn(0x7f736c06b470 0x7f736c0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 -- 172.21.15.81:0/3187378704 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970816832 0 0) 0x7f736c0d0420 con 0x7f736c06c1b0 2024-05-08T23:32:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 -- 172.21.15.81:0/3187378704 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73580035d0 con 0x7f736c06c1b0 2024-05-08T23:32:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 -- 172.21.15.81:0/3187378704 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3240991176 0 0) 0x7f73580035d0 con 0x7f736c06c1b0 2024-05-08T23:32:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 -- 172.21.15.81:0/3187378704 >> v2:172.21.15.117:3300/0 conn(0x7f736c06b470 msgr2=0x7f736c0cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 --2- 172.21.15.81:0/3187378704 >> v2:172.21.15.117:3300/0 conn(0x7f736c06b470 0x7f736c0cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 -- 172.21.15.81:0/3187378704 >> v1:172.21.15.117:6789/0 conn(0x7f736c1606b0 legacy=0x7f736c06baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 -- 172.21.15.81:0/3187378704 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c0b3850 con 0x7f736c06c1b0 2024-05-08T23:32:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 -- 172.21.15.81:0/3187378704 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f735c001670 con 0x7f736c06c1b0 2024-05-08T23:32:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 -- 172.21.15.81:0/3187378704 >> v1:172.21.15.81:6789/0 conn(0x7f736c06c1b0 legacy=0x7f736c0b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 --2- 172.21.15.81:0/3187378704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7358003a70 0x7f7358003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 --2- 172.21.15.81:0/3187378704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7358004870 0x7f7358004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 --2- 172.21.15.81:0/3187378704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7358005350 0x7f735800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 -- 172.21.15.81:0/3187378704 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c0b04a0 con 0x7f7358005350 2024-05-08T23:32:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 -- 172.21.15.81:0/3187378704 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735800c250 con 0x7f7358004870 2024-05-08T23:32:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 -- 172.21.15.81:0/3187378704 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735800c410 con 0x7f7358003a70 2024-05-08T23:32:15.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7372550700 1 --2- 172.21.15.81:0/3187378704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7358005350 0x7f735800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f737154e700 1 --2- 172.21.15.81:0/3187378704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7358004870 0x7f7358004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7371d4f700 1 --2- 172.21.15.81:0/3187378704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7358003a70 0x7f7358003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7372550700 1 -- 172.21.15.81:0/3187378704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7358003a70 msgr2=0x7f7358003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:15.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7372550700 1 --2- 172.21.15.81:0/3187378704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7358003a70 0x7f7358003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7372550700 1 -- 172.21.15.81:0/3187378704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7358004870 msgr2=0x7f7358004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:15.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7372550700 1 --2- 172.21.15.81:0/3187378704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7358004870 0x7f7358004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7372550700 1 -- 172.21.15.81:0/3187378704 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73580035d0 con 0x7f7358005350 2024-05-08T23:32:15.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7372550700 1 -- 172.21.15.81:0/3187378704 reap_dead start 2024-05-08T23:32:15.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7372550700 1 --2- 172.21.15.81:0/3187378704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7358005350 0x7f735800ba30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7364009fc0 tx=0x7f73580058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971bc535e3f40a27 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:15.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 -- 172.21.15.81:0/3187378704 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364003a40 con 0x7f7358005350 2024-05-08T23:32:15.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 -- 172.21.15.81:0/3187378704 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736400f040 con 0x7f7358005350 2024-05-08T23:32:15.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.399+0000 7f7370d4d700 1 -- 172.21.15.81:0/3187378704 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364003e00 con 0x7f7358005350 2024-05-08T23:32:15.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7372d51700 1 -- 172.21.15.81:0/3187378704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7358005350 msgr2=0x7f735800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7372d51700 1 --2- 172.21.15.81:0/3187378704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7358005350 0x7f735800ba30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7364009fc0 tx=0x7f73580058a0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7372d51700 1 -- 172.21.15.81:0/3187378704 shutdown_connections 2024-05-08T23:32:15.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7372d51700 1 --2- 172.21.15.81:0/3187378704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7358005350 0x7f735800ba30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7372d51700 1 -- 172.21.15.81:0/3187378704 >> 172.21.15.81:0/3187378704 conn(0x7f736c0b2970 msgr2=0x7f736c0b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:15.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7372d51700 1 -- 172.21.15.81:0/3187378704 shutdown_connections 2024-05-08T23:32:15.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7372d51700 1 -- 172.21.15.81:0/3187378704 wait complete. 2024-05-08T23:32:15.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7372d51700 1 Processor -- start 2024-05-08T23:32:15.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7372d51700 1 -- start start 2024-05-08T23:32:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7372d51700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7358005350 0x7f736c14b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7372d51700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f736c0b5530 0x7f736c0b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7372d51700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f736c0b3f80 0x7f736c0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7372d51700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c0b4490 con 0x7f7358005350 2024-05-08T23:32:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7372d51700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c0b4770 con 0x7f736c0b3f80 2024-05-08T23:32:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7372d51700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736c0b4a50 con 0x7f736c0b5530 2024-05-08T23:32:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7371d4f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7358005350 0x7f736c14b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7371d4f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7358005350 0x7f736c14b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39464/0 (socket says 172.21.15.81:39464) 2024-05-08T23:32:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.403+0000 7f7371d4f700 1 -- 172.21.15.81:0/3985773972 learned_addr learned my addr 172.21.15.81:0/3985773972 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f7372550700 1 --2- 172.21.15.81:0/3985773972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f736c0b3f80 0x7f736c0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f737154e700 1 --2- 172.21.15.81:0/3985773972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f736c0b5530 0x7f736c0b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f7371d4f700 1 -- 172.21.15.81:0/3985773972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f736c0b5530 msgr2=0x7f736c0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f7371d4f700 1 --2- 172.21.15.81:0/3985773972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f736c0b5530 0x7f736c0b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f7371d4f700 1 -- 172.21.15.81:0/3985773972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f736c0b3f80 msgr2=0x7f736c0b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f7371d4f700 1 --2- 172.21.15.81:0/3985773972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f736c0b3f80 0x7f736c0b62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f7371d4f700 1 -- 172.21.15.81:0/3985773972 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c0b4d30 con 0x7f7358005350 2024-05-08T23:32:15.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f7371d4f700 1 --2- 172.21.15.81:0/3985773972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7358005350 0x7f736c14b1d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f736800e580 tx=0x7f736800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:15.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f73627fc700 1 -- 172.21.15.81:0/3985773972 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73680052e0 con 0x7f7358005350 2024-05-08T23:32:15.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f73627fc700 1 -- 172.21.15.81:0/3985773972 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7368005440 con 0x7f7358005350 2024-05-08T23:32:15.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f7372d51700 1 -- 172.21.15.81:0/3985773972 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736c0b5010 con 0x7f7358005350 2024-05-08T23:32:15.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f73627fc700 1 -- 172.21.15.81:0/3985773972 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73680038c0 con 0x7f7358005350 2024-05-08T23:32:15.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f7372d51700 1 -- 172.21.15.81:0/3985773972 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736c15b8f0 con 0x7f7358005350 2024-05-08T23:32:15.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f7372d51700 1 -- 172.21.15.81:0/3985773972 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f736c0b48b0 con 0x7f7358005350 2024-05-08T23:32:15.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f73627fc700 1 -- 172.21.15.81:0/3985773972 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7368003a20 con 0x7f7358005350 2024-05-08T23:32:15.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.407+0000 7f73627fc700 1 --2- 172.21.15.81:0/3985773972 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7350066730 0x7f7350068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.411+0000 7f73627fc700 1 -- 172.21.15.81:0/3985773972 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f73680839d0 con 0x7f7358005350 2024-05-08T23:32:15.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.411+0000 7f737154e700 1 --2- 172.21.15.81:0/3985773972 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7350066730 0x7f7350068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.411+0000 7f737154e700 1 --2- 172.21.15.81:0/3985773972 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7350066730 0x7f7350068bf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f735c001120 tx=0x7f735c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:15.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.415+0000 7f73627fc700 1 -- 172.21.15.81:0/3985773972 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f736c0b48b0 con 0x7f7358005350 2024-05-08T23:32:15.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.591+0000 7f7372d51700 1 -- 172.21.15.81:0/3985773972 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f736c0b4b90 con 0x7f7358005350 2024-05-08T23:32:15.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.591+0000 7f73627fc700 1 -- 172.21.15.81:0/3985773972 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) v1 ==== 80+0+0 (secure 0 0 0) 0x7f736c0b4b90 con 0x7f7358005350 2024-05-08T23:32:15.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.595+0000 7f7372d51700 1 -- 172.21.15.81:0/3985773972 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7350066730 msgr2=0x7f7350068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.595+0000 7f7372d51700 1 --2- 172.21.15.81:0/3985773972 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7350066730 0x7f7350068bf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f735c001120 tx=0x7f735c009040 comp rx=0 tx=0).stop 2024-05-08T23:32:15.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.595+0000 7f7372d51700 1 -- 172.21.15.81:0/3985773972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7358005350 msgr2=0x7f736c14b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.595+0000 7f7372d51700 1 --2- 172.21.15.81:0/3985773972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7358005350 0x7f736c14b1d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f736800e580 tx=0x7f736800e940 comp rx=0 tx=0).stop 2024-05-08T23:32:15.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.595+0000 7f7372d51700 1 -- 172.21.15.81:0/3985773972 shutdown_connections 2024-05-08T23:32:15.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.595+0000 7f7372d51700 1 --2- 172.21.15.81:0/3985773972 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7350066730 0x7f7350068bf0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.595+0000 7f7372d51700 1 --2- 172.21.15.81:0/3985773972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f736c0b3f80 0x7f736c0b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.595+0000 7f7372d51700 1 --2- 172.21.15.81:0/3985773972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f736c0b5530 0x7f736c0b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.595+0000 7f7372d51700 1 --2- 172.21.15.81:0/3985773972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7358005350 0x7f736c14b1d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.595+0000 7f7372d51700 1 -- 172.21.15.81:0/3985773972 >> 172.21.15.81:0/3985773972 conn(0x7f736c0b2970 msgr2=0x7f736c0c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:15.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.595+0000 7f7372d51700 1 -- 172.21.15.81:0/3985773972 shutdown_connections 2024-05-08T23:32:15.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.595+0000 7f7372d51700 1 -- 172.21.15.81:0/3985773972 wait complete. 2024-05-08T23:32:15.599 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:32:15.610 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:32:15.610 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:15.617 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:15.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f3599bfe700 1 Processor -- start 2024-05-08T23:32:15.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f3599bfe700 1 -- start start 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f3599bfe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f35940cb3b0 0x7f35941478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f3599bfe700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35941656a0 con 0x7f3594144d40 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f3599bfe700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3594165970 con 0x7f3594165240 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f3599bfe700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3594161c50 con 0x7f35940cb3b0 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f3593fff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f3594144d40 0x7f359406c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45828/0 (socket says 172.21.15.81:45828) 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f3593fff700 1 -- 172.21.15.81:0/129641375 learned_addr learned my addr 172.21.15.81:0/129641375 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f35937fe700 1 --2- 172.21.15.81:0/129641375 >> v2:172.21.15.117:3300/0 conn(0x7f35940cb3b0 0x7f35941478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f35927fc700 1 -- 172.21.15.81:0/129641375 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667257812 0 0) 0x7f35941656a0 con 0x7f3594144d40 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f35927fc700 1 -- 172.21.15.81:0/129641375 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35800035d0 con 0x7f3594144d40 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f35937fe700 1 -- 172.21.15.81:0/129641375 >> v1:172.21.15.117:6789/0 conn(0x7f3594165240 legacy=0x7f3594162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f35937fe700 1 -- 172.21.15.81:0/129641375 >> v1:172.21.15.81:6789/0 conn(0x7f3594144d40 legacy=0x7f359406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f35937fe700 1 -- 172.21.15.81:0/129641375 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3594165c50 con 0x7f35940cb3b0 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f35937fe700 1 --2- 172.21.15.81:0/129641375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35940cb3b0 0x7f35941478c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f357c009850 tx=0x7f357c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232d9c33821e28c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f35927fc700 1 -- 172.21.15.81:0/129641375 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c011830 con 0x7f35940cb3b0 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f35927fc700 1 -- 172.21.15.81:0/129641375 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357c011990 con 0x7f35940cb3b0 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f35927fc700 1 -- 172.21.15.81:0/129641375 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c011d00 con 0x7f35940cb3b0 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f3599bfe700 1 -- 172.21.15.81:0/129641375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35940cb3b0 msgr2=0x7f35941478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.767+0000 7f3599bfe700 1 --2- 172.21.15.81:0/129641375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35940cb3b0 0x7f35941478c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f357c009850 tx=0x7f357c009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3599bfe700 1 -- 172.21.15.81:0/129641375 shutdown_connections 2024-05-08T23:32:15.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3599bfe700 1 --2- 172.21.15.81:0/129641375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35940cb3b0 0x7f35941478c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3599bfe700 1 -- 172.21.15.81:0/129641375 >> 172.21.15.81:0/129641375 conn(0x7f359406a360 msgr2=0x7f359406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:15.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3599bfe700 1 -- 172.21.15.81:0/129641375 shutdown_connections 2024-05-08T23:32:15.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3599bfe700 1 -- 172.21.15.81:0/129641375 wait complete. 2024-05-08T23:32:15.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3599bfe700 1 Processor -- start 2024-05-08T23:32:15.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3599bfe700 1 -- start start 2024-05-08T23:32:15.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3599bfe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35940cb3b0 0x7f35941eaf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3599bfe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3594144d40 0x7f3594148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3599bfe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3594165240 0x7f3594149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3599bfe700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3594159eb0 con 0x7f3594165240 2024-05-08T23:32:15.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3599bfe700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3594157d90 con 0x7f35940cb3b0 2024-05-08T23:32:15.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3599bfe700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35941592a0 con 0x7f3594144d40 2024-05-08T23:32:15.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3593fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3594165240 0x7f3594149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3593fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3594165240 0x7f3594149450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39474/0 (socket says 172.21.15.81:39474) 2024-05-08T23:32:15.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3593fff700 1 -- 172.21.15.81:0/271085938 learned_addr learned my addr 172.21.15.81:0/271085938 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:15.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3592ffd700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3594144d40 0x7f3594148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f35937fe700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35940cb3b0 0x7f35941eaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3593fff700 1 -- 172.21.15.81:0/271085938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3594144d40 msgr2=0x7f3594148f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3593fff700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3594144d40 0x7f3594148f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3593fff700 1 -- 172.21.15.81:0/271085938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35940cb3b0 msgr2=0x7f35941eaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3593fff700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35940cb3b0 0x7f35941eaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3593fff700 1 -- 172.21.15.81:0/271085938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357c0094b0 con 0x7f3594165240 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3592ffd700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3594144d40 0x7f3594148f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f35937fe700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35940cb3b0 0x7f35941eaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.771+0000 7f3593fff700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3594165240 0x7f3594149450 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f358400c4f0 tx=0x7f358400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.775+0000 7f358bfff700 1 -- 172.21.15.81:0/271085938 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584005d90 con 0x7f3594165240 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.775+0000 7f3599bfe700 1 -- 172.21.15.81:0/271085938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3594159580 con 0x7f3594165240 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.775+0000 7f358bfff700 1 -- 172.21.15.81:0/271085938 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35840070d0 con 0x7f3594165240 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.775+0000 7f358bfff700 1 -- 172.21.15.81:0/271085938 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584007460 con 0x7f3594165240 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.775+0000 7f3599bfe700 1 -- 172.21.15.81:0/271085938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3594159b40 con 0x7f3594165240 2024-05-08T23:32:15.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.775+0000 7f358bfff700 1 -- 172.21.15.81:0/271085938 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f35840075c0 con 0x7f3594165240 2024-05-08T23:32:15.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.775+0000 7f3599bfe700 1 -- 172.21.15.81:0/271085938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3594159d00 con 0x7f3594165240 2024-05-08T23:32:15.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.775+0000 7f358bfff700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3574066730 0x7f3574068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.775+0000 7f358bfff700 1 -- 172.21.15.81:0/271085938 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f35840838e0 con 0x7f3594165240 2024-05-08T23:32:15.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.775+0000 7f35937fe700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3574066730 0x7f3574068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.775+0000 7f35937fe700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3574066730 0x7f3574068bf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f357c009ba0 tx=0x7f357c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:15.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.783+0000 7f358bfff700 1 -- 172.21.15.81:0/271085938 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3584050a40 con 0x7f3594165240 2024-05-08T23:32:15.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.959+0000 7f3599bfe700 1 -- 172.21.15.81:0/271085938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35940684b0 con 0x7f3594165240 2024-05-08T23:32:15.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.959+0000 7f358bfff700 1 -- 172.21.15.81:0/271085938 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+17880 (secure 0 0 0) 0x7f3584054060 con 0x7f3594165240 2024-05-08T23:32:15.963 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:15.963 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":39,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:48.310750+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:31:35.620423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:31:36.878524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-05-08T23:31:46.337484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:15.965 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.963+0000 7f3599bfe700 1 -- 172.21.15.81:0/271085938 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3574066730 msgr2=0x7f3574068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.965 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.963+0000 7f3599bfe700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3574066730 0x7f3574068bf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f357c009ba0 tx=0x7f357c008030 comp rx=0 tx=0).stop 2024-05-08T23:32:15.965 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.963+0000 7f3599bfe700 1 -- 172.21.15.81:0/271085938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3594165240 msgr2=0x7f3594149450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.963+0000 7f3599bfe700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3594165240 0x7f3594149450 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f358400c4f0 tx=0x7f358400c8b0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.963+0000 7f3599bfe700 1 -- 172.21.15.81:0/271085938 shutdown_connections 2024-05-08T23:32:15.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.963+0000 7f3599bfe700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3574066730 0x7f3574068bf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.963+0000 7f3599bfe700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3594165240 0x7f3594149450 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.963+0000 7f3599bfe700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3594144d40 0x7f3594148f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.963+0000 7f3599bfe700 1 --2- 172.21.15.81:0/271085938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35940cb3b0 0x7f35941eaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.963+0000 7f3599bfe700 1 -- 172.21.15.81:0/271085938 >> 172.21.15.81:0/271085938 conn(0x7f359406a360 msgr2=0x7f359406c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:15.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.963+0000 7f3599bfe700 1 -- 172.21.15.81:0/271085938 shutdown_connections 2024-05-08T23:32:15.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:15.963+0000 7f3599bfe700 1 -- 172.21.15.81:0/271085938 wait complete. 2024-05-08T23:32:15.981 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:31:35.620423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:31:36.878524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-05-08T23:31:46.337484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:32:15.981 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:16.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.103+0000 7fbf1b59e700 1 Processor -- start 2024-05-08T23:32:16.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.103+0000 7fbf1b59e700 1 -- start start 2024-05-08T23:32:16.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.103+0000 7fbf1b59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c15d2f0 0x7fbf1c0ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.103+0000 7fbf1b59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbf1c0cf8b0 0x7fbf1c0cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.103+0000 7fbf1b59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1c0ceb40 con 0x7fbf1c0b3550 2024-05-08T23:32:16.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.103+0000 7fbf1b59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1c0d3a10 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.103+0000 7fbf1b59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c15d6d0 con 0x7fbf1c15d2f0 2024-05-08T23:32:16.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.103+0000 7fbf1a59c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c15d2f0 0x7fbf1c0ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.103+0000 7fbf19d9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbf1c0cf8b0 0x7fbf1c0cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.103+0000 7fbf1ad9d700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fbf1c0b3550 0x7fbf1c0b39e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:41800/0 (socket says 172.21.15.81:41800) 2024-05-08T23:32:16.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.103+0000 7fbf1ad9d700 1 -- 172.21.15.81:0/3248930154 learned_addr learned my addr 172.21.15.81:0/3248930154 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:16.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf1959a700 1 -- 172.21.15.81:0/3248930154 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818078125 0 0) 0x7fbf1c0ceb40 con 0x7fbf1c0b3550 2024-05-08T23:32:16.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf1959a700 1 -- 172.21.15.81:0/3248930154 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf080035d0 con 0x7fbf1c0b3550 2024-05-08T23:32:16.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf19d9b700 1 -- 172.21.15.81:0/3248930154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c15d2f0 msgr2=0x7fbf1c0ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:16.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf19d9b700 1 --2- 172.21.15.81:0/3248930154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c15d2f0 0x7fbf1c0ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf19d9b700 1 -- 172.21.15.81:0/3248930154 >> v1:172.21.15.117:6789/0 conn(0x7fbf1c0b3550 legacy=0x7fbf1c0b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf19d9b700 1 -- 172.21.15.81:0/3248930154 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c0d0250 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf19d9b700 1 --2- 172.21.15.81:0/3248930154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf1c0cf8b0 0x7fbf1c0cfd10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009850 tx=0x7fbf04009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e6e1e70e56a6998 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:16.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf1959a700 1 -- 172.21.15.81:0/3248930154 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04011830 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf1959a700 1 -- 172.21.15.81:0/3248930154 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf04011990 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf1959a700 1 -- 172.21.15.81:0/3248930154 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04011d40 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf1b59e700 1 -- 172.21.15.81:0/3248930154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf1c0cf8b0 msgr2=0x7fbf1c0cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf1b59e700 1 --2- 172.21.15.81:0/3248930154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf1c0cf8b0 0x7fbf1c0cfd10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009850 tx=0x7fbf04009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:16.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf1b59e700 1 -- 172.21.15.81:0/3248930154 shutdown_connections 2024-05-08T23:32:16.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf1b59e700 1 --2- 172.21.15.81:0/3248930154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf1c0cf8b0 0x7fbf1c0cfd10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf1b59e700 1 --2- 172.21.15.81:0/3248930154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c15d2f0 0x7fbf1c0ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf1b59e700 1 -- 172.21.15.81:0/3248930154 >> 172.21.15.81:0/3248930154 conn(0x7fbf1c09c320 msgr2=0x7fbf1c0d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:16.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf1b59e700 1 -- 172.21.15.81:0/3248930154 shutdown_connections 2024-05-08T23:32:16.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.107+0000 7fbf1b59e700 1 -- 172.21.15.81:0/3248930154 wait complete. 2024-05-08T23:32:16.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf1b59e700 1 Processor -- start 2024-05-08T23:32:16.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf1b59e700 1 -- start start 2024-05-08T23:32:16.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf1b59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf1c0b3550 0x7fbf1c154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf1b59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c0cf8b0 0x7fbf1c06c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf1b59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c15d2f0 0x7fbf1c06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf1b59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c0ca0b0 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf1b59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c1549d0 con 0x7fbf1c0b3550 2024-05-08T23:32:16.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf1b59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c0ca300 con 0x7fbf1c15d2f0 2024-05-08T23:32:16.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf19d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c0cf8b0 0x7fbf1c06c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf19d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c0cf8b0 0x7fbf1c06c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57744/0 (socket says 172.21.15.81:57744) 2024-05-08T23:32:16.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf19d9b700 1 -- 172.21.15.81:0/1150005326 learned_addr learned my addr 172.21.15.81:0/1150005326 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:16.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf1a59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf1c0b3550 0x7fbf1c154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf1ad9d700 1 --2- 172.21.15.81:0/1150005326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c15d2f0 0x7fbf1c06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf19d9b700 1 -- 172.21.15.81:0/1150005326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c15d2f0 msgr2=0x7fbf1c06c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf19d9b700 1 --2- 172.21.15.81:0/1150005326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c15d2f0 0x7fbf1c06c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf19d9b700 1 -- 172.21.15.81:0/1150005326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf1c0b3550 msgr2=0x7fbf1c154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf19d9b700 1 --2- 172.21.15.81:0/1150005326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf1c0b3550 0x7fbf1c154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf19d9b700 1 -- 172.21.15.81:0/1150005326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf040094b0 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf19d9b700 1 --2- 172.21.15.81:0/1150005326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c0cf8b0 0x7fbf1c06c1d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c08c230 tx=0x7fbf04004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:16.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf12ffd700 1 -- 172.21.15.81:0/1150005326 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04004380 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf12ffd700 1 -- 172.21.15.81:0/1150005326 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0401f070 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf1b59e700 1 -- 172.21.15.81:0/1150005326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1c0ca5e0 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf12ffd700 1 -- 172.21.15.81:0/1150005326 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf040036c0 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.111+0000 7fbf1b59e700 1 -- 172.21.15.81:0/1150005326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c0caba0 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.115+0000 7fbf1b59e700 1 -- 172.21.15.81:0/1150005326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1c0684b0 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.115+0000 7fbf12ffd700 1 -- 172.21.15.81:0/1150005326 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbf04021030 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.115+0000 7fbf12ffd700 1 --2- 172.21.15.81:0/1150005326 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbefc066730 0x7fbefc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.115+0000 7fbf12ffd700 1 -- 172.21.15.81:0/1150005326 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fbf04085550 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.115+0000 7fbf1a59c700 1 --2- 172.21.15.81:0/1150005326 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbefc066730 0x7fbefc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.115+0000 7fbf1a59c700 1 --2- 172.21.15.81:0/1150005326 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbefc066730 0x7fbefc068bf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009bf0 tx=0x7fbf14009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:16.125 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.123+0000 7fbf12ffd700 1 -- 172.21.15.81:0/1150005326 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbf040528e0 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.291+0000 7fbf1b59e700 1 -- 172.21.15.81:0/1150005326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf1c0d1a90 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.291+0000 7fbf12ffd700 1 -- 172.21.15.81:0/1150005326 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf04006ce0 con 0x7fbf1c0cf8b0 2024-05-08T23:32:16.293 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:32:16.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.291+0000 7fbf1b59e700 1 -- 172.21.15.81:0/1150005326 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbefc066730 msgr2=0x7fbefc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.291+0000 7fbf1b59e700 1 --2- 172.21.15.81:0/1150005326 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbefc066730 0x7fbefc068bf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009bf0 tx=0x7fbf14009240 comp rx=0 tx=0).stop 2024-05-08T23:32:16.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.291+0000 7fbf1b59e700 1 -- 172.21.15.81:0/1150005326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c0cf8b0 msgr2=0x7fbf1c06c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.291+0000 7fbf1b59e700 1 --2- 172.21.15.81:0/1150005326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c0cf8b0 0x7fbf1c06c1d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c08c230 tx=0x7fbf04004770 comp rx=0 tx=0).stop 2024-05-08T23:32:16.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.291+0000 7fbf1b59e700 1 -- 172.21.15.81:0/1150005326 shutdown_connections 2024-05-08T23:32:16.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.291+0000 7fbf1b59e700 1 --2- 172.21.15.81:0/1150005326 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fbefc066730 0x7fbefc068bf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.291+0000 7fbf1b59e700 1 --2- 172.21.15.81:0/1150005326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf1c15d2f0 0x7fbf1c06c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.291+0000 7fbf1b59e700 1 --2- 172.21.15.81:0/1150005326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c0cf8b0 0x7fbf1c06c1d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.291+0000 7fbf1b59e700 1 --2- 172.21.15.81:0/1150005326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf1c0b3550 0x7fbf1c154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.291+0000 7fbf1b59e700 1 -- 172.21.15.81:0/1150005326 >> 172.21.15.81:0/1150005326 conn(0x7fbf1c09c320 msgr2=0x7fbf1c0d0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:16.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.291+0000 7fbf1b59e700 1 -- 172.21.15.81:0/1150005326 shutdown_connections 2024-05-08T23:32:16.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.295+0000 7fbf1b59e700 1 -- 172.21.15.81:0/1150005326 wait complete. 2024-05-08T23:32:16.312 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:32:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.431+0000 7fef3c27d700 1 Processor -- start 2024-05-08T23:32:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.431+0000 7fef3c27d700 1 -- start start 2024-05-08T23:32:16.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.431+0000 7fef3c27d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fef3415b350 0x7fef3415b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.431+0000 7fef3c27d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef340b4560 0x7fef340b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef3c27d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3406acf0 con 0x7fef3415af70 2024-05-08T23:32:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef3c27d700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef340b4f30 con 0x7fef3415b350 2024-05-08T23:32:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef3c27d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3406a400 con 0x7fef340b4560 2024-05-08T23:32:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef39818700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fef3415b350 0x7fef3415b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef39818700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fef3415b350 0x7fef3415b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57760/0 (socket says 172.21.15.81:57760) 2024-05-08T23:32:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef39818700 1 -- 172.21.15.81:0/374035073 learned_addr learned my addr 172.21.15.81:0/374035073 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef3a81a700 1 --2- 172.21.15.81:0/374035073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef340b4560 0x7fef340b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef39818700 1 -- 172.21.15.81:0/374035073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef340b4560 msgr2=0x7fef340b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef39818700 1 --2- 172.21.15.81:0/374035073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef340b4560 0x7fef340b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef39818700 1 -- 172.21.15.81:0/374035073 >> v1:172.21.15.117:6789/0 conn(0x7fef3415af70 legacy=0x7fef3406e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef39818700 1 -- 172.21.15.81:0/374035073 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3406a6a0 con 0x7fef3415b350 2024-05-08T23:32:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef39818700 1 --2- 172.21.15.81:0/374035073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef3415b350 0x7fef3415b770 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fef24009910 tx=0x7fef24009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ce3095bc793084 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef39017700 1 -- 172.21.15.81:0/374035073 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2400ba60 con 0x7fef3415b350 2024-05-08T23:32:16.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef39017700 1 -- 172.21.15.81:0/374035073 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2400bbc0 con 0x7fef3415b350 2024-05-08T23:32:16.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef39017700 1 -- 172.21.15.81:0/374035073 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24009e60 con 0x7fef3415b350 2024-05-08T23:32:16.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef3c27d700 1 -- 172.21.15.81:0/374035073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef3415b350 msgr2=0x7fef3415b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef3c27d700 1 --2- 172.21.15.81:0/374035073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef3415b350 0x7fef3415b770 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fef24009910 tx=0x7fef24009c20 comp rx=0 tx=0).stop 2024-05-08T23:32:16.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef3c27d700 1 -- 172.21.15.81:0/374035073 shutdown_connections 2024-05-08T23:32:16.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef3c27d700 1 --2- 172.21.15.81:0/374035073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef340b4560 0x7fef340b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef3c27d700 1 --2- 172.21.15.81:0/374035073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef3415b350 0x7fef3415b770 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef3c27d700 1 -- 172.21.15.81:0/374035073 >> 172.21.15.81:0/374035073 conn(0x7fef340cf630 msgr2=0x7fef340cfa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:16.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef3c27d700 1 -- 172.21.15.81:0/374035073 shutdown_connections 2024-05-08T23:32:16.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.435+0000 7fef3c27d700 1 -- 172.21.15.81:0/374035073 wait complete. 2024-05-08T23:32:16.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.439+0000 7fef3c27d700 1 Processor -- start 2024-05-08T23:32:16.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.439+0000 7fef3c27d700 1 -- start start 2024-05-08T23:32:16.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.439+0000 7fef3c27d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef340b4560 0x7fef3414fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.439+0000 7fef3c27d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef3415af70 0x7fef34152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.439+0000 7fef3c27d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef3415b350 0x7fef341548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.439+0000 7fef3c27d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef34152040 con 0x7fef340b4560 2024-05-08T23:32:16.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.439+0000 7fef3c27d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef341513c0 con 0x7fef3415b350 2024-05-08T23:32:16.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.439+0000 7fef3c27d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef341516a0 con 0x7fef3415af70 2024-05-08T23:32:16.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.439+0000 7fef3a019700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef340b4560 0x7fef3414fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef3a019700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef340b4560 0x7fef3414fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57766/0 (socket says 172.21.15.81:57766) 2024-05-08T23:32:16.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef3a019700 1 -- 172.21.15.81:0/671681008 learned_addr learned my addr 172.21.15.81:0/671681008 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:16.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef39818700 1 --2- 172.21.15.81:0/671681008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef3415af70 0x7fef34152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef3a81a700 1 --2- 172.21.15.81:0/671681008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef3415b350 0x7fef341548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef3a019700 1 -- 172.21.15.81:0/671681008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef3415af70 msgr2=0x7fef34152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef3a019700 1 --2- 172.21.15.81:0/671681008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef3415af70 0x7fef34152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef3a019700 1 -- 172.21.15.81:0/671681008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef3415b350 msgr2=0x7fef341548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef3a019700 1 --2- 172.21.15.81:0/671681008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef3415b350 0x7fef341548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef3a019700 1 -- 172.21.15.81:0/671681008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef24009570 con 0x7fef340b4560 2024-05-08T23:32:16.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef3a019700 1 --2- 172.21.15.81:0/671681008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef340b4560 0x7fef3414fbf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fef3000c500 tx=0x7fef3000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:16.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef2affd700 1 -- 172.21.15.81:0/671681008 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef300052d0 con 0x7fef340b4560 2024-05-08T23:32:16.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef2affd700 1 -- 172.21.15.81:0/671681008 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef30005430 con 0x7fef340b4560 2024-05-08T23:32:16.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef3c27d700 1 -- 172.21.15.81:0/671681008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef34151980 con 0x7fef340b4560 2024-05-08T23:32:16.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef2affd700 1 -- 172.21.15.81:0/671681008 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef300055a0 con 0x7fef340b4560 2024-05-08T23:32:16.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef3c27d700 1 -- 172.21.15.81:0/671681008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef34150130 con 0x7fef340b4560 2024-05-08T23:32:16.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef3c27d700 1 -- 172.21.15.81:0/671681008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef34152180 con 0x7fef340b4560 2024-05-08T23:32:16.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.443+0000 7fef2affd700 1 -- 172.21.15.81:0/671681008 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fef300038e0 con 0x7fef340b4560 2024-05-08T23:32:16.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.447+0000 7fef2affd700 1 --2- 172.21.15.81:0/671681008 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fef18066730 0x7fef18068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.447+0000 7fef39818700 1 --2- 172.21.15.81:0/671681008 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fef18066730 0x7fef18068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.447+0000 7fef2affd700 1 -- 172.21.15.81:0/671681008 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fef30083600 con 0x7fef340b4560 2024-05-08T23:32:16.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.447+0000 7fef39818700 1 --2- 172.21.15.81:0/671681008 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fef18066730 0x7fef18068bf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fef24000c00 tx=0x7fef24008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:16.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.451+0000 7fef2affd700 1 -- 172.21.15.81:0/671681008 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fef34152180 con 0x7fef340b4560 2024-05-08T23:32:16.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.623+0000 7fef3c27d700 1 -- 172.21.15.81:0/671681008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef340672a0 con 0x7fef340b4560 2024-05-08T23:32:16.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.623+0000 7fef2affd700 1 -- 172.21.15.81:0/671681008 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef30050870 con 0x7fef340b4560 2024-05-08T23:32:16.624 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:32:16.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.623+0000 7fef3c27d700 1 -- 172.21.15.81:0/671681008 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fef18066730 msgr2=0x7fef18068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.623+0000 7fef3c27d700 1 --2- 172.21.15.81:0/671681008 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fef18066730 0x7fef18068bf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fef24000c00 tx=0x7fef24008030 comp rx=0 tx=0).stop 2024-05-08T23:32:16.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.623+0000 7fef3c27d700 1 -- 172.21.15.81:0/671681008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef340b4560 msgr2=0x7fef3414fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.623+0000 7fef3c27d700 1 --2- 172.21.15.81:0/671681008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef340b4560 0x7fef3414fbf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fef3000c500 tx=0x7fef3000c810 comp rx=0 tx=0).stop 2024-05-08T23:32:16.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.623+0000 7fef3c27d700 1 -- 172.21.15.81:0/671681008 shutdown_connections 2024-05-08T23:32:16.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.623+0000 7fef3c27d700 1 --2- 172.21.15.81:0/671681008 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fef18066730 0x7fef18068bf0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.623+0000 7fef3c27d700 1 --2- 172.21.15.81:0/671681008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef3415b350 0x7fef341548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.623+0000 7fef3c27d700 1 --2- 172.21.15.81:0/671681008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef3415af70 0x7fef34152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.623+0000 7fef3c27d700 1 --2- 172.21.15.81:0/671681008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef340b4560 0x7fef3414fbf0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.623+0000 7fef3c27d700 1 -- 172.21.15.81:0/671681008 >> 172.21.15.81:0/671681008 conn(0x7fef340cf630 msgr2=0x7fef340b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:16.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.623+0000 7fef3c27d700 1 -- 172.21.15.81:0/671681008 shutdown_connections 2024-05-08T23:32:16.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.623+0000 7fef3c27d700 1 -- 172.21.15.81:0/671681008 wait complete. 2024-05-08T23:32:16.639 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:32:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.755+0000 7f73ce874700 1 Processor -- start 2024-05-08T23:32:16.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.755+0000 7f73ce874700 1 -- start start 2024-05-08T23:32:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce874700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f73c81578e0 0x7f73c8161900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce874700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73c80aed50 0x7f73c80af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce874700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c815d730 0x7f73c815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce874700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c80b3ca0 con 0x7f73c81578e0 2024-05-08T23:32:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce874700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c80af6f0 con 0x7f73c80aed50 2024-05-08T23:32:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce874700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c8161470 con 0x7f73c815d730 2024-05-08T23:32:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73cd872700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f73c81578e0 0x7f73c8161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73cd872700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f73c81578e0 0x7f73c8161900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57780/0 (socket says 172.21.15.81:57780) 2024-05-08T23:32:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73cd872700 1 -- 172.21.15.81:0/3570051608 learned_addr learned my addr 172.21.15.81:0/3570051608 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73cd071700 1 --2- 172.21.15.81:0/3570051608 >> v2:172.21.15.117:3300/0 conn(0x7f73c80aed50 0x7f73c80af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce073700 1 --2- 172.21.15.81:0/3570051608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c815d730 0x7f73c815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73cd872700 1 -- 172.21.15.81:0/3570051608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c815d730 msgr2=0x7f73c815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:16.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73cd872700 1 --2- 172.21.15.81:0/3570051608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c815d730 0x7f73c815dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73cd872700 1 -- 172.21.15.81:0/3570051608 >> v2:172.21.15.117:3300/0 conn(0x7f73c80aed50 msgr2=0x7f73c80af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:16.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73cd872700 1 --2- 172.21.15.81:0/3570051608 >> v2:172.21.15.117:3300/0 conn(0x7f73c80aed50 0x7f73c80af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73cd872700 1 -- 172.21.15.81:0/3570051608 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c815e6f0 con 0x7f73c81578e0 2024-05-08T23:32:16.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73cd071700 1 --2- 172.21.15.81:0/3570051608 >> v2:172.21.15.117:3300/0 conn(0x7f73c80aed50 0x7f73c80af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:16.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73cd872700 1 --2- 172.21.15.81:0/3570051608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c81578e0 0x7f73c8161900 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f73c80cfbf0 tx=0x7f73b8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61f85500657e540 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:16.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73cc870700 1 -- 172.21.15.81:0/3570051608 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b8011830 con 0x7f73c81578e0 2024-05-08T23:32:16.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73cc870700 1 -- 172.21.15.81:0/3570051608 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b8011990 con 0x7f73c81578e0 2024-05-08T23:32:16.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73cc870700 1 -- 172.21.15.81:0/3570051608 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b8011d20 con 0x7f73c81578e0 2024-05-08T23:32:16.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce874700 1 -- 172.21.15.81:0/3570051608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c81578e0 msgr2=0x7f73c8161900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce874700 1 --2- 172.21.15.81:0/3570051608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c81578e0 0x7f73c8161900 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f73c80cfbf0 tx=0x7f73b8009bf0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce874700 1 -- 172.21.15.81:0/3570051608 shutdown_connections 2024-05-08T23:32:16.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce874700 1 --2- 172.21.15.81:0/3570051608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c815d730 0x7f73c815dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce874700 1 --2- 172.21.15.81:0/3570051608 >> v2:172.21.15.117:3300/0 conn(0x7f73c80aed50 0x7f73c80af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce874700 1 --2- 172.21.15.81:0/3570051608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c81578e0 0x7f73c8161900 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce874700 1 -- 172.21.15.81:0/3570051608 >> 172.21.15.81:0/3570051608 conn(0x7f73c80b1b40 msgr2=0x7f73c80b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:16.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce874700 1 -- 172.21.15.81:0/3570051608 shutdown_connections 2024-05-08T23:32:16.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.759+0000 7f73ce874700 1 -- 172.21.15.81:0/3570051608 wait complete. 2024-05-08T23:32:16.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73ce874700 1 Processor -- start 2024-05-08T23:32:16.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73ce874700 1 -- start start 2024-05-08T23:32:16.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73ce874700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c80aed50 0x7f73c813e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73ce874700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73c81578e0 0x7f73c81556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73ce874700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c815d730 0x7f73c8155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73ce874700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c806ea40 con 0x7f73c80aed50 2024-05-08T23:32:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73ce874700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c813ef20 con 0x7f73c81578e0 2024-05-08T23:32:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73ce874700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c806ee60 con 0x7f73c815d730 2024-05-08T23:32:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73cd872700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c80aed50 0x7f73c813e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73cd872700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c80aed50 0x7f73c813e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57790/0 (socket says 172.21.15.81:57790) 2024-05-08T23:32:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73cd872700 1 -- 172.21.15.81:0/390104642 learned_addr learned my addr 172.21.15.81:0/390104642 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73cd071700 1 --2- 172.21.15.81:0/390104642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73c81578e0 0x7f73c81556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73ce073700 1 --2- 172.21.15.81:0/390104642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c815d730 0x7f73c8155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73cd872700 1 -- 172.21.15.81:0/390104642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c815d730 msgr2=0x7f73c8155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73cd872700 1 --2- 172.21.15.81:0/390104642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c815d730 0x7f73c8155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73cd872700 1 -- 172.21.15.81:0/390104642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73c81578e0 msgr2=0x7f73c81556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73cd872700 1 --2- 172.21.15.81:0/390104642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73c81578e0 0x7f73c81556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73cd872700 1 -- 172.21.15.81:0/390104642 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c40094b0 con 0x7f73c80aed50 2024-05-08T23:32:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73cd071700 1 --2- 172.21.15.81:0/390104642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73c81578e0 0x7f73c81556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73cd872700 1 --2- 172.21.15.81:0/390104642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c80aed50 0x7f73c813e9e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f73c806c400 tx=0x7f73b8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:16.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73be7fc700 1 -- 172.21.15.81:0/390104642 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b80044d0 con 0x7f73c80aed50 2024-05-08T23:32:16.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73be7fc700 1 -- 172.21.15.81:0/390104642 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b801f070 con 0x7f73c80aed50 2024-05-08T23:32:16.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73ce874700 1 -- 172.21.15.81:0/390104642 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b8009570 con 0x7f73c80aed50 2024-05-08T23:32:16.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73be7fc700 1 -- 172.21.15.81:0/390104642 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b8003900 con 0x7f73c80aed50 2024-05-08T23:32:16.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.763+0000 7f73ce874700 1 -- 172.21.15.81:0/390104642 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c806e0a0 con 0x7f73c80aed50 2024-05-08T23:32:16.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.767+0000 7f73ce874700 1 -- 172.21.15.81:0/390104642 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c80684b0 con 0x7f73c80aed50 2024-05-08T23:32:16.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.767+0000 7f73be7fc700 1 -- 172.21.15.81:0/390104642 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f73b8003a60 con 0x7f73c80aed50 2024-05-08T23:32:16.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.767+0000 7f73be7fc700 1 --2- 172.21.15.81:0/390104642 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f73ac066730 0x7f73ac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.767+0000 7f73be7fc700 1 -- 172.21.15.81:0/390104642 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f73b8021070 con 0x7f73c80aed50 2024-05-08T23:32:16.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.767+0000 7f73cd071700 1 --2- 172.21.15.81:0/390104642 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f73ac066730 0x7f73ac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.767+0000 7f73cd071700 1 --2- 172.21.15.81:0/390104642 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f73ac066730 0x7f73ac068bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f73c80aff60 tx=0x7f73c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:16.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.775+0000 7f73be7fc700 1 -- 172.21.15.81:0/390104642 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f73b8052920 con 0x7f73c80aed50 2024-05-08T23:32:16.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.939+0000 7f73ce874700 1 -- 172.21.15.81:0/390104642 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f73c80b28a0 con 0x7f73c80aed50 2024-05-08T23:32:16.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.943+0000 7f73be7fc700 1 -- 172.21.15.81:0/390104642 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7f73b8015020 con 0x7f73c80aed50 2024-05-08T23:32:16.944 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:32:16.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.943+0000 7f73ce874700 1 -- 172.21.15.81:0/390104642 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f73ac066730 msgr2=0x7f73ac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.943+0000 7f73ce874700 1 --2- 172.21.15.81:0/390104642 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f73ac066730 0x7f73ac068bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f73c80aff60 tx=0x7f73c4008040 comp rx=0 tx=0).stop 2024-05-08T23:32:16.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.943+0000 7f73ce874700 1 -- 172.21.15.81:0/390104642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c80aed50 msgr2=0x7f73c813e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.943+0000 7f73ce874700 1 --2- 172.21.15.81:0/390104642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c80aed50 0x7f73c813e9e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f73c806c400 tx=0x7f73b8004770 comp rx=0 tx=0).stop 2024-05-08T23:32:16.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.943+0000 7f73ce874700 1 -- 172.21.15.81:0/390104642 shutdown_connections 2024-05-08T23:32:16.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.943+0000 7f73ce874700 1 --2- 172.21.15.81:0/390104642 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f73ac066730 0x7f73ac068bf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.943+0000 7f73ce874700 1 --2- 172.21.15.81:0/390104642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c815d730 0x7f73c8155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.943+0000 7f73ce874700 1 --2- 172.21.15.81:0/390104642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73c81578e0 0x7f73c81556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.943+0000 7f73ce874700 1 --2- 172.21.15.81:0/390104642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73c80aed50 0x7f73c813e9e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.943+0000 7f73ce874700 1 -- 172.21.15.81:0/390104642 >> 172.21.15.81:0/390104642 conn(0x7f73c80b1b40 msgr2=0x7f73c80b07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:16.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.943+0000 7f73ce874700 1 -- 172.21.15.81:0/390104642 shutdown_connections 2024-05-08T23:32:16.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:16.943+0000 7f73ce874700 1 -- 172.21.15.81:0/390104642 wait complete. 2024-05-08T23:32:16.959 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-05-08T23:32:17.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.079+0000 7fae26b7c700 1 Processor -- start 2024-05-08T23:32:17.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.079+0000 7fae26b7c700 1 -- start start 2024-05-08T23:32:17.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.079+0000 7fae26b7c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fae200abda0 0x7fae2015e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.079+0000 7fae26b7c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fae200adf30 0x7fae200b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.079+0000 7fae26b7c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2015a350 con 0x7fae20157b30 2024-05-08T23:32:17.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.079+0000 7fae26b7c700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae2015e980 con 0x7fae200adf30 2024-05-08T23:32:17.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.079+0000 7fae26b7c700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae2015b370 con 0x7fae200abda0 2024-05-08T23:32:17.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.079+0000 7fae2637b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fae200adf30 0x7fae200b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.079+0000 7fae2637b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fae200adf30 0x7fae200b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57794/0 (socket says 172.21.15.81:57794) 2024-05-08T23:32:17.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.079+0000 7fae2637b700 1 -- 172.21.15.81:0/698061835 learned_addr learned my addr 172.21.15.81:0/698061835 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:17.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.079+0000 7fae25379700 1 --2- 172.21.15.81:0/698061835 >> v2:172.21.15.117:3300/0 conn(0x7fae200abda0 0x7fae2015e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.079+0000 7fae2637b700 1 -- 172.21.15.81:0/698061835 >> v2:172.21.15.117:3300/0 conn(0x7fae200abda0 msgr2=0x7fae2015e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.079+0000 7fae2637b700 1 --2- 172.21.15.81:0/698061835 >> v2:172.21.15.117:3300/0 conn(0x7fae200abda0 0x7fae2015e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.079+0000 7fae2637b700 1 -- 172.21.15.81:0/698061835 >> v1:172.21.15.117:6789/0 conn(0x7fae20157b30 legacy=0x7fae200ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.079+0000 7fae2637b700 1 -- 172.21.15.81:0/698061835 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2015a5b0 con 0x7fae200adf30 2024-05-08T23:32:17.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae2637b700 1 --2- 172.21.15.81:0/698061835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae200adf30 0x7fae200b2fe0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fae200b43b0 tx=0x7fae1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387480126f373a0e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae24b78700 1 -- 172.21.15.81:0/698061835 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1800e070 con 0x7fae200adf30 2024-05-08T23:32:17.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae24b78700 1 -- 172.21.15.81:0/698061835 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1800b910 con 0x7fae200adf30 2024-05-08T23:32:17.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae24b78700 1 -- 172.21.15.81:0/698061835 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1800bcc0 con 0x7fae200adf30 2024-05-08T23:32:17.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 -- 172.21.15.81:0/698061835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae200adf30 msgr2=0x7fae200b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 --2- 172.21.15.81:0/698061835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae200adf30 0x7fae200b2fe0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fae200b43b0 tx=0x7fae1800b330 comp rx=0 tx=0).stop 2024-05-08T23:32:17.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 -- 172.21.15.81:0/698061835 shutdown_connections 2024-05-08T23:32:17.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 --2- 172.21.15.81:0/698061835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae200adf30 0x7fae200b2fe0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 --2- 172.21.15.81:0/698061835 >> v2:172.21.15.117:3300/0 conn(0x7fae200abda0 0x7fae2015e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 -- 172.21.15.81:0/698061835 >> 172.21.15.81:0/698061835 conn(0x7fae200b0de0 msgr2=0x7fae200b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 -- 172.21.15.81:0/698061835 shutdown_connections 2024-05-08T23:32:17.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 -- 172.21.15.81:0/698061835 wait complete. 2024-05-08T23:32:17.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 Processor -- start 2024-05-08T23:32:17.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 -- start start 2024-05-08T23:32:17.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae200abda0 0x7fae200b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae200adf30 0x7fae20143430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae20157b30 0x7fae20099cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae200c3f80 con 0x7fae200abda0 2024-05-08T23:32:17.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae200b3a10 con 0x7fae20157b30 2024-05-08T23:32:17.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae26b7c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae20144190 con 0x7fae200adf30 2024-05-08T23:32:17.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae25b7a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae200abda0 0x7fae200b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.083+0000 7fae25b7a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae200abda0 0x7fae200b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57798/0 (socket says 172.21.15.81:57798) 2024-05-08T23:32:17.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae25b7a700 1 -- 172.21.15.81:0/2558035065 learned_addr learned my addr 172.21.15.81:0/2558035065 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:17.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae25379700 1 --2- 172.21.15.81:0/2558035065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae200adf30 0x7fae20143430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae2637b700 1 --2- 172.21.15.81:0/2558035065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae20157b30 0x7fae20099cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae25b7a700 1 -- 172.21.15.81:0/2558035065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae200adf30 msgr2=0x7fae20143430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae25b7a700 1 --2- 172.21.15.81:0/2558035065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae200adf30 0x7fae20143430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae25b7a700 1 -- 172.21.15.81:0/2558035065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae20157b30 msgr2=0x7fae20099cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae25b7a700 1 --2- 172.21.15.81:0/2558035065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae20157b30 0x7fae20099cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae25b7a700 1 -- 172.21.15.81:0/2558035065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae18009b30 con 0x7fae200abda0 2024-05-08T23:32:17.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae25b7a700 1 --2- 172.21.15.81:0/2558035065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae200abda0 0x7fae200b34d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00c500 tx=0x7fae1c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae167fc700 1 -- 172.21.15.81:0/2558035065 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0052d0 con 0x7fae200abda0 2024-05-08T23:32:17.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae167fc700 1 -- 172.21.15.81:0/2558035065 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c005430 con 0x7fae200abda0 2024-05-08T23:32:17.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae167fc700 1 -- 172.21.15.81:0/2558035065 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0055a0 con 0x7fae200abda0 2024-05-08T23:32:17.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae26b7c700 1 -- 172.21.15.81:0/2558035065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae20144470 con 0x7fae200abda0 2024-05-08T23:32:17.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae26b7c700 1 -- 172.21.15.81:0/2558035065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae20144a30 con 0x7fae200abda0 2024-05-08T23:32:17.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae26b7c700 1 -- 172.21.15.81:0/2558035065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae200684b0 con 0x7fae200abda0 2024-05-08T23:32:17.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.087+0000 7fae167fc700 1 -- 172.21.15.81:0/2558035065 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fae1c0038e0 con 0x7fae200abda0 2024-05-08T23:32:17.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.091+0000 7fae167fc700 1 --2- 172.21.15.81:0/2558035065 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fae04066730 0x7fae04068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.091+0000 7fae167fc700 1 -- 172.21.15.81:0/2558035065 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fae1c0836b0 con 0x7fae200abda0 2024-05-08T23:32:17.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.091+0000 7fae25379700 1 --2- 172.21.15.81:0/2558035065 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fae04066730 0x7fae04068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.091+0000 7fae25379700 1 --2- 172.21.15.81:0/2558035065 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fae04066730 0x7fae04068bf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fae20146e50 tx=0x7fae10009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.095+0000 7fae167fc700 1 -- 172.21.15.81:0/2558035065 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fae1c050810 con 0x7fae200abda0 2024-05-08T23:32:17.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.263+0000 7fae26b7c700 1 -- 172.21.15.81:0/2558035065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fae2015d190 con 0x7fae200abda0 2024-05-08T23:32:17.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.267+0000 7fae167fc700 1 -- 172.21.15.81:0/2558035065 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v39) v1 ==== 97+0+11 (secure 0 0 0) 0x7fae1c053e30 con 0x7fae200abda0 2024-05-08T23:32:17.268 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:32:17.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.267+0000 7fae26b7c700 1 -- 172.21.15.81:0/2558035065 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fae04066730 msgr2=0x7fae04068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.267+0000 7fae26b7c700 1 --2- 172.21.15.81:0/2558035065 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fae04066730 0x7fae04068bf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fae20146e50 tx=0x7fae10009240 comp rx=0 tx=0).stop 2024-05-08T23:32:17.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.267+0000 7fae26b7c700 1 -- 172.21.15.81:0/2558035065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae200abda0 msgr2=0x7fae200b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.267+0000 7fae26b7c700 1 --2- 172.21.15.81:0/2558035065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae200abda0 0x7fae200b34d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00c500 tx=0x7fae1c00c810 comp rx=0 tx=0).stop 2024-05-08T23:32:17.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.267+0000 7fae26b7c700 1 -- 172.21.15.81:0/2558035065 shutdown_connections 2024-05-08T23:32:17.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.267+0000 7fae26b7c700 1 --2- 172.21.15.81:0/2558035065 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fae04066730 0x7fae04068bf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.267+0000 7fae26b7c700 1 --2- 172.21.15.81:0/2558035065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae20157b30 0x7fae20099cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.267+0000 7fae26b7c700 1 --2- 172.21.15.81:0/2558035065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae200adf30 0x7fae20143430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.267+0000 7fae26b7c700 1 --2- 172.21.15.81:0/2558035065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae200abda0 0x7fae200b34d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.267+0000 7fae26b7c700 1 -- 172.21.15.81:0/2558035065 >> 172.21.15.81:0/2558035065 conn(0x7fae200b0de0 msgr2=0x7fae2006cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.267+0000 7fae26b7c700 1 -- 172.21.15.81:0/2558035065 shutdown_connections 2024-05-08T23:32:17.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.267+0000 7fae26b7c700 1 -- 172.21.15.81:0/2558035065 wait complete. 2024-05-08T23:32:17.282 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:32:17.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.391+0000 7f0832ff3700 1 Processor -- start 2024-05-08T23:32:17.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.391+0000 7f0832ff3700 1 -- start start 2024-05-08T23:32:17.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.391+0000 7f0832ff3700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f082c0ce510 0x7f082c15a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.391+0000 7f0832ff3700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f082c15aaa0 0x7f082c13d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.391+0000 7f0832ff3700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082c0ce080 con 0x7f082c071ab0 2024-05-08T23:32:17.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.391+0000 7f0832ff3700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f082c0cd7c0 con 0x7f082c15aaa0 2024-05-08T23:32:17.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.391+0000 7f0832ff3700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f082c0cda30 con 0x7f082c0ce510 2024-05-08T23:32:17.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.391+0000 7f08327f2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f082c15aaa0 0x7f082c13d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.391+0000 7f0831ff1700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f082c071ab0 0x7f082c071e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:39720/0 (socket says 172.21.15.81:39720) 2024-05-08T23:32:17.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.391+0000 7f0831ff1700 1 -- 172.21.15.81:0/1208134720 learned_addr learned my addr 172.21.15.81:0/1208134720 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:17.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f08317f0700 1 --2- 172.21.15.81:0/1208134720 >> v2:172.21.15.117:3300/0 conn(0x7f082c0ce510 0x7f082c15a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f08327f2700 1 -- 172.21.15.81:0/1208134720 >> v2:172.21.15.117:3300/0 conn(0x7f082c0ce510 msgr2=0x7f082c15a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f08327f2700 1 --2- 172.21.15.81:0/1208134720 >> v2:172.21.15.117:3300/0 conn(0x7f082c0ce510 0x7f082c15a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f08327f2700 1 -- 172.21.15.81:0/1208134720 >> v1:172.21.15.81:6789/0 conn(0x7f082c071ab0 legacy=0x7f082c071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f08327f2700 1 -- 172.21.15.81:0/1208134720 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c0cdd10 con 0x7f082c15aaa0 2024-05-08T23:32:17.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f08327f2700 1 --2- 172.21.15.81:0/1208134720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f082c15aaa0 0x7f082c13d500 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f082c06bb20 tx=0x7f0824009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d066b75ccf079a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f0830fef700 1 -- 172.21.15.81:0/1208134720 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0824011830 con 0x7f082c15aaa0 2024-05-08T23:32:17.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f0830fef700 1 -- 172.21.15.81:0/1208134720 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0824011990 con 0x7f082c15aaa0 2024-05-08T23:32:17.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f0830fef700 1 -- 172.21.15.81:0/1208134720 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0824011d20 con 0x7f082c15aaa0 2024-05-08T23:32:17.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f0832ff3700 1 -- 172.21.15.81:0/1208134720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f082c15aaa0 msgr2=0x7f082c13d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f0832ff3700 1 --2- 172.21.15.81:0/1208134720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f082c15aaa0 0x7f082c13d500 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f082c06bb20 tx=0x7f0824009b30 comp rx=0 tx=0).stop 2024-05-08T23:32:17.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f0832ff3700 1 -- 172.21.15.81:0/1208134720 shutdown_connections 2024-05-08T23:32:17.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f0832ff3700 1 --2- 172.21.15.81:0/1208134720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f082c15aaa0 0x7f082c13d500 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f0832ff3700 1 --2- 172.21.15.81:0/1208134720 >> v2:172.21.15.117:3300/0 conn(0x7f082c0ce510 0x7f082c15a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f0832ff3700 1 -- 172.21.15.81:0/1208134720 >> 172.21.15.81:0/1208134720 conn(0x7f082c06c330 msgr2=0x7f082c06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f0832ff3700 1 -- 172.21.15.81:0/1208134720 shutdown_connections 2024-05-08T23:32:17.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f0832ff3700 1 -- 172.21.15.81:0/1208134720 wait complete. 2024-05-08T23:32:17.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f0832ff3700 1 Processor -- start 2024-05-08T23:32:17.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.395+0000 7f0832ff3700 1 -- start start 2024-05-08T23:32:17.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f0832ff3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c071ab0 0x7f082c13be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f0832ff3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f082c0ce510 0x7f082c1414c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f0832ff3700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f082c15aaa0 0x7f082c142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f0832ff3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c13c380 con 0x7f082c0ce510 2024-05-08T23:32:17.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f0832ff3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c141030 con 0x7f082c071ab0 2024-05-08T23:32:17.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f0832ff3700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082c143890 con 0x7f082c15aaa0 2024-05-08T23:32:17.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f08317f0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f082c0ce510 0x7f082c1414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f08317f0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f082c0ce510 0x7f082c1414c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57814/0 (socket says 172.21.15.81:57814) 2024-05-08T23:32:17.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f08317f0700 1 -- 172.21.15.81:0/2348896761 learned_addr learned my addr 172.21.15.81:0/2348896761 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:17.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f0831ff1700 1 --2- 172.21.15.81:0/2348896761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c071ab0 0x7f082c13be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f08327f2700 1 --2- 172.21.15.81:0/2348896761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f082c15aaa0 0x7f082c142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f08317f0700 1 -- 172.21.15.81:0/2348896761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f082c15aaa0 msgr2=0x7f082c142d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f08317f0700 1 --2- 172.21.15.81:0/2348896761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f082c15aaa0 0x7f082c142d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f08317f0700 1 -- 172.21.15.81:0/2348896761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c071ab0 msgr2=0x7f082c13be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f08317f0700 1 --2- 172.21.15.81:0/2348896761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c071ab0 0x7f082c13be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f08317f0700 1 -- 172.21.15.81:0/2348896761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08240094b0 con 0x7f082c0ce510 2024-05-08T23:32:17.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f08317f0700 1 --2- 172.21.15.81:0/2348896761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f082c0ce510 0x7f082c1414c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f082800efc0 tx=0x7f082800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f08227fc700 1 -- 172.21.15.81:0/2348896761 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082800cdc0 con 0x7f082c0ce510 2024-05-08T23:32:17.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f0832ff3700 1 -- 172.21.15.81:0/2348896761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082c140700 con 0x7f082c0ce510 2024-05-08T23:32:17.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f08227fc700 1 -- 172.21.15.81:0/2348896761 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082800f040 con 0x7f082c0ce510 2024-05-08T23:32:17.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f08227fc700 1 -- 172.21.15.81:0/2348896761 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08280146a0 con 0x7f082c0ce510 2024-05-08T23:32:17.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f0832ff3700 1 -- 172.21.15.81:0/2348896761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082c140cc0 con 0x7f082c0ce510 2024-05-08T23:32:17.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.399+0000 7f0832ff3700 1 -- 172.21.15.81:0/2348896761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082c1411a0 con 0x7f082c0ce510 2024-05-08T23:32:17.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.403+0000 7f08227fc700 1 -- 172.21.15.81:0/2348896761 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0828014800 con 0x7f082c0ce510 2024-05-08T23:32:17.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.403+0000 7f08227fc700 1 --2- 172.21.15.81:0/2348896761 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0818066760 0x7f0818068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.403+0000 7f08227fc700 1 -- 172.21.15.81:0/2348896761 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f08280852e0 con 0x7f082c0ce510 2024-05-08T23:32:17.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.403+0000 7f0831ff1700 1 --2- 172.21.15.81:0/2348896761 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0818066760 0x7f0818068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.403+0000 7f0831ff1700 1 --2- 172.21.15.81:0/2348896761 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0818066760 0x7f0818068c20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f081c000f60 tx=0x7f081c007f30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.407+0000 7f08227fc700 1 -- 172.21.15.81:0/2348896761 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0828052550 con 0x7f082c0ce510 2024-05-08T23:32:17.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.591+0000 7f0832ff3700 1 -- 172.21.15.81:0/2348896761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f082c0684b0 con 0x7f082c0ce510 2024-05-08T23:32:17.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.591+0000 7f08227fc700 1 -- 172.21.15.81:0/2348896761 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0828055b70 con 0x7f082c0ce510 2024-05-08T23:32:17.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.595+0000 7f0832ff3700 1 -- 172.21.15.81:0/2348896761 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0818066760 msgr2=0x7f0818068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.595+0000 7f0832ff3700 1 --2- 172.21.15.81:0/2348896761 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0818066760 0x7f0818068c20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f081c000f60 tx=0x7f081c007f30 comp rx=0 tx=0).stop 2024-05-08T23:32:17.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.595+0000 7f0832ff3700 1 -- 172.21.15.81:0/2348896761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f082c0ce510 msgr2=0x7f082c1414c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.595+0000 7f0832ff3700 1 --2- 172.21.15.81:0/2348896761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f082c0ce510 0x7f082c1414c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f082800efc0 tx=0x7f082800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.595+0000 7f0832ff3700 1 -- 172.21.15.81:0/2348896761 shutdown_connections 2024-05-08T23:32:17.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.595+0000 7f0832ff3700 1 --2- 172.21.15.81:0/2348896761 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0818066760 0x7f0818068c20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.595+0000 7f0832ff3700 1 --2- 172.21.15.81:0/2348896761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f082c15aaa0 0x7f082c142d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.595+0000 7f0832ff3700 1 --2- 172.21.15.81:0/2348896761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f082c0ce510 0x7f082c1414c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.595+0000 7f0832ff3700 1 --2- 172.21.15.81:0/2348896761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c071ab0 0x7f082c13be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.595+0000 7f0832ff3700 1 -- 172.21.15.81:0/2348896761 >> 172.21.15.81:0/2348896761 conn(0x7f082c06c330 msgr2=0x7f082c13f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.595+0000 7f0832ff3700 1 -- 172.21.15.81:0/2348896761 shutdown_connections 2024-05-08T23:32:17.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:17.595+0000 7f0832ff3700 1 -- 172.21.15.81:0/2348896761 wait complete. 2024-05-08T23:32:17.597 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:32:17.613 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2024-05-08T23:32:17.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:17.613 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:17.627 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[37387]: fuse finished with error 0 and tester_r 0 2024-05-08T23:32:17.629 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:32:23.730 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:32:23.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:23.730 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:32:23.738 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2024-05-08T23:32:23.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.867+0000 7f068bfff700 1 Processor -- start 2024-05-08T23:32:23.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.867+0000 7f068bfff700 1 -- start start 2024-05-08T23:32:23.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.867+0000 7f068bfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f068c157b30 0x7f068c15e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.867+0000 7f068bfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f068c0abda0 0x7f068c0ad820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.867+0000 7f068bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f068c0add60 0x7f068c0b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.867+0000 7f068bfff700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f068c15e980 con 0x7f068c0abda0 2024-05-08T23:32:23.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.867+0000 7f068bfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f068c15b370 con 0x7f068c0add60 2024-05-08T23:32:23.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.867+0000 7f068bfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068c15a350 con 0x7f068c157b30 2024-05-08T23:32:23.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.867+0000 7f068a7fc700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f068c0abda0 0x7f068c0ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068a7fc700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f068c0abda0 0x7f068c0ad820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57826/0 (socket says 172.21.15.81:57826) 2024-05-08T23:32:23.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068a7fc700 1 -- 172.21.15.81:0/1328434536 learned_addr learned my addr 172.21.15.81:0/1328434536 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068b7fe700 1 --2- 172.21.15.81:0/1328434536 >> v2:172.21.15.117:3300/0 conn(0x7f068c0add60 0x7f068c0b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068affd700 1 --2- 172.21.15.81:0/1328434536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f068c157b30 0x7f068c15e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068a7fc700 1 -- 172.21.15.81:0/1328434536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f068c157b30 msgr2=0x7f068c15e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068a7fc700 1 --2- 172.21.15.81:0/1328434536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f068c157b30 0x7f068c15e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068a7fc700 1 -- 172.21.15.81:0/1328434536 >> v2:172.21.15.117:3300/0 conn(0x7f068c0add60 msgr2=0x7f068c0b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068a7fc700 1 --2- 172.21.15.81:0/1328434536 >> v2:172.21.15.117:3300/0 conn(0x7f068c0add60 0x7f068c0b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068a7fc700 1 -- 172.21.15.81:0/1328434536 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c15a5b0 con 0x7f068c0abda0 2024-05-08T23:32:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068a7fc700 1 --2- 172.21.15.81:0/1328434536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f068c0abda0 0x7f068c0ad820 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f068c06fec0 tx=0x7f067c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b988106e442a5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f0689ffb700 1 -- 172.21.15.81:0/1328434536 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c011830 con 0x7f068c0abda0 2024-05-08T23:32:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f0689ffb700 1 -- 172.21.15.81:0/1328434536 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c011990 con 0x7f068c0abda0 2024-05-08T23:32:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f0689ffb700 1 -- 172.21.15.81:0/1328434536 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c011d40 con 0x7f068c0abda0 2024-05-08T23:32:23.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068bfff700 1 -- 172.21.15.81:0/1328434536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f068c0abda0 msgr2=0x7f068c0ad820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068bfff700 1 --2- 172.21.15.81:0/1328434536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f068c0abda0 0x7f068c0ad820 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f068c06fec0 tx=0x7f067c009c20 comp rx=0 tx=0).stop 2024-05-08T23:32:23.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068bfff700 1 -- 172.21.15.81:0/1328434536 shutdown_connections 2024-05-08T23:32:23.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068bfff700 1 --2- 172.21.15.81:0/1328434536 >> v2:172.21.15.117:3300/0 conn(0x7f068c0add60 0x7f068c0b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068bfff700 1 --2- 172.21.15.81:0/1328434536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f068c0abda0 0x7f068c0ad820 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068bfff700 1 --2- 172.21.15.81:0/1328434536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f068c157b30 0x7f068c15e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068bfff700 1 -- 172.21.15.81:0/1328434536 >> 172.21.15.81:0/1328434536 conn(0x7f068c0b0de0 msgr2=0x7f068c0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:23.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068bfff700 1 -- 172.21.15.81:0/1328434536 shutdown_connections 2024-05-08T23:32:23.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068bfff700 1 -- 172.21.15.81:0/1328434536 wait complete. 2024-05-08T23:32:23.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068bfff700 1 Processor -- start 2024-05-08T23:32:23.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.871+0000 7f068bfff700 1 -- start start 2024-05-08T23:32:23.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068bfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f068c0abda0 0x7f068c149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f068c0add60 0x7f068c162ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f068c157b30 0x7f068c163410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068bfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c0afc50 con 0x7f068c0add60 2024-05-08T23:32:23.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068bfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c0aff30 con 0x7f068c157b30 2024-05-08T23:32:23.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068bfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068c0b0210 con 0x7f068c0abda0 2024-05-08T23:32:23.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f068c0add60 0x7f068c162ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f068c0add60 0x7f068c162ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57828/0 (socket says 172.21.15.81:57828) 2024-05-08T23:32:23.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068a7fc700 1 -- 172.21.15.81:0/167673003 learned_addr learned my addr 172.21.15.81:0/167673003 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:23.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068affd700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f068c0abda0 0x7f068c149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068b7fe700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f068c157b30 0x7f068c163410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068a7fc700 1 -- 172.21.15.81:0/167673003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f068c0abda0 msgr2=0x7f068c149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068a7fc700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f068c0abda0 0x7f068c149f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068a7fc700 1 -- 172.21.15.81:0/167673003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f068c157b30 msgr2=0x7f068c163410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068a7fc700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f068c157b30 0x7f068c163410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068a7fc700 1 -- 172.21.15.81:0/167673003 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c009570 con 0x7f068c0add60 2024-05-08T23:32:23.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068affd700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f068c0abda0 0x7f068c149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:23.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068b7fe700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f068c157b30 0x7f068c163410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:23.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068a7fc700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f068c0add60 0x7f068c162ed0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f067c000c00 tx=0x7f067c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:23.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f06737fe700 1 -- 172.21.15.81:0/167673003 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c004240 con 0x7f068c0add60 2024-05-08T23:32:23.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f06737fe700 1 -- 172.21.15.81:0/167673003 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c01f070 con 0x7f068c0add60 2024-05-08T23:32:23.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068bfff700 1 -- 172.21.15.81:0/167673003 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068c0b04f0 con 0x7f068c0add60 2024-05-08T23:32:23.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f06737fe700 1 -- 172.21.15.81:0/167673003 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c0045c0 con 0x7f068c0add60 2024-05-08T23:32:23.880 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.875+0000 7f068bfff700 1 -- 172.21.15.81:0/167673003 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068c0aeee0 con 0x7f068c0add60 2024-05-08T23:32:23.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.879+0000 7f068bfff700 1 -- 172.21.15.81:0/167673003 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f068c1580a0 con 0x7f068c0add60 2024-05-08T23:32:23.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.879+0000 7f06737fe700 1 -- 172.21.15.81:0/167673003 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f067c021030 con 0x7f068c0add60 2024-05-08T23:32:23.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.879+0000 7f06737fe700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0678066730 0x7f0678068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.879+0000 7f06737fe700 1 -- 172.21.15.81:0/167673003 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f067c0853a0 con 0x7f068c0add60 2024-05-08T23:32:23.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.879+0000 7f068affd700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0678066730 0x7f0678068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.879+0000 7f068affd700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0678066730 0x7f0678068bf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0680009f40 tx=0x7f0680009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:23.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:23.887+0000 7f06737fe700 1 -- 172.21.15.81:0/167673003 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f068c1580a0 con 0x7f068c0add60 2024-05-08T23:32:24.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.119+0000 7f068bfff700 1 -- 172.21.15.81:0/167673003 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f068c1580a0 con 0x7f068c0add60 2024-05-08T23:32:24.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.127+0000 7f06737fe700 1 -- 172.21.15.81:0/167673003 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7f068c1580a0 con 0x7f068c0add60 2024-05-08T23:32:24.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.131+0000 7f068bfff700 1 -- 172.21.15.81:0/167673003 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0678066730 msgr2=0x7f0678068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.131+0000 7f068bfff700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0678066730 0x7f0678068bf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0680009f40 tx=0x7f0680009240 comp rx=0 tx=0).stop 2024-05-08T23:32:24.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.131+0000 7f068bfff700 1 -- 172.21.15.81:0/167673003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f068c0add60 msgr2=0x7f068c162ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.131+0000 7f068bfff700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f068c0add60 0x7f068c162ed0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f067c000c00 tx=0x7f067c004770 comp rx=0 tx=0).stop 2024-05-08T23:32:24.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.131+0000 7f068bfff700 1 -- 172.21.15.81:0/167673003 shutdown_connections 2024-05-08T23:32:24.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.131+0000 7f068bfff700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0678066730 0x7f0678068bf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.131+0000 7f068bfff700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f068c157b30 0x7f068c163410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.131+0000 7f068bfff700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f068c0add60 0x7f068c162ed0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.131+0000 7f068bfff700 1 --2- 172.21.15.81:0/167673003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f068c0abda0 0x7f068c149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.131+0000 7f068bfff700 1 -- 172.21.15.81:0/167673003 >> 172.21.15.81:0/167673003 conn(0x7f068c0b0de0 msgr2=0x7f068c070170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:24.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.131+0000 7f068bfff700 1 -- 172.21.15.81:0/167673003 shutdown_connections 2024-05-08T23:32:24.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.131+0000 7f068bfff700 1 -- 172.21.15.81:0/167673003 wait complete. 2024-05-08T23:32:24.133 INFO:teuthology.orchestra.run.smithi081.stderr:updated caps for client.0 2024-05-08T23:32:24.149 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:32:24.149 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:32:24.165 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:32:24.187 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:32:24.187 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:32:24.187 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:24.194 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346332k,nr_inodes=4086583,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14716 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T23:32:24.194 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:32:24.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:24.194 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:32:24.200 INFO:teuthology.orchestra.run.smithi081.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-08T23:32:24.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:32:24.201 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:32:24.201 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:24.208 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346332k,nr_inodes=4086583,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17609 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T23:32:24.208 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:32:24.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:24.208 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T23:32:24.255 INFO:teuthology.orchestra.run.smithi117.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-08T23:32:24.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:32:24.256 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:32:24.257 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:24.264 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346332k,nr_inodes=4086583,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14716 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T23:32:24.264 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:32:24.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:24.264 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-08T23:32:24.314 INFO:teuthology.orchestra.run.smithi081.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-08T23:32:24.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:32:24.315 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:32:24.315 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:24.324 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346332k,nr_inodes=4086583,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17609 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T23:32:24.324 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:32:24.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:24.324 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-08T23:32:24.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:32:24.330 INFO:teuthology.orchestra.run.smithi117.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-08T23:32:24.330 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:24.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.451+0000 7f479eb6e700 1 Processor -- start 2024-05-08T23:32:24.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.451+0000 7f479eb6e700 1 -- start start 2024-05-08T23:32:24.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479eb6e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f47900e3370 0x7f47900538d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479eb6e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4790055150 0x7f4790006ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479eb6e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4790053e10 con 0x7f479003b8c0 2024-05-08T23:32:24.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479eb6e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47900540b0 con 0x7f47900e3370 2024-05-08T23:32:24.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479eb6e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4790054390 con 0x7f4790055150 2024-05-08T23:32:24.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479db6c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f47900e3370 0x7f47900538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479db6c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f47900e3370 0x7f47900538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57844/0 (socket says 172.21.15.81:57844) 2024-05-08T23:32:24.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479db6c700 1 -- 172.21.15.81:0/3122239505 learned_addr learned my addr 172.21.15.81:0/3122239505 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:24.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479d36b700 1 --2- 172.21.15.81:0/3122239505 >> v2:172.21.15.117:3300/0 conn(0x7f4790055150 0x7f4790006ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479cb6a700 1 -- 172.21.15.81:0/3122239505 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892898229 0 0) 0x7f4790053e10 con 0x7f479003b8c0 2024-05-08T23:32:24.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479cb6a700 1 -- 172.21.15.81:0/3122239505 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47840035d0 con 0x7f479003b8c0 2024-05-08T23:32:24.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479db6c700 1 -- 172.21.15.81:0/3122239505 >> v2:172.21.15.117:3300/0 conn(0x7f4790055150 msgr2=0x7f4790006ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:24.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479db6c700 1 --2- 172.21.15.81:0/3122239505 >> v2:172.21.15.117:3300/0 conn(0x7f4790055150 0x7f4790006ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479db6c700 1 -- 172.21.15.81:0/3122239505 >> v1:172.21.15.117:6789/0 conn(0x7f479003b8c0 legacy=0x7f47900e6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479db6c700 1 -- 172.21.15.81:0/3122239505 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4790054670 con 0x7f47900e3370 2024-05-08T23:32:24.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479d36b700 1 --2- 172.21.15.81:0/3122239505 >> v2:172.21.15.117:3300/0 conn(0x7f4790055150 0x7f4790006ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:24.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479db6c700 1 --2- 172.21.15.81:0/3122239505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47900e3370 0x7f47900538d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4790056d80 tx=0x7f4788009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5980307377da7e4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479cb6a700 1 -- 172.21.15.81:0/3122239505 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788011830 con 0x7f47900e3370 2024-05-08T23:32:24.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479cb6a700 1 -- 172.21.15.81:0/3122239505 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4788011990 con 0x7f47900e3370 2024-05-08T23:32:24.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479cb6a700 1 -- 172.21.15.81:0/3122239505 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788011d00 con 0x7f47900e3370 2024-05-08T23:32:24.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479eb6e700 1 -- 172.21.15.81:0/3122239505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47900e3370 msgr2=0x7f47900538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479eb6e700 1 --2- 172.21.15.81:0/3122239505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47900e3370 0x7f47900538d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4790056d80 tx=0x7f4788009bf0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479eb6e700 1 -- 172.21.15.81:0/3122239505 shutdown_connections 2024-05-08T23:32:24.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479eb6e700 1 --2- 172.21.15.81:0/3122239505 >> v2:172.21.15.117:3300/0 conn(0x7f4790055150 0x7f4790006ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479eb6e700 1 --2- 172.21.15.81:0/3122239505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47900e3370 0x7f47900538d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479eb6e700 1 -- 172.21.15.81:0/3122239505 >> 172.21.15.81:0/3122239505 conn(0x7f4790055d00 msgr2=0x7f4790056110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:24.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479eb6e700 1 -- 172.21.15.81:0/3122239505 shutdown_connections 2024-05-08T23:32:24.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.455+0000 7f479eb6e700 1 -- 172.21.15.81:0/3122239505 wait complete. 2024-05-08T23:32:24.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479eb6e700 1 Processor -- start 2024-05-08T23:32:24.461 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479eb6e700 1 -- start start 2024-05-08T23:32:24.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479eb6e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f479003b8c0 0x7f47900d9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479eb6e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4790055150 0x7f47900e5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479eb6e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47900e3370 0x7f47900e59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479eb6e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47900df740 con 0x7f47900e3370 2024-05-08T23:32:24.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479eb6e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4790045270 con 0x7f4790055150 2024-05-08T23:32:24.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479eb6e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4790045550 con 0x7f479003b8c0 2024-05-08T23:32:24.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479e36d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47900e3370 0x7f47900e59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479db6c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f479003b8c0 0x7f47900d9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479d36b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4790055150 0x7f47900e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479e36d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47900e3370 0x7f47900e59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57854/0 (socket says 172.21.15.81:57854) 2024-05-08T23:32:24.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479e36d700 1 -- 172.21.15.81:0/3365346420 learned_addr learned my addr 172.21.15.81:0/3365346420 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:24.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479e36d700 1 -- 172.21.15.81:0/3365346420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f479003b8c0 msgr2=0x7f47900d9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479e36d700 1 --2- 172.21.15.81:0/3365346420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f479003b8c0 0x7f47900d9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.459+0000 7f479e36d700 1 -- 172.21.15.81:0/3365346420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4790055150 msgr2=0x7f47900e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.463+0000 7f479e36d700 1 --2- 172.21.15.81:0/3365346420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4790055150 0x7f47900e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.463+0000 7f479e36d700 1 -- 172.21.15.81:0/3365346420 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47940094b0 con 0x7f47900e3370 2024-05-08T23:32:24.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.463+0000 7f479e36d700 1 --2- 172.21.15.81:0/3365346420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47900e3370 0x7f47900e59a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f47980692d0 tx=0x7f47980677b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.463+0000 7f478e7fc700 1 -- 172.21.15.81:0/3365346420 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798072340 con 0x7f47900e3370 2024-05-08T23:32:24.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.463+0000 7f478e7fc700 1 -- 172.21.15.81:0/3365346420 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47980724e0 con 0x7f47900e3370 2024-05-08T23:32:24.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.463+0000 7f479eb6e700 1 -- 172.21.15.81:0/3365346420 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4788009570 con 0x7f47900e3370 2024-05-08T23:32:24.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.463+0000 7f478e7fc700 1 -- 172.21.15.81:0/3365346420 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798072890 con 0x7f47900e3370 2024-05-08T23:32:24.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.463+0000 7f479eb6e700 1 -- 172.21.15.81:0/3365346420 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4790045b10 con 0x7f47900e3370 2024-05-08T23:32:24.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.463+0000 7f479eb6e700 1 -- 172.21.15.81:0/3365346420 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4790006950 con 0x7f47900e3370 2024-05-08T23:32:24.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.463+0000 7f478e7fc700 1 -- 172.21.15.81:0/3365346420 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f479806d480 con 0x7f47900e3370 2024-05-08T23:32:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.467+0000 7f478e7fc700 1 --2- 172.21.15.81:0/3365346420 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f477c066730 0x7f477c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.467+0000 7f478e7fc700 1 -- 172.21.15.81:0/3365346420 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f47980fe1e0 con 0x7f47900e3370 2024-05-08T23:32:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.467+0000 7f479db6c700 1 --2- 172.21.15.81:0/3365346420 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f477c066730 0x7f477c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.467+0000 7f479db6c700 1 --2- 172.21.15.81:0/3365346420 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f477c066730 0x7f477c068bf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4788000c00 tx=0x7f4788008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.471+0000 7f478e7fc700 1 -- 172.21.15.81:0/3365346420 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f47980cb3c0 con 0x7f47900e3370 2024-05-08T23:32:24.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.679+0000 7f479eb6e700 1 -- 172.21.15.81:0/3365346420 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4790005890 con 0x7f47900e3370 2024-05-08T23:32:24.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.679+0000 7f478e7fc700 1 -- 172.21.15.81:0/3365346420 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f47980ce9e0 con 0x7f47900e3370 2024-05-08T23:32:24.681 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:24.682 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1243037934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1243037934},{"type":"v1","addr":"172.21.15.81:6835","nonce":1243037934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3124115182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3124115182},{"type":"v1","addr":"172.21.15.81:6837","nonce":3124115182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1423349691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":1423349691},{"type":"v1","addr":"172.21.15.117:6834","nonce":1423349691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:31:37.889298+0000","modified":"2024-05-08T23:31:47.307677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14852":{"gid":14852,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/3091184157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3091184157},{"type":"v1","addr":"172.21.15.117:6835","nonce":3091184157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:32:24.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.683+0000 7f479eb6e700 1 -- 172.21.15.81:0/3365346420 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f477c066730 msgr2=0x7f477c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.683+0000 7f479eb6e700 1 --2- 172.21.15.81:0/3365346420 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f477c066730 0x7f477c068bf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4788000c00 tx=0x7f4788008030 comp rx=0 tx=0).stop 2024-05-08T23:32:24.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.683+0000 7f479eb6e700 1 -- 172.21.15.81:0/3365346420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47900e3370 msgr2=0x7f47900e59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.683+0000 7f479eb6e700 1 --2- 172.21.15.81:0/3365346420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47900e3370 0x7f47900e59a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f47980692d0 tx=0x7f47980677b0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.683+0000 7f479eb6e700 1 -- 172.21.15.81:0/3365346420 shutdown_connections 2024-05-08T23:32:24.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.683+0000 7f479eb6e700 1 --2- 172.21.15.81:0/3365346420 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f477c066730 0x7f477c068bf0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.683+0000 7f479eb6e700 1 --2- 172.21.15.81:0/3365346420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47900e3370 0x7f47900e59a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.683+0000 7f479eb6e700 1 --2- 172.21.15.81:0/3365346420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4790055150 0x7f47900e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.683+0000 7f479eb6e700 1 --2- 172.21.15.81:0/3365346420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f479003b8c0 0x7f47900d9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.683+0000 7f479eb6e700 1 -- 172.21.15.81:0/3365346420 >> 172.21.15.81:0/3365346420 conn(0x7f4790055d00 msgr2=0x7f479003a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:24.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.683+0000 7f479eb6e700 1 -- 172.21.15.81:0/3365346420 shutdown_connections 2024-05-08T23:32:24.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.683+0000 7f479eb6e700 1 -- 172.21.15.81:0/3365346420 wait complete. 2024-05-08T23:32:24.686 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 15 2024-05-08T23:32:24.696 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:24.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf97203700 1 Processor -- start 2024-05-08T23:32:24.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf97203700 1 -- start start 2024-05-08T23:32:24.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf97203700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faf90071e70 0x7faf901624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf97203700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf90158960 con 0x7faf900b3c20 2024-05-08T23:32:24.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf97203700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf900ae990 con 0x7faf90070110 2024-05-08T23:32:24.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf97203700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf900b1f60 con 0x7faf90071e70 2024-05-08T23:32:24.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf96201700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7faf900b3c20 0x7faf900b4000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:39732/0 (socket says 172.21.15.81:39732) 2024-05-08T23:32:24.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf96201700 1 -- 172.21.15.81:0/1740984775 learned_addr learned my addr 172.21.15.81:0/1740984775 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:24.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf95a00700 1 --2- 172.21.15.81:0/1740984775 >> v2:172.21.15.117:3300/0 conn(0x7faf90071e70 0x7faf901624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 -- 172.21.15.81:0/1740984775 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3290001214 0 0) 0x7faf90158960 con 0x7faf900b3c20 2024-05-08T23:32:24.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 -- 172.21.15.81:0/1740984775 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf780035d0 con 0x7faf900b3c20 2024-05-08T23:32:24.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 -- 172.21.15.81:0/1740984775 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1902059088 0 0) 0x7faf780035d0 con 0x7faf900b3c20 2024-05-08T23:32:24.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 -- 172.21.15.81:0/1740984775 >> v2:172.21.15.117:3300/0 conn(0x7faf90071e70 msgr2=0x7faf901624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:24.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 --2- 172.21.15.81:0/1740984775 >> v2:172.21.15.117:3300/0 conn(0x7faf90071e70 0x7faf901624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 -- 172.21.15.81:0/1740984775 >> v1:172.21.15.117:6789/0 conn(0x7faf90070110 legacy=0x7faf900705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 -- 172.21.15.81:0/1740984775 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf900b2170 con 0x7faf900b3c20 2024-05-08T23:32:24.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 -- 172.21.15.81:0/1740984775 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7faf8c001670 con 0x7faf900b3c20 2024-05-08T23:32:24.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 -- 172.21.15.81:0/1740984775 >> v1:172.21.15.81:6789/0 conn(0x7faf900b3c20 legacy=0x7faf900b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 --2- 172.21.15.81:0/1740984775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf78003a20 0x7faf78003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 --2- 172.21.15.81:0/1740984775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf78004870 0x7faf78004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 --2- 172.21.15.81:0/1740984775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf78005230 0x7faf7800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 -- 172.21.15.81:0/1740984775 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf900b1f60 con 0x7faf78003a20 2024-05-08T23:32:24.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 -- 172.21.15.81:0/1740984775 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7800c1d0 con 0x7faf78005230 2024-05-08T23:32:24.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 -- 172.21.15.81:0/1740984775 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7800c390 con 0x7faf78004870 2024-05-08T23:32:24.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf96201700 1 --2- 172.21.15.81:0/1740984775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf78003a20 0x7faf78003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf95a00700 1 --2- 172.21.15.81:0/1740984775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf78004870 0x7faf78004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf96201700 1 -- 172.21.15.81:0/1740984775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf78004870 msgr2=0x7faf78004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:24.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf96a02700 1 --2- 172.21.15.81:0/1740984775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf78005230 0x7faf7800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf96201700 1 --2- 172.21.15.81:0/1740984775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf78004870 0x7faf78004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf96201700 1 -- 172.21.15.81:0/1740984775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf78005230 msgr2=0x7faf7800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:24.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf96201700 1 --2- 172.21.15.81:0/1740984775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf78005230 0x7faf7800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf96201700 1 -- 172.21.15.81:0/1740984775 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf780035d0 con 0x7faf78003a20 2024-05-08T23:32:24.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf95a00700 1 --2- 172.21.15.81:0/1740984775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf78004870 0x7faf78004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:24.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf96a02700 1 --2- 172.21.15.81:0/1740984775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf78005230 0x7faf7800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:24.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf96a02700 1 -- 172.21.15.81:0/1740984775 reap_dead start 2024-05-08T23:32:24.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf96201700 1 --2- 172.21.15.81:0/1740984775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf78003a20 0x7faf78003e40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00c320 tx=0x7faf780058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55abb565a1b3850 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 -- 172.21.15.81:0/1740984775 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c007c30 con 0x7faf78003a20 2024-05-08T23:32:24.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 -- 172.21.15.81:0/1740984775 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c00ce80 con 0x7faf78003a20 2024-05-08T23:32:24.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.807+0000 7faf951ff700 1 -- 172.21.15.81:0/1740984775 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c005bc0 con 0x7faf78003a20 2024-05-08T23:32:24.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf97203700 1 -- 172.21.15.81:0/1740984775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf78003a20 msgr2=0x7faf78003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf97203700 1 --2- 172.21.15.81:0/1740984775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf78003a20 0x7faf78003e40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00c320 tx=0x7faf780058a0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf97203700 1 -- 172.21.15.81:0/1740984775 shutdown_connections 2024-05-08T23:32:24.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf97203700 1 --2- 172.21.15.81:0/1740984775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf78003a20 0x7faf78003e40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf97203700 1 -- 172.21.15.81:0/1740984775 >> 172.21.15.81:0/1740984775 conn(0x7faf9006c330 msgr2=0x7faf9006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:24.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf97203700 1 -- 172.21.15.81:0/1740984775 shutdown_connections 2024-05-08T23:32:24.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf97203700 1 -- 172.21.15.81:0/1740984775 wait complete. 2024-05-08T23:32:24.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf97203700 1 Processor -- start 2024-05-08T23:32:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf97203700 1 -- start start 2024-05-08T23:32:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf97203700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf78003a20 0x7faf9014f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf97203700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf901f6ac0 0x7faf901538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf97203700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf901533a0 0x7faf90153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf97203700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9014d8b0 con 0x7faf901f6ac0 2024-05-08T23:32:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf97203700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf90151820 con 0x7faf901533a0 2024-05-08T23:32:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf97203700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf90150030 con 0x7faf78003a20 2024-05-08T23:32:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf95a00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf901f6ac0 0x7faf901538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf95a00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf901f6ac0 0x7faf901538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57868/0 (socket says 172.21.15.81:57868) 2024-05-08T23:32:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf95a00700 1 -- 172.21.15.81:0/728666523 learned_addr learned my addr 172.21.15.81:0/728666523 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf96201700 1 --2- 172.21.15.81:0/728666523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf78003a20 0x7faf9014f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf96a02700 1 --2- 172.21.15.81:0/728666523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf901533a0 0x7faf90153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf95a00700 1 -- 172.21.15.81:0/728666523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf78003a20 msgr2=0x7faf9014f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf95a00700 1 --2- 172.21.15.81:0/728666523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf78003a20 0x7faf9014f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf95a00700 1 -- 172.21.15.81:0/728666523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf901533a0 msgr2=0x7faf90153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf95a00700 1 --2- 172.21.15.81:0/728666523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf901533a0 0x7faf90153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf95a00700 1 -- 172.21.15.81:0/728666523 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf90150310 con 0x7faf901f6ac0 2024-05-08T23:32:24.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf95a00700 1 --2- 172.21.15.81:0/728666523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf901f6ac0 0x7faf901538f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faf8800f3d0 tx=0x7faf8800f6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf86ffd700 1 -- 172.21.15.81:0/728666523 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8801a070 con 0x7faf901f6ac0 2024-05-08T23:32:24.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.811+0000 7faf86ffd700 1 -- 172.21.15.81:0/728666523 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8800fe50 con 0x7faf901f6ac0 2024-05-08T23:32:24.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.815+0000 7faf97203700 1 -- 172.21.15.81:0/728666523 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf901514b0 con 0x7faf901f6ac0 2024-05-08T23:32:24.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.815+0000 7faf97203700 1 -- 172.21.15.81:0/728666523 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf90163b30 con 0x7faf901f6ac0 2024-05-08T23:32:24.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.815+0000 7faf86ffd700 1 -- 172.21.15.81:0/728666523 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf88003c40 con 0x7faf901f6ac0 2024-05-08T23:32:24.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.815+0000 7faf97203700 1 -- 172.21.15.81:0/728666523 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf90151310 con 0x7faf901f6ac0 2024-05-08T23:32:24.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.815+0000 7faf86ffd700 1 -- 172.21.15.81:0/728666523 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faf88004020 con 0x7faf901f6ac0 2024-05-08T23:32:24.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.815+0000 7faf86ffd700 1 --2- 172.21.15.81:0/728666523 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faf7c066760 0x7faf7c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.815+0000 7faf86ffd700 1 -- 172.21.15.81:0/728666523 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7faf88084aa0 con 0x7faf901f6ac0 2024-05-08T23:32:24.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.815+0000 7faf96201700 1 --2- 172.21.15.81:0/728666523 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faf7c066760 0x7faf7c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.815+0000 7faf96201700 1 --2- 172.21.15.81:0/728666523 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faf7c066760 0x7faf7c068c20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faf8c000c00 tx=0x7faf8c006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.825 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.823+0000 7faf86ffd700 1 -- 172.21.15.81:0/728666523 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faf90151310 con 0x7faf901f6ac0 2024-05-08T23:32:25.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:24.999+0000 7faf97203700 1 -- 172.21.15.81:0/728666523 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf90151310 con 0x7faf901f6ac0 2024-05-08T23:32:25.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.003+0000 7faf86ffd700 1 -- 172.21.15.81:0/728666523 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+17880 (secure 0 0 0) 0x7faf90151310 con 0x7faf901f6ac0 2024-05-08T23:32:25.005 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:25.005 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":39,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:48.310750+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:31:35.620423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:31:36.878524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-05-08T23:31:46.337484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:25.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.003+0000 7faf97203700 1 -- 172.21.15.81:0/728666523 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faf7c066760 msgr2=0x7faf7c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.003+0000 7faf97203700 1 --2- 172.21.15.81:0/728666523 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faf7c066760 0x7faf7c068c20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faf8c000c00 tx=0x7faf8c006040 comp rx=0 tx=0).stop 2024-05-08T23:32:25.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.003+0000 7faf97203700 1 -- 172.21.15.81:0/728666523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf901f6ac0 msgr2=0x7faf901538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.003+0000 7faf97203700 1 --2- 172.21.15.81:0/728666523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf901f6ac0 0x7faf901538f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faf8800f3d0 tx=0x7faf8800f6e0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.003+0000 7faf97203700 1 -- 172.21.15.81:0/728666523 shutdown_connections 2024-05-08T23:32:25.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.003+0000 7faf97203700 1 --2- 172.21.15.81:0/728666523 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faf7c066760 0x7faf7c068c20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.003+0000 7faf97203700 1 --2- 172.21.15.81:0/728666523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf901533a0 0x7faf90153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.003+0000 7faf97203700 1 --2- 172.21.15.81:0/728666523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf901f6ac0 0x7faf901538f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.003+0000 7faf97203700 1 --2- 172.21.15.81:0/728666523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf78003a20 0x7faf9014f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.003+0000 7faf97203700 1 -- 172.21.15.81:0/728666523 >> 172.21.15.81:0/728666523 conn(0x7faf9006c330 msgr2=0x7faf9006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:25.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.003+0000 7faf97203700 1 -- 172.21.15.81:0/728666523 shutdown_connections 2024-05-08T23:32:25.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.003+0000 7faf97203700 1 -- 172.21.15.81:0/728666523 wait complete. 2024-05-08T23:32:25.019 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:31:35.620423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:31:36.878524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-05-08T23:31:46.337484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:32:25.019 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:25.142 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.139+0000 7f1a8ed19700 1 Processor -- start 2024-05-08T23:32:25.142 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.139+0000 7f1a8ed19700 1 -- start start 2024-05-08T23:32:25.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8ed19700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a800e77f0 0x7f1a80039bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8ed19700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1a800e7c80 0x7f1a80054950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8ed19700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a8003b920 con 0x7f1a800e3700 2024-05-08T23:32:25.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8ed19700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a8003bc00 con 0x7f1a800e7c80 2024-05-08T23:32:25.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8ed19700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8003bee0 con 0x7f1a800e77f0 2024-05-08T23:32:25.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8e518700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1a800e7c80 0x7f1a80054950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8e518700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1a800e7c80 0x7f1a80054950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57870/0 (socket says 172.21.15.81:57870) 2024-05-08T23:32:25.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8e518700 1 -- 172.21.15.81:0/594860227 learned_addr learned my addr 172.21.15.81:0/594860227 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:25.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8d516700 1 --2- 172.21.15.81:0/594860227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a800e77f0 0x7f1a80039bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8e518700 1 -- 172.21.15.81:0/594860227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a800e77f0 msgr2=0x7f1a80039bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:25.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8e518700 1 --2- 172.21.15.81:0/594860227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a800e77f0 0x7f1a80039bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8e518700 1 -- 172.21.15.81:0/594860227 >> v1:172.21.15.81:6789/0 conn(0x7f1a800e3700 legacy=0x7f1a800e70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8e518700 1 -- 172.21.15.81:0/594860227 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8003c1c0 con 0x7f1a800e7c80 2024-05-08T23:32:25.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8cd15700 1 -- 172.21.15.81:0/594860227 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800769316 0 0) 0x7f1a8003b920 con 0x7f1a800e3700 2024-05-08T23:32:25.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8e518700 1 --2- 172.21.15.81:0/594860227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a800e7c80 0x7f1a80054950 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1a880698a0 tx=0x7f1a88068970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4edab88ab7dd9db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8cd15700 1 -- 172.21.15.81:0/594860227 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a880679b0 con 0x7f1a800e7c80 2024-05-08T23:32:25.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8cd15700 1 -- 172.21.15.81:0/594860227 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a88067b10 con 0x7f1a800e7c80 2024-05-08T23:32:25.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8cd15700 1 -- 172.21.15.81:0/594860227 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88072130 con 0x7f1a800e7c80 2024-05-08T23:32:25.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8ed19700 1 -- 172.21.15.81:0/594860227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a800e7c80 msgr2=0x7f1a80054950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8ed19700 1 --2- 172.21.15.81:0/594860227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a800e7c80 0x7f1a80054950 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1a880698a0 tx=0x7f1a88068970 comp rx=0 tx=0).stop 2024-05-08T23:32:25.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8ed19700 1 -- 172.21.15.81:0/594860227 shutdown_connections 2024-05-08T23:32:25.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8ed19700 1 --2- 172.21.15.81:0/594860227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a800e7c80 0x7f1a80054950 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8ed19700 1 --2- 172.21.15.81:0/594860227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a800e77f0 0x7f1a80039bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8ed19700 1 -- 172.21.15.81:0/594860227 >> 172.21.15.81:0/594860227 conn(0x7f1a80056d80 msgr2=0x7f1a80057190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:25.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8ed19700 1 -- 172.21.15.81:0/594860227 shutdown_connections 2024-05-08T23:32:25.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.143+0000 7f1a8ed19700 1 -- 172.21.15.81:0/594860227 wait complete. 2024-05-08T23:32:25.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8ed19700 1 Processor -- start 2024-05-08T23:32:25.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8ed19700 1 -- start start 2024-05-08T23:32:25.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8ed19700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a800e3700 0x7f1a800cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8ed19700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a800e77f0 0x7f1a8000a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8ed19700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a800e7c80 0x7f1a80055b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8ed19700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a800e3ae0 con 0x7f1a800e3700 2024-05-08T23:32:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8ed19700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a800cf750 con 0x7f1a800e7c80 2024-05-08T23:32:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8ed19700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a800e3d30 con 0x7f1a800e77f0 2024-05-08T23:32:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8dd17700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a800e3700 0x7f1a800cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8dd17700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a800e3700 0x7f1a800cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57886/0 (socket says 172.21.15.81:57886) 2024-05-08T23:32:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8dd17700 1 -- 172.21.15.81:0/2087643935 learned_addr learned my addr 172.21.15.81:0/2087643935 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8d516700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a800e77f0 0x7f1a8000a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8e518700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a800e7c80 0x7f1a80055b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8dd17700 1 -- 172.21.15.81:0/2087643935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a800e77f0 msgr2=0x7f1a8000a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8dd17700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a800e77f0 0x7f1a8000a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8dd17700 1 -- 172.21.15.81:0/2087643935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a800e7c80 msgr2=0x7f1a80055b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8dd17700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a800e7c80 0x7f1a80055b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8dd17700 1 -- 172.21.15.81:0/2087643935 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8804f5c0 con 0x7f1a800e3700 2024-05-08T23:32:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8e518700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a800e7c80 0x7f1a80055b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:25.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8d516700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a800e77f0 0x7f1a8000a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:25.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8dd17700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a800e3700 0x7f1a800cf210 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1a840011e0 tx=0x7f1a8400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a7e7fc700 1 -- 172.21.15.81:0/2087643935 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a840054e0 con 0x7f1a800e3700 2024-05-08T23:32:25.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a7e7fc700 1 -- 172.21.15.81:0/2087643935 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a840036a0 con 0x7f1a800e3700 2024-05-08T23:32:25.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8ed19700 1 -- 172.21.15.81:0/2087643935 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a800e4010 con 0x7f1a800e3700 2024-05-08T23:32:25.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a7e7fc700 1 -- 172.21.15.81:0/2087643935 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84003a50 con 0x7f1a800e3700 2024-05-08T23:32:25.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.147+0000 7f1a8ed19700 1 -- 172.21.15.81:0/2087643935 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a800e45d0 con 0x7f1a800e3700 2024-05-08T23:32:25.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.151+0000 7f1a8ed19700 1 -- 172.21.15.81:0/2087643935 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a80036600 con 0x7f1a800e3700 2024-05-08T23:32:25.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.151+0000 7f1a7e7fc700 1 -- 172.21.15.81:0/2087643935 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1a84003bb0 con 0x7f1a800e3700 2024-05-08T23:32:25.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.151+0000 7f1a7e7fc700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1a6c066730 0x7f1a6c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.151+0000 7f1a7e7fc700 1 -- 172.21.15.81:0/2087643935 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f1a84083850 con 0x7f1a800e3700 2024-05-08T23:32:25.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.151+0000 7f1a8d516700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1a6c066730 0x7f1a6c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.151+0000 7f1a8d516700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1a6c066730 0x7f1a6c068bf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1a80053000 tx=0x7f1a78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.155+0000 7f1a7e7fc700 1 -- 172.21.15.81:0/2087643935 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1a84050a30 con 0x7f1a800e3700 2024-05-08T23:32:25.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.323+0000 7f1a8ed19700 1 -- 172.21.15.81:0/2087643935 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a800579a0 con 0x7f1a800e3700 2024-05-08T23:32:25.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.323+0000 7f1a7e7fc700 1 -- 172.21.15.81:0/2087643935 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a84017070 con 0x7f1a800e3700 2024-05-08T23:32:25.326 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:32:25.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.327+0000 7f1a8ed19700 1 -- 172.21.15.81:0/2087643935 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1a6c066730 msgr2=0x7f1a6c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.327+0000 7f1a8ed19700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1a6c066730 0x7f1a6c068bf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1a80053000 tx=0x7f1a78008040 comp rx=0 tx=0).stop 2024-05-08T23:32:25.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.327+0000 7f1a8ed19700 1 -- 172.21.15.81:0/2087643935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a800e3700 msgr2=0x7f1a800cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.327+0000 7f1a8ed19700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a800e3700 0x7f1a800cf210 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1a840011e0 tx=0x7f1a8400cae0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.327+0000 7f1a8ed19700 1 -- 172.21.15.81:0/2087643935 shutdown_connections 2024-05-08T23:32:25.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.327+0000 7f1a8ed19700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1a6c066730 0x7f1a6c068bf0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.327+0000 7f1a8ed19700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a800e7c80 0x7f1a80055b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.327+0000 7f1a8ed19700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a800e77f0 0x7f1a8000a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.327+0000 7f1a8ed19700 1 --2- 172.21.15.81:0/2087643935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a800e3700 0x7f1a800cf210 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.327+0000 7f1a8ed19700 1 -- 172.21.15.81:0/2087643935 >> 172.21.15.81:0/2087643935 conn(0x7f1a80056d80 msgr2=0x7f1a80057160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:25.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.327+0000 7f1a8ed19700 1 -- 172.21.15.81:0/2087643935 shutdown_connections 2024-05-08T23:32:25.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.327+0000 7f1a8ed19700 1 -- 172.21.15.81:0/2087643935 wait complete. 2024-05-08T23:32:25.344 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:32:25.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a8ae96700 1 Processor -- start 2024-05-08T23:32:25.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a8ae96700 1 -- start start 2024-05-08T23:32:25.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a8ae96700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a8415d180 0x7f7a840cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a8ae96700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7a8415d560 0x7f7a840d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a8ae96700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a840ce200 con 0x7f7a840d2f60 2024-05-08T23:32:25.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a8ae96700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a840ce480 con 0x7f7a8415d560 2024-05-08T23:32:25.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a8ae96700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a840ce760 con 0x7f7a8415d180 2024-05-08T23:32:25.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a8a695700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f7a840d2f60 0x7f7a840cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:39736/0 (socket says 172.21.15.81:39736) 2024-05-08T23:32:25.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a89693700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7a8415d560 0x7f7a840d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a8a695700 1 -- 172.21.15.81:0/2273553287 learned_addr learned my addr 172.21.15.81:0/2273553287 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:25.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a89e94700 1 --2- 172.21.15.81:0/2273553287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a8415d180 0x7f7a840cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a89693700 1 -- 172.21.15.81:0/2273553287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a8415d180 msgr2=0x7f7a840cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:25.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a89693700 1 --2- 172.21.15.81:0/2273553287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a8415d180 0x7f7a840cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a89693700 1 -- 172.21.15.81:0/2273553287 >> v1:172.21.15.81:6789/0 conn(0x7f7a840d2f60 legacy=0x7f7a840cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a89693700 1 -- 172.21.15.81:0/2273553287 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a840cea40 con 0x7f7a8415d560 2024-05-08T23:32:25.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a89e94700 1 --2- 172.21.15.81:0/2273553287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a8415d180 0x7f7a840cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:25.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a89693700 1 --2- 172.21.15.81:0/2273553287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a8415d560 0x7f7a840d1bf0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009850 tx=0x7f7a74009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9f3020b1a58cff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a88e92700 1 -- 172.21.15.81:0/2273553287 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74011830 con 0x7f7a8415d560 2024-05-08T23:32:25.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a88e92700 1 -- 172.21.15.81:0/2273553287 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a74011990 con 0x7f7a8415d560 2024-05-08T23:32:25.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.463+0000 7f7a88e92700 1 -- 172.21.15.81:0/2273553287 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74011d20 con 0x7f7a8415d560 2024-05-08T23:32:25.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.467+0000 7f7a8ae96700 1 -- 172.21.15.81:0/2273553287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a8415d560 msgr2=0x7f7a840d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.467+0000 7f7a8ae96700 1 --2- 172.21.15.81:0/2273553287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a8415d560 0x7f7a840d1bf0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009850 tx=0x7f7a74009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:25.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.467+0000 7f7a8ae96700 1 -- 172.21.15.81:0/2273553287 shutdown_connections 2024-05-08T23:32:25.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.467+0000 7f7a8ae96700 1 --2- 172.21.15.81:0/2273553287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a8415d560 0x7f7a840d1bf0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.467+0000 7f7a8ae96700 1 --2- 172.21.15.81:0/2273553287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a8415d180 0x7f7a840cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.467+0000 7f7a8ae96700 1 -- 172.21.15.81:0/2273553287 >> 172.21.15.81:0/2273553287 conn(0x7f7a8408c530 msgr2=0x7f7a8409c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:25.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.467+0000 7f7a8ae96700 1 -- 172.21.15.81:0/2273553287 shutdown_connections 2024-05-08T23:32:25.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.467+0000 7f7a8ae96700 1 -- 172.21.15.81:0/2273553287 wait complete. 2024-05-08T23:32:25.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.467+0000 7f7a8ae96700 1 Processor -- start 2024-05-08T23:32:25.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.467+0000 7f7a8ae96700 1 -- start start 2024-05-08T23:32:25.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a8ae96700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a840d2f60 0x7f7a84152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a8ae96700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a8415d180 0x7f7a8415e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a8ae96700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a8415d560 0x7f7a840c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a8ae96700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a84152b60 con 0x7f7a8415d180 2024-05-08T23:32:25.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a8ae96700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8415ed10 con 0x7f7a8415d560 2024-05-08T23:32:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a8ae96700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a840c98f0 con 0x7f7a840d2f60 2024-05-08T23:32:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a89693700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a8415d180 0x7f7a8415e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a89693700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a8415d180 0x7f7a8415e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57912/0 (socket says 172.21.15.81:57912) 2024-05-08T23:32:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a89693700 1 -- 172.21.15.81:0/1348068439 learned_addr learned my addr 172.21.15.81:0/1348068439 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a89e94700 1 --2- 172.21.15.81:0/1348068439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a840d2f60 0x7f7a84152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a8a695700 1 --2- 172.21.15.81:0/1348068439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a8415d560 0x7f7a840c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a89693700 1 -- 172.21.15.81:0/1348068439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a840d2f60 msgr2=0x7f7a84152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a89693700 1 --2- 172.21.15.81:0/1348068439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a840d2f60 0x7f7a84152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a89693700 1 -- 172.21.15.81:0/1348068439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a8415d560 msgr2=0x7f7a840c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a89693700 1 --2- 172.21.15.81:0/1348068439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a8415d560 0x7f7a840c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a89693700 1 -- 172.21.15.81:0/1348068439 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a740094b0 con 0x7f7a8415d180 2024-05-08T23:32:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a89693700 1 --2- 172.21.15.81:0/1348068439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a8415d180 0x7f7a8415e7d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7a74013000 tx=0x7f7a74004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a7a7fc700 1 -- 172.21.15.81:0/1348068439 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74004610 con 0x7f7a8415d180 2024-05-08T23:32:25.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a7a7fc700 1 -- 172.21.15.81:0/1348068439 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7401e070 con 0x7f7a8415d180 2024-05-08T23:32:25.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a8ae96700 1 -- 172.21.15.81:0/1348068439 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8415ef40 con 0x7f7a8415d180 2024-05-08T23:32:25.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a7a7fc700 1 -- 172.21.15.81:0/1348068439 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a740038e0 con 0x7f7a8415d180 2024-05-08T23:32:25.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a8ae96700 1 -- 172.21.15.81:0/1348068439 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8415f500 con 0x7f7a8415d180 2024-05-08T23:32:25.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.471+0000 7f7a8ae96700 1 -- 172.21.15.81:0/1348068439 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a840672a0 con 0x7f7a8415d180 2024-05-08T23:32:25.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.475+0000 7f7a7a7fc700 1 -- 172.21.15.81:0/1348068439 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7a74003a40 con 0x7f7a8415d180 2024-05-08T23:32:25.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.475+0000 7f7a7a7fc700 1 --2- 172.21.15.81:0/1348068439 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7a68066730 0x7f7a68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.475+0000 7f7a7a7fc700 1 -- 172.21.15.81:0/1348068439 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f7a74085160 con 0x7f7a8415d180 2024-05-08T23:32:25.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.475+0000 7f7a89e94700 1 --2- 172.21.15.81:0/1348068439 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7a68066730 0x7f7a68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.475+0000 7f7a89e94700 1 --2- 172.21.15.81:0/1348068439 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7a68066730 0x7f7a68068bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7a800044b0 tx=0x7f7a80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.483+0000 7f7a7a7fc700 1 -- 172.21.15.81:0/1348068439 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7a74052330 con 0x7f7a8415d180 2024-05-08T23:32:25.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.655+0000 7f7a8ae96700 1 -- 172.21.15.81:0/1348068439 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a840cfca0 con 0x7f7a8415d180 2024-05-08T23:32:25.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.655+0000 7f7a7a7fc700 1 -- 172.21.15.81:0/1348068439 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a74055950 con 0x7f7a8415d180 2024-05-08T23:32:25.657 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:32:25.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.655+0000 7f7a8ae96700 1 -- 172.21.15.81:0/1348068439 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7a68066730 msgr2=0x7f7a68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.655+0000 7f7a8ae96700 1 --2- 172.21.15.81:0/1348068439 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7a68066730 0x7f7a68068bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7a800044b0 tx=0x7f7a80008040 comp rx=0 tx=0).stop 2024-05-08T23:32:25.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.655+0000 7f7a8ae96700 1 -- 172.21.15.81:0/1348068439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a8415d180 msgr2=0x7f7a8415e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.659+0000 7f7a8ae96700 1 --2- 172.21.15.81:0/1348068439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a8415d180 0x7f7a8415e7d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7a74013000 tx=0x7f7a74004770 comp rx=0 tx=0).stop 2024-05-08T23:32:25.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.659+0000 7f7a8ae96700 1 -- 172.21.15.81:0/1348068439 shutdown_connections 2024-05-08T23:32:25.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.659+0000 7f7a8ae96700 1 --2- 172.21.15.81:0/1348068439 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7a68066730 0x7f7a68068bf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.659+0000 7f7a8ae96700 1 --2- 172.21.15.81:0/1348068439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a8415d560 0x7f7a840c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.659+0000 7f7a8ae96700 1 --2- 172.21.15.81:0/1348068439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a8415d180 0x7f7a8415e7d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.659+0000 7f7a8ae96700 1 --2- 172.21.15.81:0/1348068439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a840d2f60 0x7f7a84152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.659+0000 7f7a8ae96700 1 -- 172.21.15.81:0/1348068439 >> 172.21.15.81:0/1348068439 conn(0x7f7a8408c530 msgr2=0x7f7a840d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:25.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.659+0000 7f7a8ae96700 1 -- 172.21.15.81:0/1348068439 shutdown_connections 2024-05-08T23:32:25.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.659+0000 7f7a8ae96700 1 -- 172.21.15.81:0/1348068439 wait complete. 2024-05-08T23:32:25.672 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:32:25.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.795+0000 7fcd2a53f700 1 Processor -- start 2024-05-08T23:32:25.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.795+0000 7fcd2a53f700 1 -- start start 2024-05-08T23:32:25.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd2a53f700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcd2415d010 0x7fcd2409a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd2a53f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd2406bfb0 0x7fcd2406c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd2a53f700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd2409a9c0 con 0x7fcd2406b890 2024-05-08T23:32:25.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd2a53f700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd240b63c0 con 0x7fcd2415d010 2024-05-08T23:32:25.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd2a53f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2406c980 con 0x7fcd2406bfb0 2024-05-08T23:32:25.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd28d3c700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcd2406b890 0x7fcd240b5cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:39752/0 (socket says 172.21.15.81:39752) 2024-05-08T23:32:25.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd28d3c700 1 -- 172.21.15.81:0/1917350257 learned_addr learned my addr 172.21.15.81:0/1917350257 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:25.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd2953d700 1 --2- 172.21.15.81:0/1917350257 >> v2:172.21.15.117:3300/0 conn(0x7fcd2415d010 0x7fcd2409a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd29d3e700 1 --2- 172.21.15.81:0/1917350257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd2406bfb0 0x7fcd2406c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd17fff700 1 -- 172.21.15.81:0/1917350257 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257318187 0 0) 0x7fcd20001110 con 0x7fcd2406b890 2024-05-08T23:32:25.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd17fff700 1 -- 172.21.15.81:0/1917350257 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd100035d0 con 0x7fcd2406b890 2024-05-08T23:32:25.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd29d3e700 1 -- 172.21.15.81:0/1917350257 >> v2:172.21.15.117:3300/0 conn(0x7fcd2415d010 msgr2=0x7fcd2409a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:25.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd29d3e700 1 --2- 172.21.15.81:0/1917350257 >> v2:172.21.15.117:3300/0 conn(0x7fcd2415d010 0x7fcd2409a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd29d3e700 1 -- 172.21.15.81:0/1917350257 >> v1:172.21.15.81:6789/0 conn(0x7fcd2406b890 legacy=0x7fcd240b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd29d3e700 1 -- 172.21.15.81:0/1917350257 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd240b4c50 con 0x7fcd2406bfb0 2024-05-08T23:32:25.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd29d3e700 1 --2- 172.21.15.81:0/1917350257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd2406bfb0 0x7fcd2406c440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009d90 tx=0x7fcd1c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c53e566eea42e240 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd17fff700 1 -- 172.21.15.81:0/1917350257 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c00e070 con 0x7fcd2406bfb0 2024-05-08T23:32:25.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd17fff700 1 -- 172.21.15.81:0/1917350257 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c00bb20 con 0x7fcd2406bfb0 2024-05-08T23:32:25.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd17fff700 1 -- 172.21.15.81:0/1917350257 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c00be90 con 0x7fcd2406bfb0 2024-05-08T23:32:25.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd2a53f700 1 -- 172.21.15.81:0/1917350257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd2406bfb0 msgr2=0x7fcd2406c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd2a53f700 1 --2- 172.21.15.81:0/1917350257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd2406bfb0 0x7fcd2406c440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009d90 tx=0x7fcd1c00b330 comp rx=0 tx=0).stop 2024-05-08T23:32:25.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd2a53f700 1 -- 172.21.15.81:0/1917350257 shutdown_connections 2024-05-08T23:32:25.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd2a53f700 1 --2- 172.21.15.81:0/1917350257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd2406bfb0 0x7fcd2406c440 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd2a53f700 1 --2- 172.21.15.81:0/1917350257 >> v2:172.21.15.117:3300/0 conn(0x7fcd2415d010 0x7fcd2409a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd2a53f700 1 -- 172.21.15.81:0/1917350257 >> 172.21.15.81:0/1917350257 conn(0x7fcd240b2ae0 msgr2=0x7fcd240b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:25.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd2a53f700 1 -- 172.21.15.81:0/1917350257 shutdown_connections 2024-05-08T23:32:25.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.799+0000 7fcd2a53f700 1 -- 172.21.15.81:0/1917350257 wait complete. 2024-05-08T23:32:25.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd2a53f700 1 Processor -- start 2024-05-08T23:32:25.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd2a53f700 1 -- start start 2024-05-08T23:32:25.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd2a53f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd2406b890 0x7fcd240cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd2a53f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd2406bfb0 0x7fcd240d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd2a53f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd2415d010 0x7fcd240ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd2a53f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2415d760 con 0x7fcd2406bfb0 2024-05-08T23:32:25.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd2a53f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2415da40 con 0x7fcd2406b890 2024-05-08T23:32:25.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd2a53f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2415dd20 con 0x7fcd2415d010 2024-05-08T23:32:25.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd28d3c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd2406bfb0 0x7fcd240d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd28d3c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd2406bfb0 0x7fcd240d02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57918/0 (socket says 172.21.15.81:57918) 2024-05-08T23:32:25.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd28d3c700 1 -- 172.21.15.81:0/408030741 learned_addr learned my addr 172.21.15.81:0/408030741 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:25.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd29d3e700 1 --2- 172.21.15.81:0/408030741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd2415d010 0x7fcd240ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd2953d700 1 --2- 172.21.15.81:0/408030741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd2406b890 0x7fcd240cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd28d3c700 1 -- 172.21.15.81:0/408030741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd2415d010 msgr2=0x7fcd240ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd28d3c700 1 --2- 172.21.15.81:0/408030741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd2415d010 0x7fcd240ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd28d3c700 1 -- 172.21.15.81:0/408030741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd2406b890 msgr2=0x7fcd240cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd28d3c700 1 --2- 172.21.15.81:0/408030741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd2406b890 0x7fcd240cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd28d3c700 1 -- 172.21.15.81:0/408030741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c009a70 con 0x7fcd2406bfb0 2024-05-08T23:32:25.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd2953d700 1 --2- 172.21.15.81:0/408030741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd2406b890 0x7fcd240cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:25.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd28d3c700 1 --2- 172.21.15.81:0/408030741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd2406bfb0 0x7fcd240d02e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000c4f0 tx=0x7fcd2000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd15ffb700 1 -- 172.21.15.81:0/408030741 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20005d90 con 0x7fcd2406bfb0 2024-05-08T23:32:25.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd15ffb700 1 -- 172.21.15.81:0/408030741 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd200070d0 con 0x7fcd2406bfb0 2024-05-08T23:32:25.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd2a53f700 1 -- 172.21.15.81:0/408030741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2415e000 con 0x7fcd2406bfb0 2024-05-08T23:32:25.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd15ffb700 1 -- 172.21.15.81:0/408030741 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20007480 con 0x7fcd2406bfb0 2024-05-08T23:32:25.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd2a53f700 1 -- 172.21.15.81:0/408030741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2415d3f0 con 0x7fcd2406bfb0 2024-05-08T23:32:25.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.803+0000 7fcd2a53f700 1 -- 172.21.15.81:0/408030741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd240672a0 con 0x7fcd2406bfb0 2024-05-08T23:32:25.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.807+0000 7fcd15ffb700 1 -- 172.21.15.81:0/408030741 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcd200075e0 con 0x7fcd2406bfb0 2024-05-08T23:32:25.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.807+0000 7fcd15ffb700 1 --2- 172.21.15.81:0/408030741 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcd08066730 0x7fcd08068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.807+0000 7fcd15ffb700 1 -- 172.21.15.81:0/408030741 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fcd200837d0 con 0x7fcd2406bfb0 2024-05-08T23:32:25.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.807+0000 7fcd2953d700 1 --2- 172.21.15.81:0/408030741 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcd08066730 0x7fcd08068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.807+0000 7fcd2953d700 1 --2- 172.21.15.81:0/408030741 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcd08066730 0x7fcd08068bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009ad0 tx=0x7fcd18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.811+0000 7fcd15ffb700 1 -- 172.21.15.81:0/408030741 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcd20050930 con 0x7fcd2406bfb0 2024-05-08T23:32:25.988 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.987+0000 7fcd2a53f700 1 -- 172.21.15.81:0/408030741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd2409a9c0 con 0x7fcd2406bfb0 2024-05-08T23:32:25.988 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.987+0000 7fcd15ffb700 1 -- 172.21.15.81:0/408030741 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd20017020 con 0x7fcd2406bfb0 2024-05-08T23:32:25.988 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:32:25.991 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.987+0000 7fcd2a53f700 1 -- 172.21.15.81:0/408030741 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcd08066730 msgr2=0x7fcd08068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.991 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.987+0000 7fcd2a53f700 1 --2- 172.21.15.81:0/408030741 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcd08066730 0x7fcd08068bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009ad0 tx=0x7fcd18008040 comp rx=0 tx=0).stop 2024-05-08T23:32:25.991 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.987+0000 7fcd2a53f700 1 -- 172.21.15.81:0/408030741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd2406bfb0 msgr2=0x7fcd240d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.991 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.987+0000 7fcd2a53f700 1 --2- 172.21.15.81:0/408030741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd2406bfb0 0x7fcd240d02e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000c4f0 tx=0x7fcd2000c8b0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.987+0000 7fcd2a53f700 1 -- 172.21.15.81:0/408030741 shutdown_connections 2024-05-08T23:32:25.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.987+0000 7fcd2a53f700 1 --2- 172.21.15.81:0/408030741 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcd08066730 0x7fcd08068bf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.987+0000 7fcd2a53f700 1 --2- 172.21.15.81:0/408030741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd2415d010 0x7fcd240ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.987+0000 7fcd2a53f700 1 --2- 172.21.15.81:0/408030741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd2406bfb0 0x7fcd240d02e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.987+0000 7fcd2a53f700 1 --2- 172.21.15.81:0/408030741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd2406b890 0x7fcd240cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.987+0000 7fcd2a53f700 1 -- 172.21.15.81:0/408030741 >> 172.21.15.81:0/408030741 conn(0x7fcd240b2ae0 msgr2=0x7fcd2406fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:25.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.987+0000 7fcd2a53f700 1 -- 172.21.15.81:0/408030741 shutdown_connections 2024-05-08T23:32:25.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:25.987+0000 7fcd2a53f700 1 -- 172.21.15.81:0/408030741 wait complete. 2024-05-08T23:32:26.004 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-05-08T23:32:26.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.123+0000 7f2040f0d700 1 Processor -- start 2024-05-08T23:32:26.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.123+0000 7f2040f0d700 1 -- start start 2024-05-08T23:32:26.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f2040f0d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f203c159730 0x7f203c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f2040f0d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203c06ed70 0x7f203c0b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f2040f0d700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c15e220 con 0x7f203c0cb770 2024-05-08T23:32:26.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f2040f0d700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203c0b2790 con 0x7f203c159730 2024-05-08T23:32:26.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f2040f0d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c06f340 con 0x7f203c06ed70 2024-05-08T23:32:26.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f2033fff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f203c0cb770 0x7f203c06e660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55374/0 (socket says 172.21.15.81:55374) 2024-05-08T23:32:26.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f2033fff700 1 -- 172.21.15.81:0/1803775216 learned_addr learned my addr 172.21.15.81:0/1803775216 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:26.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f203b7fe700 1 --2- 172.21.15.81:0/1803775216 >> v2:172.21.15.117:3300/0 conn(0x7f203c159730 0x7f203c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f203bfff700 1 --2- 172.21.15.81:0/1803775216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203c06ed70 0x7f203c0b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f203affd700 1 -- 172.21.15.81:0/1803775216 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899200623 0 0) 0x7f203c15e220 con 0x7f203c0cb770 2024-05-08T23:32:26.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f203affd700 1 -- 172.21.15.81:0/1803775216 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20280046e0 con 0x7f203c0cb770 2024-05-08T23:32:26.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f203b7fe700 1 -- 172.21.15.81:0/1803775216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203c06ed70 msgr2=0x7f203c0b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:26.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f203b7fe700 1 --2- 172.21.15.81:0/1803775216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203c06ed70 0x7f203c0b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f203b7fe700 1 -- 172.21.15.81:0/1803775216 >> v1:172.21.15.81:6789/0 conn(0x7f203c0cb770 legacy=0x7f203c06e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f203b7fe700 1 -- 172.21.15.81:0/1803775216 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c159b10 con 0x7f203c159730 2024-05-08T23:32:26.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f203b7fe700 1 --2- 172.21.15.81:0/1803775216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f203c159730 0x7f203c15d2a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f202c009850 tx=0x7f202c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fedc5aa9ee1b0ab3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:26.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f203affd700 1 -- 172.21.15.81:0/1803775216 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c011830 con 0x7f203c159730 2024-05-08T23:32:26.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f203affd700 1 -- 172.21.15.81:0/1803775216 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c011990 con 0x7f203c159730 2024-05-08T23:32:26.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f203affd700 1 -- 172.21.15.81:0/1803775216 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c011d40 con 0x7f203c159730 2024-05-08T23:32:26.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f2040f0d700 1 -- 172.21.15.81:0/1803775216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f203c159730 msgr2=0x7f203c15d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f2040f0d700 1 --2- 172.21.15.81:0/1803775216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f203c159730 0x7f203c15d2a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f202c009850 tx=0x7f202c009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:26.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f2040f0d700 1 -- 172.21.15.81:0/1803775216 shutdown_connections 2024-05-08T23:32:26.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f2040f0d700 1 --2- 172.21.15.81:0/1803775216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203c06ed70 0x7f203c0b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f2040f0d700 1 --2- 172.21.15.81:0/1803775216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f203c159730 0x7f203c15d2a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f2040f0d700 1 -- 172.21.15.81:0/1803775216 >> 172.21.15.81:0/1803775216 conn(0x7f203c0717b0 msgr2=0x7f203c06a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:26.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f2040f0d700 1 -- 172.21.15.81:0/1803775216 shutdown_connections 2024-05-08T23:32:26.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.127+0000 7f2040f0d700 1 -- 172.21.15.81:0/1803775216 wait complete. 2024-05-08T23:32:26.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.131+0000 7f2040f0d700 1 Processor -- start 2024-05-08T23:32:26.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.131+0000 7f2040f0d700 1 -- start start 2024-05-08T23:32:26.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f2040f0d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f203c06ed70 0x7f203c0a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f2040f0d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f203c0cb770 0x7f203c0aa050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f2040f0d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203c159730 0x7f203c148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f2040f0d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0aa590 con 0x7f203c06ed70 2024-05-08T23:32:26.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f2040f0d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c149470 con 0x7f203c0cb770 2024-05-08T23:32:26.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f2040f0d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0a8b60 con 0x7f203c159730 2024-05-08T23:32:26.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f203b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f203c06ed70 0x7f203c0a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f203b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f203c06ed70 0x7f203c0a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39632/0 (socket says 172.21.15.81:39632) 2024-05-08T23:32:26.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f203b7fe700 1 -- 172.21.15.81:0/4169204456 learned_addr learned my addr 172.21.15.81:0/4169204456 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:26.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f2033fff700 1 --2- 172.21.15.81:0/4169204456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f203c0cb770 0x7f203c0aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f203bfff700 1 --2- 172.21.15.81:0/4169204456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203c159730 0x7f203c148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f203b7fe700 1 -- 172.21.15.81:0/4169204456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203c159730 msgr2=0x7f203c148f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f203b7fe700 1 --2- 172.21.15.81:0/4169204456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203c159730 0x7f203c148f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f203b7fe700 1 -- 172.21.15.81:0/4169204456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f203c0cb770 msgr2=0x7f203c0aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f203b7fe700 1 --2- 172.21.15.81:0/4169204456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f203c0cb770 0x7f203c0aa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f203b7fe700 1 -- 172.21.15.81:0/4169204456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c0094b0 con 0x7f203c06ed70 2024-05-08T23:32:26.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f203b7fe700 1 --2- 172.21.15.81:0/4169204456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f203c06ed70 0x7f203c0a84d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f202c014040 tx=0x7f202c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:26.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f2038ff9700 1 -- 172.21.15.81:0/4169204456 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c004510 con 0x7f203c06ed70 2024-05-08T23:32:26.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f2038ff9700 1 -- 172.21.15.81:0/4169204456 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c01e070 con 0x7f203c06ed70 2024-05-08T23:32:26.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f2040f0d700 1 -- 172.21.15.81:0/4169204456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c0a8e40 con 0x7f203c06ed70 2024-05-08T23:32:26.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f2038ff9700 1 -- 172.21.15.81:0/4169204456 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c003900 con 0x7f203c06ed70 2024-05-08T23:32:26.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f2040f0d700 1 -- 172.21.15.81:0/4169204456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c0a9400 con 0x7f203c06ed70 2024-05-08T23:32:26.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.135+0000 7f2040f0d700 1 -- 172.21.15.81:0/4169204456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203c0672a0 con 0x7f203c06ed70 2024-05-08T23:32:26.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.139+0000 7f2038ff9700 1 -- 172.21.15.81:0/4169204456 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f202c003a60 con 0x7f203c06ed70 2024-05-08T23:32:26.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.139+0000 7f2038ff9700 1 --2- 172.21.15.81:0/4169204456 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f201c066730 0x7f201c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.139+0000 7f2033fff700 1 --2- 172.21.15.81:0/4169204456 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f201c066730 0x7f201c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.139+0000 7f2038ff9700 1 -- 172.21.15.81:0/4169204456 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f202c020070 con 0x7f203c06ed70 2024-05-08T23:32:26.142 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.139+0000 7f2033fff700 1 --2- 172.21.15.81:0/4169204456 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f201c066730 0x7f201c068bf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f203c06d520 tx=0x7f2024007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:26.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.143+0000 7f2038ff9700 1 -- 172.21.15.81:0/4169204456 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f202c0527d0 con 0x7f203c06ed70 2024-05-08T23:32:26.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.311+0000 7f2040f0d700 1 -- 172.21.15.81:0/4169204456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f203c0b4440 con 0x7f203c06ed70 2024-05-08T23:32:26.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.311+0000 7f2038ff9700 1 -- 172.21.15.81:0/4169204456 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v39) v1 ==== 97+0+11 (secure 0 0 0) 0x7f202c006ce0 con 0x7f203c06ed70 2024-05-08T23:32:26.313 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:32:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.315+0000 7f2040f0d700 1 -- 172.21.15.81:0/4169204456 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f201c066730 msgr2=0x7f201c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.315+0000 7f2040f0d700 1 --2- 172.21.15.81:0/4169204456 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f201c066730 0x7f201c068bf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f203c06d520 tx=0x7f2024007de0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.315+0000 7f2040f0d700 1 -- 172.21.15.81:0/4169204456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f203c06ed70 msgr2=0x7f203c0a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.315+0000 7f2040f0d700 1 --2- 172.21.15.81:0/4169204456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f203c06ed70 0x7f203c0a84d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f202c014040 tx=0x7f202c004770 comp rx=0 tx=0).stop 2024-05-08T23:32:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.315+0000 7f2040f0d700 1 -- 172.21.15.81:0/4169204456 shutdown_connections 2024-05-08T23:32:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.315+0000 7f2040f0d700 1 --2- 172.21.15.81:0/4169204456 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f201c066730 0x7f201c068bf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.315+0000 7f2040f0d700 1 --2- 172.21.15.81:0/4169204456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f203c159730 0x7f203c148f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.315+0000 7f2040f0d700 1 --2- 172.21.15.81:0/4169204456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f203c0cb770 0x7f203c0aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.315+0000 7f2040f0d700 1 --2- 172.21.15.81:0/4169204456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f203c06ed70 0x7f203c0a84d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.315+0000 7f2040f0d700 1 -- 172.21.15.81:0/4169204456 >> 172.21.15.81:0/4169204456 conn(0x7f203c0717b0 msgr2=0x7f203c157a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:26.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.315+0000 7f2040f0d700 1 -- 172.21.15.81:0/4169204456 shutdown_connections 2024-05-08T23:32:26.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.315+0000 7f2040f0d700 1 -- 172.21.15.81:0/4169204456 wait complete. 2024-05-08T23:32:26.329 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:26.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7734f700 1 Processor -- start 2024-05-08T23:32:26.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7734f700 1 -- start start 2024-05-08T23:32:26.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7734f700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fed70071350 0x7fed700b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7734f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed70151110 0x7fed7015a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7734f700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed700b4890 con 0x7fed70071730 2024-05-08T23:32:26.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7734f700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7015b470 con 0x7fed70071350 2024-05-08T23:32:26.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7734f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed700b38b0 con 0x7fed70151110 2024-05-08T23:32:26.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7634d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fed70071350 0x7fed700b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7634d700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fed70071350 0x7fed700b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39634/0 (socket says 172.21.15.81:39634) 2024-05-08T23:32:26.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7634d700 1 -- 172.21.15.81:0/2050012828 learned_addr learned my addr 172.21.15.81:0/2050012828 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:26.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed76b4e700 1 --2- 172.21.15.81:0/2050012828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed70151110 0x7fed7015a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7634d700 1 -- 172.21.15.81:0/2050012828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed70151110 msgr2=0x7fed7015a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:26.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7634d700 1 --2- 172.21.15.81:0/2050012828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed70151110 0x7fed7015a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7634d700 1 -- 172.21.15.81:0/2050012828 >> v1:172.21.15.117:6789/0 conn(0x7fed70071730 legacy=0x7fed70150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:26.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7634d700 1 -- 172.21.15.81:0/2050012828 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7015b190 con 0x7fed70071350 2024-05-08T23:32:26.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed76b4e700 1 --2- 172.21.15.81:0/2050012828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed70151110 0x7fed7015a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:26.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7634d700 1 --2- 172.21.15.81:0/2050012828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed70071350 0x7fed700b4350 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fed6c009910 tx=0x7fed6c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299d82d8a17d4a11 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:26.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7534b700 1 -- 172.21.15.81:0/2050012828 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c00ba60 con 0x7fed70071350 2024-05-08T23:32:26.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7534b700 1 -- 172.21.15.81:0/2050012828 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6c00bbc0 con 0x7fed70071350 2024-05-08T23:32:26.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.435+0000 7fed7534b700 1 -- 172.21.15.81:0/2050012828 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c009e60 con 0x7fed70071350 2024-05-08T23:32:26.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.439+0000 7fed7734f700 1 -- 172.21.15.81:0/2050012828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed70071350 msgr2=0x7fed700b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.439+0000 7fed7734f700 1 --2- 172.21.15.81:0/2050012828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed70071350 0x7fed700b4350 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fed6c009910 tx=0x7fed6c009c20 comp rx=0 tx=0).stop 2024-05-08T23:32:26.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.439+0000 7fed7734f700 1 -- 172.21.15.81:0/2050012828 shutdown_connections 2024-05-08T23:32:26.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.439+0000 7fed7734f700 1 --2- 172.21.15.81:0/2050012828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed70151110 0x7fed7015a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.439+0000 7fed7734f700 1 --2- 172.21.15.81:0/2050012828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed70071350 0x7fed700b4350 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.439+0000 7fed7734f700 1 -- 172.21.15.81:0/2050012828 >> 172.21.15.81:0/2050012828 conn(0x7fed7006b470 msgr2=0x7fed7006b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:26.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.439+0000 7fed7734f700 1 -- 172.21.15.81:0/2050012828 shutdown_connections 2024-05-08T23:32:26.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.439+0000 7fed7734f700 1 -- 172.21.15.81:0/2050012828 wait complete. 2024-05-08T23:32:26.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.439+0000 7fed7734f700 1 Processor -- start 2024-05-08T23:32:26.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.439+0000 7fed7734f700 1 -- start start 2024-05-08T23:32:26.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7734f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed70071350 0x7fed7014f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7734f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed70071730 0x7fed701f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7734f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed70151110 0x7fed701466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7734f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed70147370 con 0x7fed70071350 2024-05-08T23:32:26.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7734f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed701f5070 con 0x7fed70071730 2024-05-08T23:32:26.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7734f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed701f4ab0 con 0x7fed70151110 2024-05-08T23:32:26.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7634d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed70071350 0x7fed7014f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7634d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed70071350 0x7fed7014f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39650/0 (socket says 172.21.15.81:39650) 2024-05-08T23:32:26.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7634d700 1 -- 172.21.15.81:0/4066648102 learned_addr learned my addr 172.21.15.81:0/4066648102 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:26.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed75b4c700 1 --2- 172.21.15.81:0/4066648102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed70071730 0x7fed701f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed76b4e700 1 --2- 172.21.15.81:0/4066648102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed70151110 0x7fed701466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7634d700 1 -- 172.21.15.81:0/4066648102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed70151110 msgr2=0x7fed701466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7634d700 1 --2- 172.21.15.81:0/4066648102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed70151110 0x7fed701466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7634d700 1 -- 172.21.15.81:0/4066648102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed70071730 msgr2=0x7fed701f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7634d700 1 --2- 172.21.15.81:0/4066648102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed70071730 0x7fed701f4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7634d700 1 -- 172.21.15.81:0/4066648102 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c009570 con 0x7fed70071350 2024-05-08T23:32:26.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed76b4e700 1 --2- 172.21.15.81:0/4066648102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed70151110 0x7fed701466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:26.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7634d700 1 --2- 172.21.15.81:0/4066648102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed70071350 0x7fed7014f3d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00b3c0 tx=0x7fed6c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:26.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed62ffd700 1 -- 172.21.15.81:0/4066648102 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c004610 con 0x7fed70071350 2024-05-08T23:32:26.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed62ffd700 1 -- 172.21.15.81:0/4066648102 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6c01f070 con 0x7fed70071350 2024-05-08T23:32:26.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7734f700 1 -- 172.21.15.81:0/4066648102 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed701f4d90 con 0x7fed70071350 2024-05-08T23:32:26.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed62ffd700 1 -- 172.21.15.81:0/4066648102 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c0038e0 con 0x7fed70071350 2024-05-08T23:32:26.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7734f700 1 -- 172.21.15.81:0/4066648102 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed70152e40 con 0x7fed70071350 2024-05-08T23:32:26.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.443+0000 7fed7734f700 1 -- 172.21.15.81:0/4066648102 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed70152fd0 con 0x7fed70071350 2024-05-08T23:32:26.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.447+0000 7fed62ffd700 1 -- 172.21.15.81:0/4066648102 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fed6c003b00 con 0x7fed70071350 2024-05-08T23:32:26.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.447+0000 7fed62ffd700 1 --2- 172.21.15.81:0/4066648102 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fed54066730 0x7fed54068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.447+0000 7fed62ffd700 1 -- 172.21.15.81:0/4066648102 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fed6c021070 con 0x7fed70071350 2024-05-08T23:32:26.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.447+0000 7fed75b4c700 1 --2- 172.21.15.81:0/4066648102 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fed54066730 0x7fed54068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.447+0000 7fed75b4c700 1 --2- 172.21.15.81:0/4066648102 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fed54066730 0x7fed54068bf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fed7014fd00 tx=0x7fed64009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:26.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.455+0000 7fed62ffd700 1 -- 172.21.15.81:0/4066648102 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fed6c052e10 con 0x7fed70071350 2024-05-08T23:32:26.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.667+0000 7fed7734f700 1 -- 172.21.15.81:0/4066648102 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed700672a0 con 0x7fed70071350 2024-05-08T23:32:26.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.667+0000 7fed62ffd700 1 -- 172.21.15.81:0/4066648102 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4745 (secure 0 0 0) 0x7fed6c015090 con 0x7fed70071350 2024-05-08T23:32:26.671 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:26.671 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1243037934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1243037934},{"type":"v1","addr":"172.21.15.81:6835","nonce":1243037934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3124115182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3124115182},{"type":"v1","addr":"172.21.15.81:6837","nonce":3124115182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1423349691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":1423349691},{"type":"v1","addr":"172.21.15.117:6834","nonce":1423349691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:31:37.889298+0000","modified":"2024-05-08T23:31:47.307677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14852":{"gid":14852,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/3091184157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3091184157},{"type":"v1","addr":"172.21.15.117:6835","nonce":3091184157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:32:26.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.671+0000 7fed7734f700 1 -- 172.21.15.81:0/4066648102 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fed54066730 msgr2=0x7fed54068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.671+0000 7fed7734f700 1 --2- 172.21.15.81:0/4066648102 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fed54066730 0x7fed54068bf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fed7014fd00 tx=0x7fed64009040 comp rx=0 tx=0).stop 2024-05-08T23:32:26.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.671+0000 7fed7734f700 1 -- 172.21.15.81:0/4066648102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed70071350 msgr2=0x7fed7014f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.671+0000 7fed7734f700 1 --2- 172.21.15.81:0/4066648102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed70071350 0x7fed7014f3d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00b3c0 tx=0x7fed6c004770 comp rx=0 tx=0).stop 2024-05-08T23:32:26.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.671+0000 7fed7734f700 1 -- 172.21.15.81:0/4066648102 shutdown_connections 2024-05-08T23:32:26.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.671+0000 7fed7734f700 1 --2- 172.21.15.81:0/4066648102 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fed54066730 0x7fed54068bf0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.671+0000 7fed7734f700 1 --2- 172.21.15.81:0/4066648102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed70151110 0x7fed701466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.671+0000 7fed7734f700 1 --2- 172.21.15.81:0/4066648102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed70071730 0x7fed701f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.671+0000 7fed7734f700 1 --2- 172.21.15.81:0/4066648102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed70071350 0x7fed7014f3d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.671+0000 7fed7734f700 1 -- 172.21.15.81:0/4066648102 >> 172.21.15.81:0/4066648102 conn(0x7fed7006b470 msgr2=0x7fed701590e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:26.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.671+0000 7fed7734f700 1 -- 172.21.15.81:0/4066648102 shutdown_connections 2024-05-08T23:32:26.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.671+0000 7fed7734f700 1 -- 172.21.15.81:0/4066648102 wait complete. 2024-05-08T23:32:26.675 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 15 2024-05-08T23:32:26.686 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:26.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.799+0000 7f296dfd3700 1 Processor -- start 2024-05-08T23:32:26.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.799+0000 7f296dfd3700 1 -- start start 2024-05-08T23:32:26.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.799+0000 7f296dfd3700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680ce990 0x7f296815f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.799+0000 7f296dfd3700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f296815f770 0x7f29680b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.799+0000 7f296dfd3700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296815eda0 con 0x7f296815afa0 2024-05-08T23:32:26.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.799+0000 7f296dfd3700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f296815ea70 con 0x7f296815f770 2024-05-08T23:32:26.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.799+0000 7f296dfd3700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296815ba90 con 0x7f29680ce990 2024-05-08T23:32:26.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.799+0000 7f296cfd1700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f296815afa0 0x7f296815b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55390/0 (socket says 172.21.15.81:55390) 2024-05-08T23:32:26.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.799+0000 7f296cfd1700 1 -- 172.21.15.81:0/4081686280 learned_addr learned my addr 172.21.15.81:0/4081686280 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:26.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.799+0000 7f2967fff700 1 --2- 172.21.15.81:0/4081686280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680ce990 0x7f296815f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.799+0000 7f296d7d2700 1 --2- 172.21.15.81:0/4081686280 >> v2:172.21.15.117:3300/0 conn(0x7f296815f770 0x7f29680b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f29677fe700 1 -- 172.21.15.81:0/4081686280 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812787339 0 0) 0x7f296815eda0 con 0x7f296815afa0 2024-05-08T23:32:26.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f29677fe700 1 -- 172.21.15.81:0/4081686280 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29540035d0 con 0x7f296815afa0 2024-05-08T23:32:26.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f2967fff700 1 -- 172.21.15.81:0/4081686280 >> v2:172.21.15.117:3300/0 conn(0x7f296815f770 msgr2=0x7f29680b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:26.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f2967fff700 1 --2- 172.21.15.81:0/4081686280 >> v2:172.21.15.117:3300/0 conn(0x7f296815f770 0x7f29680b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f2967fff700 1 -- 172.21.15.81:0/4081686280 >> v1:172.21.15.81:6789/0 conn(0x7f296815afa0 legacy=0x7f296815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f2967fff700 1 -- 172.21.15.81:0/4081686280 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296815bd40 con 0x7f29680ce990 2024-05-08T23:32:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f296d7d2700 1 --2- 172.21.15.81:0/4081686280 >> v2:172.21.15.117:3300/0 conn(0x7f296815f770 0x7f29680b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f2967fff700 1 --2- 172.21.15.81:0/4081686280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680ce990 0x7f296815f230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2950009850 tx=0x7f2950009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bad84c7eb1ade68f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f29677fe700 1 -- 172.21.15.81:0/4081686280 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950011830 con 0x7f29680ce990 2024-05-08T23:32:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f29677fe700 1 -- 172.21.15.81:0/4081686280 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2950011990 con 0x7f29680ce990 2024-05-08T23:32:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f29677fe700 1 -- 172.21.15.81:0/4081686280 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950011d40 con 0x7f29680ce990 2024-05-08T23:32:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f296dfd3700 1 -- 172.21.15.81:0/4081686280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680ce990 msgr2=0x7f296815f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f296dfd3700 1 --2- 172.21.15.81:0/4081686280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680ce990 0x7f296815f230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2950009850 tx=0x7f2950009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f296dfd3700 1 -- 172.21.15.81:0/4081686280 shutdown_connections 2024-05-08T23:32:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f296dfd3700 1 --2- 172.21.15.81:0/4081686280 >> v2:172.21.15.117:3300/0 conn(0x7f296815f770 0x7f29680b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f296dfd3700 1 --2- 172.21.15.81:0/4081686280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680ce990 0x7f296815f230 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f296dfd3700 1 -- 172.21.15.81:0/4081686280 >> 172.21.15.81:0/4081686280 conn(0x7f296806c330 msgr2=0x7f296806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f296dfd3700 1 -- 172.21.15.81:0/4081686280 shutdown_connections 2024-05-08T23:32:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f296dfd3700 1 -- 172.21.15.81:0/4081686280 wait complete. 2024-05-08T23:32:26.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f296dfd3700 1 Processor -- start 2024-05-08T23:32:26.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.803+0000 7f296dfd3700 1 -- start start 2024-05-08T23:32:26.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296dfd3700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680ce990 0x7f2968149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296dfd3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f296815afa0 0x7f29681f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296dfd3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f296815f770 0x7f29681476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296dfd3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29681f6de0 con 0x7f296815f770 2024-05-08T23:32:26.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296dfd3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296814a6c0 con 0x7f296815afa0 2024-05-08T23:32:26.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296dfd3700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2968149e20 con 0x7f29680ce990 2024-05-08T23:32:26.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296d7d2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f296815f770 0x7f29681476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296d7d2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f296815f770 0x7f29681476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39662/0 (socket says 172.21.15.81:39662) 2024-05-08T23:32:26.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296d7d2700 1 -- 172.21.15.81:0/1113071284 learned_addr learned my addr 172.21.15.81:0/1113071284 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:26.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296cfd1700 1 --2- 172.21.15.81:0/1113071284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680ce990 0x7f2968149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f2967fff700 1 --2- 172.21.15.81:0/1113071284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f296815afa0 0x7f29681f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296d7d2700 1 -- 172.21.15.81:0/1113071284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680ce990 msgr2=0x7f2968149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296d7d2700 1 --2- 172.21.15.81:0/1113071284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680ce990 0x7f2968149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296d7d2700 1 -- 172.21.15.81:0/1113071284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f296815afa0 msgr2=0x7f29681f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296d7d2700 1 --2- 172.21.15.81:0/1113071284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f296815afa0 0x7f29681f68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296d7d2700 1 -- 172.21.15.81:0/1113071284 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2960009a70 con 0x7f296815f770 2024-05-08T23:32:26.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296d7d2700 1 --2- 172.21.15.81:0/1113071284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f296815f770 0x7f29681476d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f296000efc0 tx=0x7f296000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:26.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f29657fa700 1 -- 172.21.15.81:0/1113071284 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296000cc90 con 0x7f296815f770 2024-05-08T23:32:26.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f29657fa700 1 -- 172.21.15.81:0/1113071284 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296000f040 con 0x7f296815f770 2024-05-08T23:32:26.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f29657fa700 1 -- 172.21.15.81:0/1113071284 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29600146c0 con 0x7f296815f770 2024-05-08T23:32:26.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296dfd3700 1 -- 172.21.15.81:0/1113071284 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29500094b0 con 0x7f296815f770 2024-05-08T23:32:26.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296dfd3700 1 -- 172.21.15.81:0/1113071284 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296814a3e0 con 0x7f296815f770 2024-05-08T23:32:26.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.807+0000 7f296dfd3700 1 -- 172.21.15.81:0/1113071284 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29680684b0 con 0x7f296815f770 2024-05-08T23:32:26.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.811+0000 7f29657fa700 1 -- 172.21.15.81:0/1113071284 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2960014820 con 0x7f296815f770 2024-05-08T23:32:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.811+0000 7f29657fa700 1 --2- 172.21.15.81:0/1113071284 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2948066730 0x7f2948068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.811+0000 7f296cfd1700 1 --2- 172.21.15.81:0/1113071284 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2948066730 0x7f2948068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.811+0000 7f29657fa700 1 -- 172.21.15.81:0/1113071284 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f2960084e10 con 0x7f296815f770 2024-05-08T23:32:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.811+0000 7f296cfd1700 1 --2- 172.21.15.81:0/1113071284 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2948066730 0x7f2948068bf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2958007fd0 tx=0x7f2958007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:26.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.815+0000 7f29657fa700 1 -- 172.21.15.81:0/1113071284 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2960088030 con 0x7f296815f770 2024-05-08T23:32:26.991 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.987+0000 7f296dfd3700 1 -- 172.21.15.81:0/1113071284 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f296806e220 con 0x7f296815f770 2024-05-08T23:32:26.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.991+0000 7f29657fa700 1 -- 172.21.15.81:0/1113071284 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+17880 (secure 0 0 0) 0x7f2960018020 con 0x7f296815f770 2024-05-08T23:32:26.994 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:26.994 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":39,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:48.310750+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:31:35.620423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:31:36.878524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-05-08T23:31:46.337484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:26.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.995+0000 7f296dfd3700 1 -- 172.21.15.81:0/1113071284 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2948066730 msgr2=0x7f2948068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.995+0000 7f296dfd3700 1 --2- 172.21.15.81:0/1113071284 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2948066730 0x7f2948068bf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2958007fd0 tx=0x7f2958007de0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.995+0000 7f296dfd3700 1 -- 172.21.15.81:0/1113071284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f296815f770 msgr2=0x7f29681476d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.995+0000 7f296dfd3700 1 --2- 172.21.15.81:0/1113071284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f296815f770 0x7f29681476d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f296000efc0 tx=0x7f296000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.995+0000 7f296dfd3700 1 -- 172.21.15.81:0/1113071284 shutdown_connections 2024-05-08T23:32:26.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.995+0000 7f296dfd3700 1 --2- 172.21.15.81:0/1113071284 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2948066730 0x7f2948068bf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.995+0000 7f296dfd3700 1 --2- 172.21.15.81:0/1113071284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f296815f770 0x7f29681476d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.995+0000 7f296dfd3700 1 --2- 172.21.15.81:0/1113071284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f296815afa0 0x7f29681f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.995+0000 7f296dfd3700 1 --2- 172.21.15.81:0/1113071284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29680ce990 0x7f2968149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.995+0000 7f296dfd3700 1 -- 172.21.15.81:0/1113071284 >> 172.21.15.81:0/1113071284 conn(0x7f296806c330 msgr2=0x7f29680cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:26.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.995+0000 7f296dfd3700 1 -- 172.21.15.81:0/1113071284 shutdown_connections 2024-05-08T23:32:26.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:26.995+0000 7f296dfd3700 1 -- 172.21.15.81:0/1113071284 wait complete. 2024-05-08T23:32:27.009 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T23:32:27.009 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:27.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 Processor -- start 2024-05-08T23:32:27.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 -- start start 2024-05-08T23:32:27.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f33f80b3c20 0x7f33f81624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f33f8071e70 0x7f33f80b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33f8165070 0x7f33f8165500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f80b1f60 con 0x7f33f80b3c20 2024-05-08T23:32:27.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f8158960 con 0x7f33f8071e70 2024-05-08T23:32:27.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80b4540 con 0x7f33f8165070 2024-05-08T23:32:27.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33f7fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f33f80b3c20 0x7f33f81624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33f7fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f33f80b3c20 0x7f33f81624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39676/0 (socket says 172.21.15.81:39676) 2024-05-08T23:32:27.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33f7fff700 1 -- 172.21.15.81:0/4278300833 learned_addr learned my addr 172.21.15.81:0/4278300833 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:27.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33f77fe700 1 --2- 172.21.15.81:0/4278300833 >> v2:172.21.15.117:3300/0 conn(0x7f33f8071e70 0x7f33f80b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fca02700 1 --2- 172.21.15.81:0/4278300833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33f8165070 0x7f33f8165500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33f7fff700 1 -- 172.21.15.81:0/4278300833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33f8165070 msgr2=0x7f33f8165500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:27.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33f7fff700 1 --2- 172.21.15.81:0/4278300833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33f8165070 0x7f33f8165500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33f7fff700 1 -- 172.21.15.81:0/4278300833 >> v2:172.21.15.117:3300/0 conn(0x7f33f8071e70 msgr2=0x7f33f80b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:27.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33f7fff700 1 --2- 172.21.15.81:0/4278300833 >> v2:172.21.15.117:3300/0 conn(0x7f33f8071e70 0x7f33f80b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33f7fff700 1 -- 172.21.15.81:0/4278300833 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f80ae990 con 0x7f33f80b3c20 2024-05-08T23:32:27.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33f7fff700 1 --2- 172.21.15.81:0/4278300833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f80b3c20 0x7f33f81624b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f33f806b800 tx=0x7f33ec009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3bb20b2b60272 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33f6ffd700 1 -- 172.21.15.81:0/4278300833 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec011830 con 0x7f33f80b3c20 2024-05-08T23:32:27.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33f6ffd700 1 -- 172.21.15.81:0/4278300833 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ec011990 con 0x7f33f80b3c20 2024-05-08T23:32:27.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33f6ffd700 1 -- 172.21.15.81:0/4278300833 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec011d20 con 0x7f33f80b3c20 2024-05-08T23:32:27.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 -- 172.21.15.81:0/4278300833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f80b3c20 msgr2=0x7f33f81624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 --2- 172.21.15.81:0/4278300833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f80b3c20 0x7f33f81624b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f33f806b800 tx=0x7f33ec009bf0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 -- 172.21.15.81:0/4278300833 shutdown_connections 2024-05-08T23:32:27.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 --2- 172.21.15.81:0/4278300833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33f8165070 0x7f33f8165500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 --2- 172.21.15.81:0/4278300833 >> v2:172.21.15.117:3300/0 conn(0x7f33f8071e70 0x7f33f80b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 --2- 172.21.15.81:0/4278300833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f80b3c20 0x7f33f81624b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 -- 172.21.15.81:0/4278300833 >> 172.21.15.81:0/4278300833 conn(0x7f33f806c330 msgr2=0x7f33f806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:27.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 -- 172.21.15.81:0/4278300833 shutdown_connections 2024-05-08T23:32:27.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.123+0000 7f33fd203700 1 -- 172.21.15.81:0/4278300833 wait complete. 2024-05-08T23:32:27.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33fd203700 1 Processor -- start 2024-05-08T23:32:27.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33fd203700 1 -- start start 2024-05-08T23:32:27.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33fd203700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f8071e70 0x7f33f813e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33fd203700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33f80b3c20 0x7f33f80cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33fd203700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33f8165070 0x7f33f80cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33fd203700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f815bc30 con 0x7f33f8071e70 2024-05-08T23:32:27.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33fd203700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f815b0b0 con 0x7f33f80b3c20 2024-05-08T23:32:27.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33fd203700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f815b390 con 0x7f33f8165070 2024-05-08T23:32:27.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33f7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f8071e70 0x7f33f813e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33f7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f8071e70 0x7f33f813e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39692/0 (socket says 172.21.15.81:39692) 2024-05-08T23:32:27.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33f7fff700 1 -- 172.21.15.81:0/117221480 learned_addr learned my addr 172.21.15.81:0/117221480 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:27.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33fca02700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33f8165070 0x7f33f80cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33f77fe700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33f80b3c20 0x7f33f80cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33f7fff700 1 -- 172.21.15.81:0/117221480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33f8165070 msgr2=0x7f33f80cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33f7fff700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33f8165070 0x7f33f80cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33f7fff700 1 -- 172.21.15.81:0/117221480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33f80b3c20 msgr2=0x7f33f80cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33f7fff700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33f80b3c20 0x7f33f80cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33f7fff700 1 -- 172.21.15.81:0/117221480 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec009570 con 0x7f33f8071e70 2024-05-08T23:32:27.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33f77fe700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33f80b3c20 0x7f33f80cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:27.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33fca02700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33f8165070 0x7f33f80cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:27.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33f7fff700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f8071e70 0x7f33f813e5f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f33ec000c00 tx=0x7f33ec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33f4ff9700 1 -- 172.21.15.81:0/117221480 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec004610 con 0x7f33f8071e70 2024-05-08T23:32:27.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33fd203700 1 -- 172.21.15.81:0/117221480 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f815b670 con 0x7f33f8071e70 2024-05-08T23:32:27.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33fd203700 1 -- 172.21.15.81:0/117221480 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f813eb30 con 0x7f33f8071e70 2024-05-08T23:32:27.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33f4ff9700 1 -- 172.21.15.81:0/117221480 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ec01e070 con 0x7f33f8071e70 2024-05-08T23:32:27.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.127+0000 7f33f4ff9700 1 -- 172.21.15.81:0/117221480 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec0038c0 con 0x7f33f8071e70 2024-05-08T23:32:27.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.131+0000 7f33fd203700 1 -- 172.21.15.81:0/117221480 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d8005310 con 0x7f33f8071e70 2024-05-08T23:32:27.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.131+0000 7f33f4ff9700 1 -- 172.21.15.81:0/117221480 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f33ec003a20 con 0x7f33f8071e70 2024-05-08T23:32:27.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.131+0000 7f33f4ff9700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f33e4066730 0x7f33e4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.131+0000 7f33f77fe700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f33e4066730 0x7f33e4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.131+0000 7f33f4ff9700 1 -- 172.21.15.81:0/117221480 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f33ec0851a0 con 0x7f33f8071e70 2024-05-08T23:32:27.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.131+0000 7f33f77fe700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f33e4066730 0x7f33e4068bf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f33f81554f0 tx=0x7f33f000d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.135+0000 7f33f4ff9700 1 -- 172.21.15.81:0/117221480 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f33ec0524b0 con 0x7f33f8071e70 2024-05-08T23:32:27.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.351+0000 7f33fd203700 1 -- 172.21.15.81:0/117221480 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33d80055c0 con 0x7f33f8071e70 2024-05-08T23:32:27.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.351+0000 7f33f4ff9700 1 -- 172.21.15.81:0/117221480 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f33ec055ad0 con 0x7f33f8071e70 2024-05-08T23:32:27.352 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:27.353 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1243037934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1243037934},{"type":"v1","addr":"172.21.15.81:6835","nonce":1243037934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3124115182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3124115182},{"type":"v1","addr":"172.21.15.81:6837","nonce":3124115182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1423349691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":1423349691},{"type":"v1","addr":"172.21.15.117:6834","nonce":1423349691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:31:37.889298+0000","modified":"2024-05-08T23:31:47.307677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14852":{"gid":14852,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/3091184157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3091184157},{"type":"v1","addr":"172.21.15.117:6835","nonce":3091184157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:32:27.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.351+0000 7f33fd203700 1 -- 172.21.15.81:0/117221480 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f33e4066730 msgr2=0x7f33e4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.351+0000 7f33fd203700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f33e4066730 0x7f33e4068bf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f33f81554f0 tx=0x7f33f000d040 comp rx=0 tx=0).stop 2024-05-08T23:32:27.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.351+0000 7f33fd203700 1 -- 172.21.15.81:0/117221480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f8071e70 msgr2=0x7f33f813e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.351+0000 7f33fd203700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f8071e70 0x7f33f813e5f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f33ec000c00 tx=0x7f33ec004770 comp rx=0 tx=0).stop 2024-05-08T23:32:27.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.351+0000 7f33fd203700 1 -- 172.21.15.81:0/117221480 shutdown_connections 2024-05-08T23:32:27.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.351+0000 7f33fd203700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f33e4066730 0x7f33e4068bf0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.351+0000 7f33fd203700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33f8165070 0x7f33f80cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.351+0000 7f33fd203700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33f80b3c20 0x7f33f80cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.351+0000 7f33fd203700 1 --2- 172.21.15.81:0/117221480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f8071e70 0x7f33f813e5f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.351+0000 7f33fd203700 1 -- 172.21.15.81:0/117221480 >> 172.21.15.81:0/117221480 conn(0x7f33f806c330 msgr2=0x7f33f8162c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:27.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.351+0000 7f33fd203700 1 -- 172.21.15.81:0/117221480 shutdown_connections 2024-05-08T23:32:27.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.351+0000 7f33fd203700 1 -- 172.21.15.81:0/117221480 wait complete. 2024-05-08T23:32:27.358 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 15 2024-05-08T23:32:27.367 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:27.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.475+0000 7f9d498d0700 1 Processor -- start 2024-05-08T23:32:27.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.475+0000 7f9d498d0700 1 -- start start 2024-05-08T23:32:27.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.475+0000 7f9d498d0700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9d3c033fb0 0x7f9d3c0eb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.475+0000 7f9d498d0700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d3c04fe20 con 0x7f9d3c048210 2024-05-08T23:32:27.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.475+0000 7f9d498d0700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d3c0ebe70 con 0x7f9d3c047cc0 2024-05-08T23:32:27.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.475+0000 7f9d498d0700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d3c04fb80 con 0x7f9d3c033fb0 2024-05-08T23:32:27.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.475+0000 7f9d43fff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f9d3c048210 0x7f9d3c0eb760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55396/0 (socket says 172.21.15.81:55396) 2024-05-08T23:32:27.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.475+0000 7f9d43fff700 1 -- 172.21.15.81:0/1998942856 learned_addr learned my addr 172.21.15.81:0/1998942856 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:27.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.475+0000 7f9d488ce700 1 --2- 172.21.15.81:0/1998942856 >> v2:172.21.15.81:3300/0 conn(0x7f9d3c033fb0 0x7f9d3c0eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.475+0000 7f9d437fe700 1 -- 172.21.15.81:0/1998942856 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509542439 0 0) 0x7f9d3c04fe20 con 0x7f9d3c048210 2024-05-08T23:32:27.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.475+0000 7f9d437fe700 1 -- 172.21.15.81:0/1998942856 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d340035d0 con 0x7f9d3c048210 2024-05-08T23:32:27.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.475+0000 7f9d488ce700 1 -- 172.21.15.81:0/1998942856 >> v1:172.21.15.117:6789/0 conn(0x7f9d3c047cc0 legacy=0x7f9d3c052f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.475+0000 7f9d488ce700 1 -- 172.21.15.81:0/1998942856 >> v1:172.21.15.81:6789/0 conn(0x7f9d3c048210 legacy=0x7f9d3c0eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.475+0000 7f9d488ce700 1 -- 172.21.15.81:0/1998942856 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c0ec0c0 con 0x7f9d3c033fb0 2024-05-08T23:32:27.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.479+0000 7f9d488ce700 1 --2- 172.21.15.81:0/1998942856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d3c033fb0 0x7f9d3c0eb220 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009850 tx=0x7f9d38009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359b440ff2b696e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.479+0000 7f9d437fe700 1 -- 172.21.15.81:0/1998942856 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38011830 con 0x7f9d3c033fb0 2024-05-08T23:32:27.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.479+0000 7f9d437fe700 1 -- 172.21.15.81:0/1998942856 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d38011990 con 0x7f9d3c033fb0 2024-05-08T23:32:27.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.479+0000 7f9d437fe700 1 -- 172.21.15.81:0/1998942856 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38011d00 con 0x7f9d3c033fb0 2024-05-08T23:32:27.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.479+0000 7f9d498d0700 1 -- 172.21.15.81:0/1998942856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d3c033fb0 msgr2=0x7f9d3c0eb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.479+0000 7f9d498d0700 1 --2- 172.21.15.81:0/1998942856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d3c033fb0 0x7f9d3c0eb220 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009850 tx=0x7f9d38009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:27.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.479+0000 7f9d498d0700 1 -- 172.21.15.81:0/1998942856 shutdown_connections 2024-05-08T23:32:27.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.479+0000 7f9d498d0700 1 --2- 172.21.15.81:0/1998942856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d3c033fb0 0x7f9d3c0eb220 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.479+0000 7f9d498d0700 1 -- 172.21.15.81:0/1998942856 >> 172.21.15.81:0/1998942856 conn(0x7f9d3c054b70 msgr2=0x7f9d3c054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:27.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.479+0000 7f9d498d0700 1 -- 172.21.15.81:0/1998942856 shutdown_connections 2024-05-08T23:32:27.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.479+0000 7f9d498d0700 1 -- 172.21.15.81:0/1998942856 wait complete. 2024-05-08T23:32:27.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.479+0000 7f9d498d0700 1 Processor -- start 2024-05-08T23:32:27.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.479+0000 7f9d498d0700 1 -- start start 2024-05-08T23:32:27.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d498d0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d3c033fb0 0x7f9d3c049870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d498d0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d3c047cc0 0x7f9d3c049db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d498d0700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d3c048210 0x7f9d3c0e8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d498d0700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c0487d0 con 0x7f9d3c047cc0 2024-05-08T23:32:27.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d498d0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c048ab0 con 0x7f9d3c033fb0 2024-05-08T23:32:27.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d498d0700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c048d90 con 0x7f9d3c048210 2024-05-08T23:32:27.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d43fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d3c047cc0 0x7f9d3c049db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d43fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d3c047cc0 0x7f9d3c049db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39700/0 (socket says 172.21.15.81:39700) 2024-05-08T23:32:27.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d43fff700 1 -- 172.21.15.81:0/384110525 learned_addr learned my addr 172.21.15.81:0/384110525 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:27.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d488ce700 1 --2- 172.21.15.81:0/384110525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d3c033fb0 0x7f9d3c049870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d490cf700 1 --2- 172.21.15.81:0/384110525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d3c048210 0x7f9d3c0e8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d43fff700 1 -- 172.21.15.81:0/384110525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d3c048210 msgr2=0x7f9d3c0e8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d43fff700 1 --2- 172.21.15.81:0/384110525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d3c048210 0x7f9d3c0e8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d43fff700 1 -- 172.21.15.81:0/384110525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d3c033fb0 msgr2=0x7f9d3c049870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d43fff700 1 --2- 172.21.15.81:0/384110525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d3c033fb0 0x7f9d3c049870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d43fff700 1 -- 172.21.15.81:0/384110525 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d380094b0 con 0x7f9d3c047cc0 2024-05-08T23:32:27.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d43fff700 1 --2- 172.21.15.81:0/384110525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d3c047cc0 0x7f9d3c049db0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9d300010e0 tx=0x7f9d3000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d417fa700 1 -- 172.21.15.81:0/384110525 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30005d90 con 0x7f9d3c047cc0 2024-05-08T23:32:27.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d417fa700 1 -- 172.21.15.81:0/384110525 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3000ce40 con 0x7f9d3c047cc0 2024-05-08T23:32:27.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d498d0700 1 -- 172.21.15.81:0/384110525 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3c049070 con 0x7f9d3c047cc0 2024-05-08T23:32:27.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d417fa700 1 -- 172.21.15.81:0/384110525 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d300072f0 con 0x7f9d3c047cc0 2024-05-08T23:32:27.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.483+0000 7f9d498d0700 1 -- 172.21.15.81:0/384110525 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3c0e94a0 con 0x7f9d3c047cc0 2024-05-08T23:32:27.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.487+0000 7f9d417fa700 1 -- 172.21.15.81:0/384110525 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9d30007450 con 0x7f9d3c047cc0 2024-05-08T23:32:27.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.487+0000 7f9d498d0700 1 -- 172.21.15.81:0/384110525 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d3c050290 con 0x7f9d3c047cc0 2024-05-08T23:32:27.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.487+0000 7f9d417fa700 1 --2- 172.21.15.81:0/384110525 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9d28066730 0x7f9d28068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.487+0000 7f9d417fa700 1 -- 172.21.15.81:0/384110525 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f9d30083640 con 0x7f9d3c047cc0 2024-05-08T23:32:27.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.487+0000 7f9d488ce700 1 --2- 172.21.15.81:0/384110525 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9d28066730 0x7f9d28068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.487+0000 7f9d488ce700 1 --2- 172.21.15.81:0/384110525 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9d28066730 0x7f9d28068bf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9d38000c00 tx=0x7f9d38008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.495+0000 7f9d417fa700 1 -- 172.21.15.81:0/384110525 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9d3c050290 con 0x7f9d3c047cc0 2024-05-08T23:32:27.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.695+0000 7f9d498d0700 1 -- 172.21.15.81:0/384110525 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d3c050290 con 0x7f9d3c047cc0 2024-05-08T23:32:27.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.695+0000 7f9d417fa700 1 -- 172.21.15.81:0/384110525 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f9d3c050290 con 0x7f9d3c047cc0 2024-05-08T23:32:27.699 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:27.700 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1243037934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1243037934},{"type":"v1","addr":"172.21.15.81:6835","nonce":1243037934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3124115182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3124115182},{"type":"v1","addr":"172.21.15.81:6837","nonce":3124115182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1423349691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":1423349691},{"type":"v1","addr":"172.21.15.117:6834","nonce":1423349691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:31:37.889298+0000","modified":"2024-05-08T23:31:47.307677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14852":{"gid":14852,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/3091184157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3091184157},{"type":"v1","addr":"172.21.15.117:6835","nonce":3091184157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-08T23:32:27.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.699+0000 7f9d498d0700 1 -- 172.21.15.81:0/384110525 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9d28066730 msgr2=0x7f9d28068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.699+0000 7f9d498d0700 1 --2- 172.21.15.81:0/384110525 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9d28066730 0x7f9d28068bf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9d38000c00 tx=0x7f9d38008030 comp rx=0 tx=0).stop 2024-05-08T23:32:27.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.699+0000 7f9d498d0700 1 -- 172.21.15.81:0/384110525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d3c047cc0 msgr2=0x7f9d3c049db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.699+0000 7f9d498d0700 1 --2- 172.21.15.81:0/384110525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d3c047cc0 0x7f9d3c049db0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9d300010e0 tx=0x7f9d3000c7b0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.699+0000 7f9d498d0700 1 -- 172.21.15.81:0/384110525 shutdown_connections 2024-05-08T23:32:27.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.699+0000 7f9d498d0700 1 --2- 172.21.15.81:0/384110525 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9d28066730 0x7f9d28068bf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.699+0000 7f9d498d0700 1 --2- 172.21.15.81:0/384110525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d3c048210 0x7f9d3c0e8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.699+0000 7f9d498d0700 1 --2- 172.21.15.81:0/384110525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d3c047cc0 0x7f9d3c049db0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.699+0000 7f9d498d0700 1 --2- 172.21.15.81:0/384110525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d3c033fb0 0x7f9d3c049870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.699+0000 7f9d498d0700 1 -- 172.21.15.81:0/384110525 >> 172.21.15.81:0/384110525 conn(0x7f9d3c054b70 msgr2=0x7f9d3c0070a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:27.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.699+0000 7f9d498d0700 1 -- 172.21.15.81:0/384110525 shutdown_connections 2024-05-08T23:32:27.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.699+0000 7f9d498d0700 1 -- 172.21.15.81:0/384110525 wait complete. 2024-05-08T23:32:27.703 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 15 2024-05-08T23:32:27.714 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:27.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.827+0000 7f191cfec700 1 Processor -- start 2024-05-08T23:32:27.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.827+0000 7f191cfec700 1 -- start start 2024-05-08T23:32:27.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f191cfec700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f19100df7a0 0x7f1910054d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f191cfec700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1910053ce0 con 0x7f191003ae00 2024-05-08T23:32:27.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f191cfec700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1910053fc0 con 0x7f1910050140 2024-05-08T23:32:27.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f191cfec700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19100dfb80 con 0x7f19100df7a0 2024-05-08T23:32:27.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f19177fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f19100df7a0 0x7f1910054d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f19177fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f19100df7a0 0x7f1910054d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39716/0 (socket says 172.21.15.81:39716) 2024-05-08T23:32:27.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f19177fe700 1 -- 172.21.15.81:0/2627223185 learned_addr learned my addr 172.21.15.81:0/2627223185 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:27.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f19177fe700 1 -- 172.21.15.81:0/2627223185 >> v1:172.21.15.117:6789/0 conn(0x7f1910050140 legacy=0x7f1910050550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f19177fe700 1 -- 172.21.15.81:0/2627223185 >> v1:172.21.15.81:6789/0 conn(0x7f191003ae00 legacy=0x7f19100534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f19177fe700 1 -- 172.21.15.81:0/2627223185 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19100542a0 con 0x7f19100df7a0 2024-05-08T23:32:27.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f19177fe700 1 --2- 172.21.15.81:0/2627223185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19100df7a0 0x7f1910054d80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f190c009910 tx=0x7f190c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c1c423b9649eac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f19167fc700 1 -- 172.21.15.81:0/2627223185 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c00ba60 con 0x7f19100df7a0 2024-05-08T23:32:27.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f19167fc700 1 -- 172.21.15.81:0/2627223185 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190c00bbc0 con 0x7f19100df7a0 2024-05-08T23:32:27.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f19167fc700 1 -- 172.21.15.81:0/2627223185 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c009e60 con 0x7f19100df7a0 2024-05-08T23:32:27.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f191cfec700 1 -- 172.21.15.81:0/2627223185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19100df7a0 msgr2=0x7f1910054d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f191cfec700 1 --2- 172.21.15.81:0/2627223185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19100df7a0 0x7f1910054d80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f190c009910 tx=0x7f190c009c20 comp rx=0 tx=0).stop 2024-05-08T23:32:27.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f191cfec700 1 -- 172.21.15.81:0/2627223185 shutdown_connections 2024-05-08T23:32:27.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f191cfec700 1 --2- 172.21.15.81:0/2627223185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19100df7a0 0x7f1910054d80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f191cfec700 1 -- 172.21.15.81:0/2627223185 >> 172.21.15.81:0/2627223185 conn(0x7f1910055400 msgr2=0x7f1910055810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:27.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f191cfec700 1 -- 172.21.15.81:0/2627223185 shutdown_connections 2024-05-08T23:32:27.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.831+0000 7f191cfec700 1 -- 172.21.15.81:0/2627223185 wait complete. 2024-05-08T23:32:27.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f191cfec700 1 Processor -- start 2024-05-08T23:32:27.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f191cfec700 1 -- start start 2024-05-08T23:32:27.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f191cfec700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f191003ae00 0x7f19100e06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f191cfec700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1910050140 0x7f1910038810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f191cfec700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19100df7a0 0x7f19100d60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f191cfec700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19100e1f90 con 0x7f1910050140 2024-05-08T23:32:27.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f191cfec700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19100e0c10 con 0x7f19100df7a0 2024-05-08T23:32:27.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f191cfec700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19100e1380 con 0x7f191003ae00 2024-05-08T23:32:27.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f1916ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1910050140 0x7f1910038810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f1916ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1910050140 0x7f1910038810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39722/0 (socket says 172.21.15.81:39722) 2024-05-08T23:32:27.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f1916ffd700 1 -- 172.21.15.81:0/1340315766 learned_addr learned my addr 172.21.15.81:0/1340315766 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:27.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f19177fe700 1 --2- 172.21.15.81:0/1340315766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f191003ae00 0x7f19100e06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f1917fff700 1 --2- 172.21.15.81:0/1340315766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19100df7a0 0x7f19100d60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f1916ffd700 1 -- 172.21.15.81:0/1340315766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f191003ae00 msgr2=0x7f19100e06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f1916ffd700 1 --2- 172.21.15.81:0/1340315766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f191003ae00 0x7f19100e06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f1916ffd700 1 -- 172.21.15.81:0/1340315766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19100df7a0 msgr2=0x7f19100d60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f1916ffd700 1 --2- 172.21.15.81:0/1340315766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19100df7a0 0x7f19100d60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.835+0000 7f1916ffd700 1 -- 172.21.15.81:0/1340315766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c009570 con 0x7f1910050140 2024-05-08T23:32:27.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.839+0000 7f1916ffd700 1 --2- 172.21.15.81:0/1340315766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1910050140 0x7f1910038810 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f190400c4b0 tx=0x7f190400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.839+0000 7f1903fff700 1 -- 172.21.15.81:0/1340315766 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19040052d0 con 0x7f1910050140 2024-05-08T23:32:27.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.839+0000 7f1903fff700 1 -- 172.21.15.81:0/1340315766 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1904005430 con 0x7f1910050140 2024-05-08T23:32:27.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.839+0000 7f191cfec700 1 -- 172.21.15.81:0/1340315766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19100e1660 con 0x7f1910050140 2024-05-08T23:32:27.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.839+0000 7f1903fff700 1 -- 172.21.15.81:0/1340315766 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19040055a0 con 0x7f1910050140 2024-05-08T23:32:27.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.839+0000 7f191cfec700 1 -- 172.21.15.81:0/1340315766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19100e1c20 con 0x7f1910050140 2024-05-08T23:32:27.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.839+0000 7f191cfec700 1 -- 172.21.15.81:0/1340315766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19100e2140 con 0x7f1910050140 2024-05-08T23:32:27.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.839+0000 7f1903fff700 1 -- 172.21.15.81:0/1340315766 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f19040036a0 con 0x7f1910050140 2024-05-08T23:32:27.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.839+0000 7f1903fff700 1 --2- 172.21.15.81:0/1340315766 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f18fc066730 0x7f18fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.839+0000 7f1903fff700 1 -- 172.21.15.81:0/1340315766 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f19040834b0 con 0x7f1910050140 2024-05-08T23:32:27.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.839+0000 7f19177fe700 1 --2- 172.21.15.81:0/1340315766 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f18fc066730 0x7f18fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.839+0000 7f19177fe700 1 --2- 172.21.15.81:0/1340315766 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f18fc066730 0x7f18fc068bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f190c014040 tx=0x7f190c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:27.847+0000 7f1903fff700 1 -- 172.21.15.81:0/1340315766 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f19100e2140 con 0x7f1910050140 2024-05-08T23:32:28.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.015+0000 7f191cfec700 1 -- 172.21.15.81:0/1340315766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1910005890 con 0x7f1910050140 2024-05-08T23:32:28.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.019+0000 7f1903fff700 1 -- 172.21.15.81:0/1340315766 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+17880 (secure 0 0 0) 0x7f1904050610 con 0x7f1910050140 2024-05-08T23:32:28.021 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:28.022 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":39,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:31:48.310750+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:31:35.620423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-08T23:31:36.878524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-05-08T23:31:46.337484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:28.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.023+0000 7f191cfec700 1 -- 172.21.15.81:0/1340315766 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f18fc066730 msgr2=0x7f18fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:28.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.023+0000 7f191cfec700 1 --2- 172.21.15.81:0/1340315766 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f18fc066730 0x7f18fc068bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f190c014040 tx=0x7f190c008030 comp rx=0 tx=0).stop 2024-05-08T23:32:28.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.023+0000 7f191cfec700 1 -- 172.21.15.81:0/1340315766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1910050140 msgr2=0x7f1910038810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:28.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.023+0000 7f191cfec700 1 --2- 172.21.15.81:0/1340315766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1910050140 0x7f1910038810 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f190400c4b0 tx=0x7f190400c7c0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.023+0000 7f191cfec700 1 -- 172.21.15.81:0/1340315766 shutdown_connections 2024-05-08T23:32:28.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.023+0000 7f191cfec700 1 --2- 172.21.15.81:0/1340315766 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f18fc066730 0x7f18fc068bf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.023+0000 7f191cfec700 1 --2- 172.21.15.81:0/1340315766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19100df7a0 0x7f19100d60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.023+0000 7f191cfec700 1 --2- 172.21.15.81:0/1340315766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1910050140 0x7f1910038810 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.023+0000 7f191cfec700 1 --2- 172.21.15.81:0/1340315766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f191003ae00 0x7f19100e06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.023+0000 7f191cfec700 1 -- 172.21.15.81:0/1340315766 >> 172.21.15.81:0/1340315766 conn(0x7f1910055400 msgr2=0x7f1910048f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:28.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.023+0000 7f191cfec700 1 -- 172.21.15.81:0/1340315766 shutdown_connections 2024-05-08T23:32:28.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.023+0000 7f191cfec700 1 -- 172.21.15.81:0/1340315766 wait complete. 2024-05-08T23:32:28.040 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T23:32:28.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc307fff700 1 Processor -- start 2024-05-08T23:32:28.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc307fff700 1 -- start start 2024-05-08T23:32:28.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc307fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc3081624b0 0x7fc308070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:28.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc307fff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc308158960 con 0x7fc3080b3c20 2024-05-08T23:32:28.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc307fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3080ae990 con 0x7fc308071e70 2024-05-08T23:32:28.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc307fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3080b1f60 con 0x7fc3081624b0 2024-05-08T23:32:28.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc306ffd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc3080b3c20 0x7fc3080b4000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55420/0 (socket says 172.21.15.81:55420) 2024-05-08T23:32:28.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc306ffd700 1 -- 172.21.15.81:0/1354924001 learned_addr learned my addr 172.21.15.81:0/1354924001 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:28.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc3077fe700 1 --2- 172.21.15.81:0/1354924001 >> v2:172.21.15.117:3300/0 conn(0x7fc3081624b0 0x7fc308070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:28.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc305ffb700 1 -- 172.21.15.81:0/1354924001 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723754000 0 0) 0x7fc308158960 con 0x7fc3080b3c20 2024-05-08T23:32:28.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc305ffb700 1 -- 172.21.15.81:0/1354924001 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2f00035d0 con 0x7fc3080b3c20 2024-05-08T23:32:28.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc3077fe700 1 -- 172.21.15.81:0/1354924001 >> v1:172.21.15.117:6789/0 conn(0x7fc308071e70 legacy=0x7fc308070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc3077fe700 1 -- 172.21.15.81:0/1354924001 >> v1:172.21.15.81:6789/0 conn(0x7fc3080b3c20 legacy=0x7fc3080b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc305ffb700 1 -- 172.21.15.81:0/1354924001 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3340421569 0 0) 0x7fc2f00035d0 con 0x7fc3080b3c20 2024-05-08T23:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc3077fe700 1 -- 172.21.15.81:0/1354924001 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3080b2170 con 0x7fc3081624b0 2024-05-08T23:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc3077fe700 1 --2- 172.21.15.81:0/1354924001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3081624b0 0x7fc308070820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009850 tx=0x7fc2f8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c76eae7dc94c05d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc305ffb700 1 -- 172.21.15.81:0/1354924001 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc001670 con 0x7fc3081624b0 2024-05-08T23:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc305ffb700 1 -- 172.21.15.81:0/1354924001 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f8004030 con 0x7fc3081624b0 2024-05-08T23:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc305ffb700 1 -- 172.21.15.81:0/1354924001 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80043c0 con 0x7fc3081624b0 2024-05-08T23:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc307fff700 1 -- 172.21.15.81:0/1354924001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3081624b0 msgr2=0x7fc308070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.155+0000 7fc307fff700 1 --2- 172.21.15.81:0/1354924001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3081624b0 0x7fc308070820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009850 tx=0x7fc2f8009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc307fff700 1 -- 172.21.15.81:0/1354924001 shutdown_connections 2024-05-08T23:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc307fff700 1 --2- 172.21.15.81:0/1354924001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3081624b0 0x7fc308070820 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc307fff700 1 -- 172.21.15.81:0/1354924001 >> 172.21.15.81:0/1354924001 conn(0x7fc30806c330 msgr2=0x7fc30806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc307fff700 1 -- 172.21.15.81:0/1354924001 shutdown_connections 2024-05-08T23:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc307fff700 1 -- 172.21.15.81:0/1354924001 wait complete. 2024-05-08T23:32:28.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc307fff700 1 Processor -- start 2024-05-08T23:32:28.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc307fff700 1 -- start start 2024-05-08T23:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc307fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc308071e70 0x7fc3081f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc307fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3080b3c20 0x7fc3081fa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc307fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3081624b0 0x7fc3081fa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc307fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30814dd00 con 0x7fc3080b3c20 2024-05-08T23:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc307fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc308163a80 con 0x7fc3081624b0 2024-05-08T23:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc307fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc308163d60 con 0x7fc308071e70 2024-05-08T23:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc3067fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3080b3c20 0x7fc3081fa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc3067fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3080b3c20 0x7fc3081fa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39734/0 (socket says 172.21.15.81:39734) 2024-05-08T23:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc3067fc700 1 -- 172.21.15.81:0/2225117912 learned_addr learned my addr 172.21.15.81:0/2225117912 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc3077fe700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3081624b0 0x7fc3081fa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc306ffd700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc308071e70 0x7fc3081f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc3067fc700 1 -- 172.21.15.81:0/2225117912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc308071e70 msgr2=0x7fc3081f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc3067fc700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc308071e70 0x7fc3081f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc3067fc700 1 -- 172.21.15.81:0/2225117912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3081624b0 msgr2=0x7fc3081fa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc3067fc700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3081624b0 0x7fc3081fa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc3067fc700 1 -- 172.21.15.81:0/2225117912 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f80094b0 con 0x7fc3080b3c20 2024-05-08T23:32:28.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc3077fe700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3081624b0 0x7fc3081fa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:28.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc306ffd700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc308071e70 0x7fc3081f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:28.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc3067fc700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3080b3c20 0x7fc3081fa490 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc30000cbd0 tx=0x7fc30000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:28.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc2effff700 1 -- 172.21.15.81:0/2225117912 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300003c20 con 0x7fc3080b3c20 2024-05-08T23:32:28.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc2effff700 1 -- 172.21.15.81:0/2225117912 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc300003d80 con 0x7fc3080b3c20 2024-05-08T23:32:28.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc307fff700 1 -- 172.21.15.81:0/2225117912 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc308164040 con 0x7fc3080b3c20 2024-05-08T23:32:28.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc2effff700 1 -- 172.21.15.81:0/2225117912 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30000abf0 con 0x7fc3080b3c20 2024-05-08T23:32:28.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.159+0000 7fc307fff700 1 -- 172.21.15.81:0/2225117912 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3081f6c40 con 0x7fc3080b3c20 2024-05-08T23:32:28.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.163+0000 7fc307fff700 1 -- 172.21.15.81:0/2225117912 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3081f6dd0 con 0x7fc3080b3c20 2024-05-08T23:32:28.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.163+0000 7fc2effff700 1 -- 172.21.15.81:0/2225117912 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc30000ad50 con 0x7fc3080b3c20 2024-05-08T23:32:28.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.163+0000 7fc2effff700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc2f4066730 0x7fc2f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:28.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.163+0000 7fc306ffd700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc2f4066730 0x7fc2f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:28.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.163+0000 7fc2effff700 1 -- 172.21.15.81:0/2225117912 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fc300084480 con 0x7fc3080b3c20 2024-05-08T23:32:28.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.163+0000 7fc306ffd700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc2f4066730 0x7fc2f4068bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc001eb0 tx=0x7fc2fc001c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:28.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.171+0000 7fc2effff700 1 -- 172.21.15.81:0/2225117912 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc3000515d0 con 0x7fc3080b3c20 2024-05-08T23:32:28.377 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.375+0000 7fc307fff700 1 -- 172.21.15.81:0/2225117912 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc308066990 con 0x7fc3080b3c20 2024-05-08T23:32:28.378 INFO:tasks.ceph.mon.a.smithi081.stderr:2024-05-08T23:32:28.375+0000 7f7c02c75700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:32:28.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.395+0000 7fc2effff700 1 -- 172.21.15.81:0/2225117912 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc308066990 con 0x7fc3080b3c20 2024-05-08T23:32:28.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.399+0000 7fc307fff700 1 -- 172.21.15.81:0/2225117912 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc2f4066730 msgr2=0x7fc2f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:28.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.399+0000 7fc307fff700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc2f4066730 0x7fc2f4068bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc001eb0 tx=0x7fc2fc001c60 comp rx=0 tx=0).stop 2024-05-08T23:32:28.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.399+0000 7fc307fff700 1 -- 172.21.15.81:0/2225117912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3080b3c20 msgr2=0x7fc3081fa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:28.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.399+0000 7fc307fff700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3080b3c20 0x7fc3081fa490 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc30000cbd0 tx=0x7fc30000cf90 comp rx=0 tx=0).stop 2024-05-08T23:32:28.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.399+0000 7fc307fff700 1 -- 172.21.15.81:0/2225117912 shutdown_connections 2024-05-08T23:32:28.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.399+0000 7fc307fff700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc2f4066730 0x7fc2f4068bf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.399+0000 7fc307fff700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3081624b0 0x7fc3081fa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.399+0000 7fc307fff700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3080b3c20 0x7fc3081fa490 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.399+0000 7fc307fff700 1 --2- 172.21.15.81:0/2225117912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc308071e70 0x7fc3081f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.399+0000 7fc307fff700 1 -- 172.21.15.81:0/2225117912 >> 172.21.15.81:0/2225117912 conn(0x7fc30806c330 msgr2=0x7fc3081574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:28.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.399+0000 7fc307fff700 1 -- 172.21.15.81:0/2225117912 shutdown_connections 2024-05-08T23:32:28.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.399+0000 7fc307fff700 1 -- 172.21.15.81:0/2225117912 wait complete. 2024-05-08T23:32:28.401 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T23:32:28.401 INFO:tasks.ceph.mds.d.smithi117.stderr: -3773> 2024-05-08T23:31:41.465+0000 7fa112c82700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:32:28.414 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T23:32:28.435 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-05-08T23:32:28.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf8d1a700 1 Processor -- start 2024-05-08T23:32:28.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf8d1a700 1 -- start start 2024-05-08T23:32:28.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf8d1a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8cf40b1f30 0x7f8cf406af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:28.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf8d1a700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf415d850 con 0x7f8cf415d470 2024-05-08T23:32:28.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf8d1a700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf40d00a0 con 0x7f8cf40d0e40 2024-05-08T23:32:28.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf8d1a700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cf40d0380 con 0x7f8cf40b1f30 2024-05-08T23:32:28.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf37fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f8cf415d470 0x7f8cf40ce7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55428/0 (socket says 172.21.15.81:55428) 2024-05-08T23:32:28.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf37fe700 1 -- 172.21.15.81:0/1310075737 learned_addr learned my addr 172.21.15.81:0/1310075737 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8ceaffd700 1 --2- 172.21.15.81:0/1310075737 >> v2:172.21.15.81:3300/0 conn(0x7f8cf40b1f30 0x7f8cf406af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167984338 0 0) 0x7f8cf415d850 con 0x7f8cf415d470 2024-05-08T23:32:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ce00035d0 con 0x7f8cf415d470 2024-05-08T23:32:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976955900 0 0) 0x7f8cf40d00a0 con 0x7f8cf40d0e40 2024-05-08T23:32:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cf415d850 con 0x7f8cf40d0e40 2024-05-08T23:32:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 760594579 0 0) 0x7f8ce00035d0 con 0x7f8cf415d470 2024-05-08T23:32:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 >> v2:172.21.15.81:3300/0 conn(0x7f8cf40b1f30 msgr2=0x7f8cf406af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 --2- 172.21.15.81:0/1310075737 >> v2:172.21.15.81:3300/0 conn(0x7f8cf40b1f30 0x7f8cf406af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 >> v1:172.21.15.117:6789/0 conn(0x7f8cf40d0e40 legacy=0x7f8cf40d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf40cdfc0 con 0x7f8cf415d470 2024-05-08T23:32:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f8ce4001670 con 0x7f8cf415d470 2024-05-08T23:32:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 >> v1:172.21.15.81:6789/0 conn(0x7f8cf415d470 legacy=0x7f8cf40ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 --2- 172.21.15.81:0/1310075737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ce0003bb0 0x7f8ce0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 --2- 172.21.15.81:0/1310075737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ce0004870 0x7f8ce0004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:28.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 --2- 172.21.15.81:0/1310075737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce0005230 0x7f8ce000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:28.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf40d0380 con 0x7f8ce0003bb0 2024-05-08T23:32:28.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce000c0b0 con 0x7f8ce0005230 2024-05-08T23:32:28.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce000c270 con 0x7f8ce0004870 2024-05-08T23:32:28.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8cf37fe700 1 --2- 172.21.15.81:0/1310075737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ce0003bb0 0x7f8ce0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:28.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.535+0000 7f8ceaffd700 1 --2- 172.21.15.81:0/1310075737 >> v2:172.21.15.81:3300/0 conn(0x7f8cf40b1f30 0x7f8cf406af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:28.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf3fff700 1 --2- 172.21.15.81:0/1310075737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce0005230 0x7f8ce000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:28.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf37fe700 1 -- 172.21.15.81:0/1310075737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ce0004870 msgr2=0x7f8ce0004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:28.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf37fe700 1 --2- 172.21.15.81:0/1310075737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ce0004870 0x7f8ce0004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf37fe700 1 -- 172.21.15.81:0/1310075737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce0005230 msgr2=0x7f8ce000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:28.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf37fe700 1 --2- 172.21.15.81:0/1310075737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce0005230 0x7f8ce000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf37fe700 1 -- 172.21.15.81:0/1310075737 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce00035d0 con 0x7f8ce0003bb0 2024-05-08T23:32:28.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf3fff700 1 -- 172.21.15.81:0/1310075737 reap_dead start 2024-05-08T23:32:28.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf37fe700 1 --2- 172.21.15.81:0/1310075737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ce0003bb0 0x7f8ce0003fb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4007fd0 tx=0x7f8ce00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec67943356caf604 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:28.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400cd80 con 0x7f8ce0003bb0 2024-05-08T23:32:28.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce4001af0 con 0x7f8ce0003bb0 2024-05-08T23:32:28.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf2ffd700 1 -- 172.21.15.81:0/1310075737 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4005bc0 con 0x7f8ce0003bb0 2024-05-08T23:32:28.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf8d1a700 1 -- 172.21.15.81:0/1310075737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ce0003bb0 msgr2=0x7f8ce0003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:28.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf8d1a700 1 --2- 172.21.15.81:0/1310075737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ce0003bb0 0x7f8ce0003fb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4007fd0 tx=0x7f8ce00058a0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf8d1a700 1 -- 172.21.15.81:0/1310075737 shutdown_connections 2024-05-08T23:32:28.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf8d1a700 1 --2- 172.21.15.81:0/1310075737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ce0003bb0 0x7f8ce0003fb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf8d1a700 1 -- 172.21.15.81:0/1310075737 >> 172.21.15.81:0/1310075737 conn(0x7f8cf409c370 msgr2=0x7f8cf40d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:28.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf8d1a700 1 -- 172.21.15.81:0/1310075737 shutdown_connections 2024-05-08T23:32:28.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.539+0000 7f8cf8d1a700 1 -- 172.21.15.81:0/1310075737 wait complete. 2024-05-08T23:32:28.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf8d1a700 1 Processor -- start 2024-05-08T23:32:28.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf8d1a700 1 -- start start 2024-05-08T23:32:28.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf8d1a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ce0003bb0 0x7f8cf415da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:28.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf8d1a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf415b3d0 0x7f8cf41522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:28.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf8d1a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cf40c8480 0x7f8cf40c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:28.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf8d1a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf40c8e40 con 0x7f8cf40c8480 2024-05-08T23:32:28.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf8d1a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf415e340 con 0x7f8cf415b3d0 2024-05-08T23:32:28.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf8d1a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf415e620 con 0x7f8ce0003bb0 2024-05-08T23:32:28.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cf40c8480 0x7f8cf40c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:28.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cf40c8480 0x7f8cf40c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39762/0 (socket says 172.21.15.81:39762) 2024-05-08T23:32:28.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf3fff700 1 -- 172.21.15.81:0/2194085724 learned_addr learned my addr 172.21.15.81:0/2194085724 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:28.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf37fe700 1 --2- 172.21.15.81:0/2194085724 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ce0003bb0 0x7f8cf415da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:28.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8ceaffd700 1 --2- 172.21.15.81:0/2194085724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf415b3d0 0x7f8cf41522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:28.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf3fff700 1 -- 172.21.15.81:0/2194085724 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ce0003bb0 msgr2=0x7f8cf415da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:28.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf3fff700 1 --2- 172.21.15.81:0/2194085724 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ce0003bb0 0x7f8cf415da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf3fff700 1 -- 172.21.15.81:0/2194085724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf415b3d0 msgr2=0x7f8cf41522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:28.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf3fff700 1 --2- 172.21.15.81:0/2194085724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf415b3d0 0x7f8cf41522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:28.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.543+0000 7f8cf3fff700 1 -- 172.21.15.81:0/2194085724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf415e900 con 0x7f8cf40c8480 2024-05-08T23:32:28.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.547+0000 7f8cf3fff700 1 --2- 172.21.15.81:0/2194085724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cf40c8480 0x7f8cf40c8900 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40cf2f0 tx=0x7f8cec00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:28.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.547+0000 7f8cf0ff9700 1 -- 172.21.15.81:0/2194085724 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec003c30 con 0x7f8cf40c8480 2024-05-08T23:32:28.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.547+0000 7f8cf0ff9700 1 -- 172.21.15.81:0/2194085724 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cec003db0 con 0x7f8cf40c8480 2024-05-08T23:32:28.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.547+0000 7f8cf8d1a700 1 -- 172.21.15.81:0/2194085724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf415ebe0 con 0x7f8cf40c8480 2024-05-08T23:32:28.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.547+0000 7f8cf0ff9700 1 -- 172.21.15.81:0/2194085724 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec005310 con 0x7f8cf40c8480 2024-05-08T23:32:28.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.547+0000 7f8cf8d1a700 1 -- 172.21.15.81:0/2194085724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf415f3e0 con 0x7f8cf40c8480 2024-05-08T23:32:28.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.547+0000 7f8cf8d1a700 1 -- 172.21.15.81:0/2194085724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf415f100 con 0x7f8cf40c8480 2024-05-08T23:32:28.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.547+0000 7f8cf0ff9700 1 -- 172.21.15.81:0/2194085724 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8cec005470 con 0x7f8cf40c8480 2024-05-08T23:32:28.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.547+0000 7f8cf0ff9700 1 --2- 172.21.15.81:0/2194085724 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8cd4066730 0x7f8cd4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:28.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.547+0000 7f8cf0ff9700 1 -- 172.21.15.81:0/2194085724 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6190+0+0 (secure 0 0 0) 0x7f8cec084260 con 0x7f8cf40c8480 2024-05-08T23:32:28.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.547+0000 7f8cf37fe700 1 --2- 172.21.15.81:0/2194085724 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8cd4066730 0x7f8cd4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:28.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.551+0000 7f8cf37fe700 1 --2- 172.21.15.81:0/2194085724 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8cd4066730 0x7f8cd4068bf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4000c00 tx=0x7f8ce400a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:28.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.559+0000 7f8cf0ff9700 1 -- 172.21.15.81:0/2194085724 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8cf415f100 con 0x7f8cf40c8480 2024-05-08T23:32:28.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:28.763+0000 7f8cf8d1a700 1 -- 172.21.15.81:0/2194085724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8cf40d2b70 con 0x7f8cf40c8480 2024-05-08T23:32:29.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.399+0000 7f8cf0ff9700 1 -- 172.21.15.81:0/2194085724 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8cec0513f0 con 0x7f8cf40c8480 2024-05-08T23:32:29.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.403+0000 7f8cf8d1a700 1 -- 172.21.15.81:0/2194085724 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8cd4066730 msgr2=0x7f8cd4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:29.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.403+0000 7f8cf8d1a700 1 --2- 172.21.15.81:0/2194085724 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8cd4066730 0x7f8cd4068bf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4000c00 tx=0x7f8ce400a000 comp rx=0 tx=0).stop 2024-05-08T23:32:29.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.403+0000 7f8cf8d1a700 1 -- 172.21.15.81:0/2194085724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cf40c8480 msgr2=0x7f8cf40c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:29.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.403+0000 7f8cf8d1a700 1 --2- 172.21.15.81:0/2194085724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cf40c8480 0x7f8cf40c8900 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40cf2f0 tx=0x7f8cec00edd0 comp rx=0 tx=0).stop 2024-05-08T23:32:29.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.403+0000 7f8cf8d1a700 1 -- 172.21.15.81:0/2194085724 shutdown_connections 2024-05-08T23:32:29.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.403+0000 7f8cf8d1a700 1 --2- 172.21.15.81:0/2194085724 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8cd4066730 0x7f8cd4068bf0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:29.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.403+0000 7f8cf8d1a700 1 --2- 172.21.15.81:0/2194085724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cf40c8480 0x7f8cf40c8900 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:29.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.403+0000 7f8cf8d1a700 1 --2- 172.21.15.81:0/2194085724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf415b3d0 0x7f8cf41522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:29.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.403+0000 7f8cf8d1a700 1 --2- 172.21.15.81:0/2194085724 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ce0003bb0 0x7f8cf415da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:29.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.403+0000 7f8cf8d1a700 1 -- 172.21.15.81:0/2194085724 >> 172.21.15.81:0/2194085724 conn(0x7f8cf409c370 msgr2=0x7f8cf40b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:29.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.403+0000 7f8cf8d1a700 1 -- 172.21.15.81:0/2194085724 shutdown_connections 2024-05-08T23:32:29.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.403+0000 7f8cf8d1a700 1 -- 172.21.15.81:0/2194085724 wait complete. 2024-05-08T23:32:29.419 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T23:32:29.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.539+0000 7ff0d7fff700 1 Processor -- start 2024-05-08T23:32:29.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.539+0000 7ff0d7fff700 1 -- start start 2024-05-08T23:32:29.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d7fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff0d815d300 0x7ff0d80d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:29.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0d80d02b0 0x7ff0d80d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:29.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d7fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff0d80d1a30 0x7ff0d80cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:29.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d7fff700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d80d0710 con 0x7ff0d815d300 2024-05-08T23:32:29.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d7fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d80b3030 con 0x7ff0d80d1a30 2024-05-08T23:32:29.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d7fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d80d1210 con 0x7ff0d80d02b0 2024-05-08T23:32:29.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d6ffd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff0d815d300 0x7ff0d80d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:29.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d6ffd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff0d815d300 0x7ff0d80d0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39770/0 (socket says 172.21.15.81:39770) 2024-05-08T23:32:29.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d6ffd700 1 -- 172.21.15.81:0/2124216046 learned_addr learned my addr 172.21.15.81:0/2124216046 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:29.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d77fe700 1 --2- 172.21.15.81:0/2124216046 >> v2:172.21.15.117:3300/0 conn(0x7ff0d80d1a30 0x7ff0d80cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:29.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d67fc700 1 --2- 172.21.15.81:0/2124216046 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0d80d02b0 0x7ff0d80d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:29.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d6ffd700 1 -- 172.21.15.81:0/2124216046 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0d80d02b0 msgr2=0x7ff0d80d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:29.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d6ffd700 1 --2- 172.21.15.81:0/2124216046 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0d80d02b0 0x7ff0d80d14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:29.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d6ffd700 1 -- 172.21.15.81:0/2124216046 >> v2:172.21.15.117:3300/0 conn(0x7ff0d80d1a30 msgr2=0x7ff0d80cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:29.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d6ffd700 1 --2- 172.21.15.81:0/2124216046 >> v2:172.21.15.117:3300/0 conn(0x7ff0d80d1a30 0x7ff0d80cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:29.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d6ffd700 1 -- 172.21.15.81:0/2124216046 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d80b29e0 con 0x7ff0d815d300 2024-05-08T23:32:29.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d6ffd700 1 --2- 172.21.15.81:0/2124216046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0d815d300 0x7ff0d80d0c30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009910 tx=0x7ff0cc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4f460ccbc45a67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:29.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d5ffb700 1 -- 172.21.15.81:0/2124216046 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc011830 con 0x7ff0d815d300 2024-05-08T23:32:29.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d5ffb700 1 -- 172.21.15.81:0/2124216046 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0cc011990 con 0x7ff0d815d300 2024-05-08T23:32:29.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d5ffb700 1 -- 172.21.15.81:0/2124216046 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc011d40 con 0x7ff0d815d300 2024-05-08T23:32:29.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d7fff700 1 -- 172.21.15.81:0/2124216046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0d815d300 msgr2=0x7ff0d80d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:29.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d7fff700 1 --2- 172.21.15.81:0/2124216046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0d815d300 0x7ff0d80d0c30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009910 tx=0x7ff0cc009c20 comp rx=0 tx=0).stop 2024-05-08T23:32:29.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d7fff700 1 -- 172.21.15.81:0/2124216046 shutdown_connections 2024-05-08T23:32:29.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d7fff700 1 --2- 172.21.15.81:0/2124216046 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0d80d02b0 0x7ff0d80d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:29.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d7fff700 1 --2- 172.21.15.81:0/2124216046 >> v2:172.21.15.117:3300/0 conn(0x7ff0d80d1a30 0x7ff0d80cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:29.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d7fff700 1 --2- 172.21.15.81:0/2124216046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0d815d300 0x7ff0d80d0c30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:29.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.543+0000 7ff0d7fff700 1 -- 172.21.15.81:0/2124216046 >> 172.21.15.81:0/2124216046 conn(0x7ff0d809c3b0 msgr2=0x7ff0d80d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:29.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.547+0000 7ff0d7fff700 1 -- 172.21.15.81:0/2124216046 shutdown_connections 2024-05-08T23:32:29.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.547+0000 7ff0d7fff700 1 -- 172.21.15.81:0/2124216046 wait complete. 2024-05-08T23:32:29.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.547+0000 7ff0d7fff700 1 Processor -- start 2024-05-08T23:32:29.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.547+0000 7ff0d7fff700 1 -- start start 2024-05-08T23:32:29.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0d80d02b0 0x7ff0d80c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:29.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0d80d1a30 0x7ff0d8152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:29.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0d815d300 0x7ff0d81f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:29.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d7fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d80c8380 con 0x7ff0d80d02b0 2024-05-08T23:32:29.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d7fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d80c87a0 con 0x7ff0d815d300 2024-05-08T23:32:29.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d7fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d80c7700 con 0x7ff0d80d1a30 2024-05-08T23:32:29.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0d80d02b0 0x7ff0d80c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:29.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0d80d02b0 0x7ff0d80c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39784/0 (socket says 172.21.15.81:39784) 2024-05-08T23:32:29.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d6ffd700 1 -- 172.21.15.81:0/726877862 learned_addr learned my addr 172.21.15.81:0/726877862 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:29.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d67fc700 1 --2- 172.21.15.81:0/726877862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0d80d1a30 0x7ff0d8152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:29.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d77fe700 1 --2- 172.21.15.81:0/726877862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0d815d300 0x7ff0d81f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:29.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d6ffd700 1 -- 172.21.15.81:0/726877862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0d80d1a30 msgr2=0x7ff0d8152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:29.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d6ffd700 1 --2- 172.21.15.81:0/726877862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0d80d1a30 0x7ff0d8152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:29.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d6ffd700 1 -- 172.21.15.81:0/726877862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0d815d300 msgr2=0x7ff0d81f5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:29.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d6ffd700 1 --2- 172.21.15.81:0/726877862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0d815d300 0x7ff0d81f5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:29.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d6ffd700 1 -- 172.21.15.81:0/726877862 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc009570 con 0x7ff0d80d02b0 2024-05-08T23:32:29.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d6ffd700 1 --2- 172.21.15.81:0/726877862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0d80d02b0 0x7ff0d80c9550 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc000c00 tx=0x7ff0cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:29.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0bffff700 1 -- 172.21.15.81:0/726877862 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc004510 con 0x7ff0d80d02b0 2024-05-08T23:32:29.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0bffff700 1 -- 172.21.15.81:0/726877862 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0cc01e070 con 0x7ff0d80d02b0 2024-05-08T23:32:29.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0bffff700 1 -- 172.21.15.81:0/726877862 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc003900 con 0x7ff0d80d02b0 2024-05-08T23:32:29.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d7fff700 1 -- 172.21.15.81:0/726877862 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d80c79e0 con 0x7ff0d80d02b0 2024-05-08T23:32:29.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.551+0000 7ff0d7fff700 1 -- 172.21.15.81:0/726877862 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d80c7fa0 con 0x7ff0d80d02b0 2024-05-08T23:32:29.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.555+0000 7ff0d7fff700 1 -- 172.21.15.81:0/726877862 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d80c85f0 con 0x7ff0d80d02b0 2024-05-08T23:32:29.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.555+0000 7ff0bffff700 1 -- 172.21.15.81:0/726877862 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff0cc003a60 con 0x7ff0d80d02b0 2024-05-08T23:32:29.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.555+0000 7ff0bffff700 1 --2- 172.21.15.81:0/726877862 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff0c4066730 0x7ff0c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:29.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.555+0000 7ff0bffff700 1 -- 172.21.15.81:0/726877862 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6190+0+0 (secure 0 0 0) 0x7ff0cc085190 con 0x7ff0d80d02b0 2024-05-08T23:32:29.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.555+0000 7ff0d67fc700 1 --2- 172.21.15.81:0/726877862 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff0c4066730 0x7ff0c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:29.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.555+0000 7ff0d67fc700 1 --2- 172.21.15.81:0/726877862 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff0c4066730 0x7ff0c4068bf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80720c0 tx=0x7ff0c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:29.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.563+0000 7ff0bffff700 1 -- 172.21.15.81:0/726877862 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff0cc052470 con 0x7ff0d80d02b0 2024-05-08T23:32:29.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:29.731+0000 7ff0d7fff700 1 -- 172.21.15.81:0/726877862 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0d80684b0 con 0x7ff0d80d02b0 2024-05-08T23:32:30.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.403+0000 7ff0bffff700 1 -- 172.21.15.81:0/726877862 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v41) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff0cc055a90 con 0x7ff0d80d02b0 2024-05-08T23:32:30.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.411+0000 7ff0d7fff700 1 -- 172.21.15.81:0/726877862 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff0c4066730 msgr2=0x7ff0c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.411+0000 7ff0d7fff700 1 --2- 172.21.15.81:0/726877862 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff0c4066730 0x7ff0c4068bf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80720c0 tx=0x7ff0c8008040 comp rx=0 tx=0).stop 2024-05-08T23:32:30.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.411+0000 7ff0d7fff700 1 -- 172.21.15.81:0/726877862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0d80d02b0 msgr2=0x7ff0d80c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.411+0000 7ff0d7fff700 1 --2- 172.21.15.81:0/726877862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0d80d02b0 0x7ff0d80c9550 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc000c00 tx=0x7ff0cc004770 comp rx=0 tx=0).stop 2024-05-08T23:32:30.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.411+0000 7ff0d7fff700 1 -- 172.21.15.81:0/726877862 shutdown_connections 2024-05-08T23:32:30.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.411+0000 7ff0d7fff700 1 --2- 172.21.15.81:0/726877862 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff0c4066730 0x7ff0c4068bf0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.411+0000 7ff0d7fff700 1 --2- 172.21.15.81:0/726877862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0d815d300 0x7ff0d81f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.411+0000 7ff0d7fff700 1 --2- 172.21.15.81:0/726877862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0d80d1a30 0x7ff0d8152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.411+0000 7ff0d7fff700 1 --2- 172.21.15.81:0/726877862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0d80d02b0 0x7ff0d80c9550 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.411+0000 7ff0d7fff700 1 -- 172.21.15.81:0/726877862 >> 172.21.15.81:0/726877862 conn(0x7ff0d809c3b0 msgr2=0x7ff0d80cf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:30.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.411+0000 7ff0d7fff700 1 -- 172.21.15.81:0/726877862 shutdown_connections 2024-05-08T23:32:30.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.411+0000 7ff0d7fff700 1 -- 172.21.15.81:0/726877862 wait complete. 2024-05-08T23:32:30.416 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' removed 2024-05-08T23:32:30.428 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T23:32:30.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.535+0000 7f6210936700 1 Processor -- start 2024-05-08T23:32:30.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.535+0000 7f6210936700 1 -- start start 2024-05-08T23:32:30.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6210936700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f620c159540 0x7f620c0cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6210936700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f620c0b26e0 0x7f620c0b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6210936700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620c0cbaa0 con 0x7f620c0ce390 2024-05-08T23:32:30.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6210936700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620c0721a0 con 0x7f620c0b26e0 2024-05-08T23:32:30.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6210936700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620c071ec0 con 0x7f620c159540 2024-05-08T23:32:30.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f620b59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f620c0b26e0 0x7f620c0b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f620a59c700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f620c0ce390 0x7f620c0b1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55432/0 (socket says 172.21.15.81:55432) 2024-05-08T23:32:30.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f620a59c700 1 -- 172.21.15.81:0/1887498246 learned_addr learned my addr 172.21.15.81:0/1887498246 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:30.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f620ad9d700 1 --2- 172.21.15.81:0/1887498246 >> v2:172.21.15.117:3300/0 conn(0x7f620c159540 0x7f620c0cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f620b59e700 1 -- 172.21.15.81:0/1887498246 >> v2:172.21.15.117:3300/0 conn(0x7f620c159540 msgr2=0x7f620c0cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:30.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f620b59e700 1 --2- 172.21.15.81:0/1887498246 >> v2:172.21.15.117:3300/0 conn(0x7f620c159540 0x7f620c0cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6209d9b700 1 -- 172.21.15.81:0/1887498246 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936531689 0 0) 0x7f620c0cbaa0 con 0x7f620c0ce390 2024-05-08T23:32:30.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f620b59e700 1 -- 172.21.15.81:0/1887498246 >> v1:172.21.15.81:6789/0 conn(0x7f620c0ce390 legacy=0x7f620c0b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f620b59e700 1 -- 172.21.15.81:0/1887498246 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c0cd090 con 0x7f620c0b26e0 2024-05-08T23:32:30.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f620ad9d700 1 --2- 172.21.15.81:0/1887498246 >> v2:172.21.15.117:3300/0 conn(0x7f620c159540 0x7f620c0cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:30.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f620b59e700 1 --2- 172.21.15.81:0/1887498246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f620c0b26e0 0x7f620c0b0c60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f620c06e330 tx=0x7f620400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba300b48e7ac6a46 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:30.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6209d9b700 1 -- 172.21.15.81:0/1887498246 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620400f070 con 0x7f620c0b26e0 2024-05-08T23:32:30.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6209d9b700 1 -- 172.21.15.81:0/1887498246 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620400a2e0 con 0x7f620c0b26e0 2024-05-08T23:32:30.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6209d9b700 1 -- 172.21.15.81:0/1887498246 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620400cc00 con 0x7f620c0b26e0 2024-05-08T23:32:30.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6210936700 1 -- 172.21.15.81:0/1887498246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f620c0b26e0 msgr2=0x7f620c0b0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6210936700 1 --2- 172.21.15.81:0/1887498246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f620c0b26e0 0x7f620c0b0c60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f620c06e330 tx=0x7f620400c330 comp rx=0 tx=0).stop 2024-05-08T23:32:30.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6210936700 1 -- 172.21.15.81:0/1887498246 shutdown_connections 2024-05-08T23:32:30.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6210936700 1 --2- 172.21.15.81:0/1887498246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f620c0b26e0 0x7f620c0b0c60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6210936700 1 --2- 172.21.15.81:0/1887498246 >> v2:172.21.15.117:3300/0 conn(0x7f620c159540 0x7f620c0cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6210936700 1 -- 172.21.15.81:0/1887498246 >> 172.21.15.81:0/1887498246 conn(0x7f620c06d650 msgr2=0x7f620c06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:30.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6210936700 1 -- 172.21.15.81:0/1887498246 shutdown_connections 2024-05-08T23:32:30.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.539+0000 7f6210936700 1 -- 172.21.15.81:0/1887498246 wait complete. 2024-05-08T23:32:30.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.543+0000 7f6210936700 1 Processor -- start 2024-05-08T23:32:30.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.543+0000 7f6210936700 1 -- start start 2024-05-08T23:32:30.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.543+0000 7f6210936700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c0b26e0 0x7f620c13f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.543+0000 7f6210936700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f620c0ce390 0x7f620c1e3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f6210936700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f620c159540 0x7f620c1e4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f6210936700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c0aff30 con 0x7f620c0ce390 2024-05-08T23:32:30.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f6210936700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c14cc30 con 0x7f620c0b26e0 2024-05-08T23:32:30.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f6210936700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620c0af320 con 0x7f620c159540 2024-05-08T23:32:30.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f620a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f620c0ce390 0x7f620c1e3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f620ad9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c0b26e0 0x7f620c13f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f620a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f620c0ce390 0x7f620c1e3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39790/0 (socket says 172.21.15.81:39790) 2024-05-08T23:32:30.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f620a59c700 1 -- 172.21.15.81:0/4294390800 learned_addr learned my addr 172.21.15.81:0/4294390800 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:30.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f620b59e700 1 --2- 172.21.15.81:0/4294390800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f620c159540 0x7f620c1e4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f620a59c700 1 -- 172.21.15.81:0/4294390800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f620c159540 msgr2=0x7f620c1e4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f620a59c700 1 --2- 172.21.15.81:0/4294390800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f620c159540 0x7f620c1e4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f620a59c700 1 -- 172.21.15.81:0/4294390800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c0b26e0 msgr2=0x7f620c13f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f620a59c700 1 --2- 172.21.15.81:0/4294390800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c0b26e0 0x7f620c13f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f620a59c700 1 -- 172.21.15.81:0/4294390800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620400ab80 con 0x7f620c0ce390 2024-05-08T23:32:30.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f620b59e700 1 --2- 172.21.15.81:0/4294390800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f620c159540 0x7f620c1e4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:30.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f620ad9d700 1 --2- 172.21.15.81:0/4294390800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c0b26e0 0x7f620c13f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:30.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f620a59c700 1 --2- 172.21.15.81:0/4294390800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f620c0ce390 0x7f620c1e3c20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00c580 tx=0x7f61fc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:30.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f61f37fe700 1 -- 172.21.15.81:0/4294390800 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc004ca0 con 0x7f620c0ce390 2024-05-08T23:32:30.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f61f37fe700 1 -- 172.21.15.81:0/4294390800 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61fc004e20 con 0x7f620c0ce390 2024-05-08T23:32:30.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f6210936700 1 -- 172.21.15.81:0/4294390800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620c0af600 con 0x7f620c0ce390 2024-05-08T23:32:30.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f61f37fe700 1 -- 172.21.15.81:0/4294390800 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc007310 con 0x7f620c0ce390 2024-05-08T23:32:30.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f6210936700 1 -- 172.21.15.81:0/4294390800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620c0afbc0 con 0x7f620c0ce390 2024-05-08T23:32:30.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.547+0000 7f6210936700 1 -- 172.21.15.81:0/4294390800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620c0684b0 con 0x7f620c0ce390 2024-05-08T23:32:30.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.551+0000 7f61f37fe700 1 -- 172.21.15.81:0/4294390800 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f61fc007470 con 0x7f620c0ce390 2024-05-08T23:32:30.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.551+0000 7f61f37fe700 1 --2- 172.21.15.81:0/4294390800 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f61f8066730 0x7f61f8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.551+0000 7f61f37fe700 1 -- 172.21.15.81:0/4294390800 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5752+0+0 (secure 0 0 0) 0x7f61fc083db0 con 0x7f620c0ce390 2024-05-08T23:32:30.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.551+0000 7f620ad9d700 1 --2- 172.21.15.81:0/4294390800 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f61f8066730 0x7f61f8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.551+0000 7f620ad9d700 1 --2- 172.21.15.81:0/4294390800 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f61f8066730 0x7f61f8068bf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f620000af30 tx=0x7f6200008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:30.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.559+0000 7f61f37fe700 1 -- 172.21.15.81:0/4294390800 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f61fc0510f0 con 0x7f620c0ce390 2024-05-08T23:32:30.739 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:30.735+0000 7f6210936700 1 -- 172.21.15.81:0/4294390800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f620c0b2f00 con 0x7f620c0ce390 2024-05-08T23:32:31.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.411+0000 7f61f37fe700 1 -- 172.21.15.81:0/4294390800 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v42) v1 ==== 169+0+0 (secure 0 0 0) 0x7f61fc007720 con 0x7f620c0ce390 2024-05-08T23:32:31.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.415+0000 7f6210936700 1 -- 172.21.15.81:0/4294390800 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f61f8066730 msgr2=0x7f61f8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.415+0000 7f6210936700 1 --2- 172.21.15.81:0/4294390800 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f61f8066730 0x7f61f8068bf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f620000af30 tx=0x7f6200008040 comp rx=0 tx=0).stop 2024-05-08T23:32:31.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.415+0000 7f6210936700 1 -- 172.21.15.81:0/4294390800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f620c0ce390 msgr2=0x7f620c1e3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.415+0000 7f6210936700 1 --2- 172.21.15.81:0/4294390800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f620c0ce390 0x7f620c1e3c20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00c580 tx=0x7f61fc00c890 comp rx=0 tx=0).stop 2024-05-08T23:32:31.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.415+0000 7f6210936700 1 -- 172.21.15.81:0/4294390800 shutdown_connections 2024-05-08T23:32:31.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.415+0000 7f6210936700 1 --2- 172.21.15.81:0/4294390800 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f61f8066730 0x7f61f8068bf0 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f620000af30 tx=0x7f6200008040 comp rx=0 tx=0).stop 2024-05-08T23:32:31.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.415+0000 7f6210936700 1 --2- 172.21.15.81:0/4294390800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f620c159540 0x7f620c1e4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.415+0000 7f6210936700 1 --2- 172.21.15.81:0/4294390800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f620c0ce390 0x7f620c1e3c20 secure :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00c580 tx=0x7f61fc00c890 comp rx=0 tx=0).stop 2024-05-08T23:32:31.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.415+0000 7f6210936700 1 --2- 172.21.15.81:0/4294390800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c0b26e0 0x7f620c13f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.415+0000 7f6210936700 1 -- 172.21.15.81:0/4294390800 >> 172.21.15.81:0/4294390800 conn(0x7f620c06d650 msgr2=0x7f620c0700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:31.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.415+0000 7f6210936700 1 -- 172.21.15.81:0/4294390800 shutdown_connections 2024-05-08T23:32:31.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.415+0000 7f6210936700 1 -- 172.21.15.81:0/4294390800 wait complete. 2024-05-08T23:32:31.419 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' removed 2024-05-08T23:32:31.431 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2024-05-08T23:32:31.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.551+0000 7fd842d1e700 1 Processor -- start 2024-05-08T23:32:31.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.551+0000 7fd842d1e700 1 -- start start 2024-05-08T23:32:31.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd842d1e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd83c15d300 0x7fd83c0d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd842d1e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd83c0d1c00 0x7fd83c0b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd842d1e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83c0b3030 con 0x7fd83c0d02b0 2024-05-08T23:32:31.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd842d1e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83c0d1210 con 0x7fd83c15d300 2024-05-08T23:32:31.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd842d1e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0b29e0 con 0x7fd83c0d1c00 2024-05-08T23:32:31.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd84151b700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd83c0d02b0 0x7fd83c0d14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55444/0 (socket says 172.21.15.81:55444) 2024-05-08T23:32:31.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd84151b700 1 -- 172.21.15.81:0/528868149 learned_addr learned my addr 172.21.15.81:0/528868149 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:31.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd841d1c700 1 --2- 172.21.15.81:0/528868149 >> v2:172.21.15.81:3300/0 conn(0x7fd83c15d300 0x7fd83c0d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd84251d700 1 --2- 172.21.15.81:0/528868149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd83c0d1c00 0x7fd83c0b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd840d1a700 1 -- 172.21.15.81:0/528868149 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3886359454 0 0) 0x7fd83c0b3030 con 0x7fd83c0d02b0 2024-05-08T23:32:31.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd840d1a700 1 -- 172.21.15.81:0/528868149 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8280035d0 con 0x7fd83c0d02b0 2024-05-08T23:32:31.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd841d1c700 1 -- 172.21.15.81:0/528868149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd83c0d1c00 msgr2=0x7fd83c0b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:31.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd841d1c700 1 --2- 172.21.15.81:0/528868149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd83c0d1c00 0x7fd83c0b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd841d1c700 1 -- 172.21.15.81:0/528868149 >> v1:172.21.15.81:6789/0 conn(0x7fd83c0d02b0 legacy=0x7fd83c0d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd841d1c700 1 -- 172.21.15.81:0/528868149 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c0b2cc0 con 0x7fd83c15d300 2024-05-08T23:32:31.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd841d1c700 1 --2- 172.21.15.81:0/528868149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd83c15d300 0x7fd83c0d0c30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd830009850 tx=0x7fd830009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62268ca799200bd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:31.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd840d1a700 1 -- 172.21.15.81:0/528868149 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830011830 con 0x7fd83c15d300 2024-05-08T23:32:31.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd840d1a700 1 -- 172.21.15.81:0/528868149 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd830011990 con 0x7fd83c15d300 2024-05-08T23:32:31.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd840d1a700 1 -- 172.21.15.81:0/528868149 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830011d00 con 0x7fd83c15d300 2024-05-08T23:32:31.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd842d1e700 1 -- 172.21.15.81:0/528868149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd83c15d300 msgr2=0x7fd83c0d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd842d1e700 1 --2- 172.21.15.81:0/528868149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd83c15d300 0x7fd83c0d0c30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd830009850 tx=0x7fd830009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:31.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd842d1e700 1 -- 172.21.15.81:0/528868149 shutdown_connections 2024-05-08T23:32:31.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd842d1e700 1 --2- 172.21.15.81:0/528868149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd83c0d1c00 0x7fd83c0b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd842d1e700 1 --2- 172.21.15.81:0/528868149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd83c15d300 0x7fd83c0d0c30 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.555+0000 7fd842d1e700 1 -- 172.21.15.81:0/528868149 >> 172.21.15.81:0/528868149 conn(0x7fd83c09c3b0 msgr2=0x7fd83c0d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:31.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.559+0000 7fd842d1e700 1 -- 172.21.15.81:0/528868149 shutdown_connections 2024-05-08T23:32:31.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.559+0000 7fd842d1e700 1 -- 172.21.15.81:0/528868149 wait complete. 2024-05-08T23:32:31.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.559+0000 7fd842d1e700 1 Processor -- start 2024-05-08T23:32:31.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.559+0000 7fd842d1e700 1 -- start start 2024-05-08T23:32:31.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd842d1e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd83c0d02b0 0x7fd83c1f0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd842d1e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd83c0d1c00 0x7fd83c160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd842d1e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd83c15d300 0x7fd83c14c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd842d1e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c1f0650 con 0x7fd83c0d02b0 2024-05-08T23:32:31.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd842d1e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0c9790 con 0x7fd83c0d1c00 2024-05-08T23:32:31.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd842d1e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0c8ef0 con 0x7fd83c15d300 2024-05-08T23:32:31.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd841d1c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd83c0d02b0 0x7fd83c1f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd841d1c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd83c0d02b0 0x7fd83c1f0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39810/0 (socket says 172.21.15.81:39810) 2024-05-08T23:32:31.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd841d1c700 1 -- 172.21.15.81:0/790769017 learned_addr learned my addr 172.21.15.81:0/790769017 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:31.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd84151b700 1 --2- 172.21.15.81:0/790769017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd83c0d1c00 0x7fd83c160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd841d1c700 1 -- 172.21.15.81:0/790769017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd83c15d300 msgr2=0x7fd83c14c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd841d1c700 1 --2- 172.21.15.81:0/790769017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd83c15d300 0x7fd83c14c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd841d1c700 1 -- 172.21.15.81:0/790769017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd83c0d1c00 msgr2=0x7fd83c160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd841d1c700 1 --2- 172.21.15.81:0/790769017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd83c0d1c00 0x7fd83c160220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd841d1c700 1 -- 172.21.15.81:0/790769017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8300094b0 con 0x7fd83c0d02b0 2024-05-08T23:32:31.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.563+0000 7fd841d1c700 1 --2- 172.21.15.81:0/790769017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd83c0d02b0 0x7fd83c1f0110 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd83c06ae80 tx=0x7fd830004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:31.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.571+0000 7fd82e7fc700 1 -- 172.21.15.81:0/790769017 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830004590 con 0x7fd83c0d02b0 2024-05-08T23:32:31.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.571+0000 7fd82e7fc700 1 -- 172.21.15.81:0/790769017 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83001e070 con 0x7fd83c0d02b0 2024-05-08T23:32:31.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.571+0000 7fd842d1e700 1 -- 172.21.15.81:0/790769017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83c0c91d0 con 0x7fd83c0d02b0 2024-05-08T23:32:31.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.571+0000 7fd82e7fc700 1 -- 172.21.15.81:0/790769017 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830003900 con 0x7fd83c0d02b0 2024-05-08T23:32:31.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.571+0000 7fd842d1e700 1 -- 172.21.15.81:0/790769017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83c0b5d70 con 0x7fd83c0d02b0 2024-05-08T23:32:31.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.571+0000 7fd81ffff700 1 -- 172.21.15.81:0/790769017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83c0684b0 con 0x7fd83c0d02b0 2024-05-08T23:32:31.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.579+0000 7fd82e7fc700 1 -- 172.21.15.81:0/790769017 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd830003a60 con 0x7fd83c0d02b0 2024-05-08T23:32:31.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.579+0000 7fd82e7fc700 1 --2- 172.21.15.81:0/790769017 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd820066830 0x7fd820068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.579+0000 7fd82e7fc700 1 -- 172.21.15.81:0/790769017 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5338+0+0 (secure 0 0 0) 0x7fd830084d80 con 0x7fd83c0d02b0 2024-05-08T23:32:31.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.579+0000 7fd82e7fc700 1 -- 172.21.15.81:0/790769017 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd830089370 con 0x7fd83c0d02b0 2024-05-08T23:32:31.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.579+0000 7fd84151b700 1 --2- 172.21.15.81:0/790769017 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd820066830 0x7fd820068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.583+0000 7fd84151b700 1 --2- 172.21.15.81:0/790769017 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd820066830 0x7fd820068cf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd838007fd0 tx=0x7fd838009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:31.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:31.743+0000 7fd81ffff700 1 -- 172.21.15.81:0/790769017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd83c0d33a0 con 0x7fd83c0d02b0 2024-05-08T23:32:32.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.411+0000 7fd82e7fc700 1 -- 172.21.15.81:0/790769017 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v43) v1 ==== 160+0+0 (secure 0 0 0) 0x7fd8300150d0 con 0x7fd83c0d02b0 2024-05-08T23:32:32.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.415+0000 7fd81ffff700 1 -- 172.21.15.81:0/790769017 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd820066830 msgr2=0x7fd820068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.419+0000 7fd81ffff700 1 --2- 172.21.15.81:0/790769017 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd820066830 0x7fd820068cf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd838007fd0 tx=0x7fd838009040 comp rx=0 tx=0).stop 2024-05-08T23:32:32.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.419+0000 7fd81ffff700 1 -- 172.21.15.81:0/790769017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd83c0d02b0 msgr2=0x7fd83c1f0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.419+0000 7fd81ffff700 1 --2- 172.21.15.81:0/790769017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd83c0d02b0 0x7fd83c1f0110 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd83c06ae80 tx=0x7fd830004770 comp rx=0 tx=0).stop 2024-05-08T23:32:32.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.419+0000 7fd81ffff700 1 -- 172.21.15.81:0/790769017 shutdown_connections 2024-05-08T23:32:32.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.419+0000 7fd81ffff700 1 --2- 172.21.15.81:0/790769017 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd820066830 0x7fd820068cf0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.419+0000 7fd81ffff700 1 --2- 172.21.15.81:0/790769017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd83c15d300 0x7fd83c14c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.419+0000 7fd81ffff700 1 --2- 172.21.15.81:0/790769017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd83c0d1c00 0x7fd83c160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.419+0000 7fd81ffff700 1 --2- 172.21.15.81:0/790769017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd83c0d02b0 0x7fd83c1f0110 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.419+0000 7fd81ffff700 1 -- 172.21.15.81:0/790769017 >> 172.21.15.81:0/790769017 conn(0x7fd83c09c3b0 msgr2=0x7fd83c06c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:32.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.419+0000 7fd81ffff700 1 -- 172.21.15.81:0/790769017 shutdown_connections 2024-05-08T23:32:32.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.419+0000 7fd81ffff700 1 -- 172.21.15.81:0/790769017 wait complete. 2024-05-08T23:32:32.422 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'data_new' removed 2024-05-08T23:32:32.436 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T23:32:32.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16ffff700 1 Processor -- start 2024-05-08T23:32:32.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16ffff700 1 -- start start 2024-05-08T23:32:32.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1700b3de0 0x7ff17006a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16ffff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff170165950 con 0x7ff170161610 2024-05-08T23:32:32.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16ffff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff170165c30 con 0x7ff17006acc0 2024-05-08T23:32:32.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1701619f0 con 0x7ff1700b3de0 2024-05-08T23:32:32.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16effd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff170161610 0x7ff170165090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55452/0 (socket says 172.21.15.81:55452) 2024-05-08T23:32:32.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16effd700 1 -- 172.21.15.81:0/3524227772 learned_addr learned my addr 172.21.15.81:0/3524227772 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:32.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16e7fc700 1 --2- 172.21.15.81:0/3524227772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1700b3de0 0x7ff17006a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16dffb700 1 -- 172.21.15.81:0/3524227772 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409932974 0 0) 0x7ff170165950 con 0x7ff170161610 2024-05-08T23:32:32.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16dffb700 1 -- 172.21.15.81:0/3524227772 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff15c0035d0 con 0x7ff170161610 2024-05-08T23:32:32.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16dffb700 1 -- 172.21.15.81:0/3524227772 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840057667 0 0) 0x7ff170165c30 con 0x7ff17006acc0 2024-05-08T23:32:32.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16dffb700 1 -- 172.21.15.81:0/3524227772 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff170165950 con 0x7ff17006acc0 2024-05-08T23:32:32.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16e7fc700 1 -- 172.21.15.81:0/3524227772 >> v1:172.21.15.117:6789/0 conn(0x7ff17006acc0 legacy=0x7ff1700ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16e7fc700 1 -- 172.21.15.81:0/3524227772 >> v1:172.21.15.81:6789/0 conn(0x7ff170161610 legacy=0x7ff170165090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16e7fc700 1 -- 172.21.15.81:0/3524227772 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff170165f10 con 0x7ff1700b3de0 2024-05-08T23:32:32.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.555+0000 7ff16e7fc700 1 --2- 172.21.15.81:0/3524227772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1700b3de0 0x7ff17006a780 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff160009dc0 tx=0x7ff16000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0167fcd69eae24f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:32.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16dffb700 1 -- 172.21.15.81:0/3524227772 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16000e070 con 0x7ff1700b3de0 2024-05-08T23:32:32.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16dffb700 1 -- 172.21.15.81:0/3524227772 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16000bb20 con 0x7ff1700b3de0 2024-05-08T23:32:32.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16dffb700 1 -- 172.21.15.81:0/3524227772 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff160012430 con 0x7ff1700b3de0 2024-05-08T23:32:32.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16ffff700 1 -- 172.21.15.81:0/3524227772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1700b3de0 msgr2=0x7ff17006a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16ffff700 1 --2- 172.21.15.81:0/3524227772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1700b3de0 0x7ff17006a780 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff160009dc0 tx=0x7ff16000b330 comp rx=0 tx=0).stop 2024-05-08T23:32:32.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16ffff700 1 -- 172.21.15.81:0/3524227772 shutdown_connections 2024-05-08T23:32:32.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16ffff700 1 --2- 172.21.15.81:0/3524227772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1700b3de0 0x7ff17006a780 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16ffff700 1 -- 172.21.15.81:0/3524227772 >> 172.21.15.81:0/3524227772 conn(0x7ff1700b0e00 msgr2=0x7ff1700b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:32.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16ffff700 1 -- 172.21.15.81:0/3524227772 shutdown_connections 2024-05-08T23:32:32.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16ffff700 1 -- 172.21.15.81:0/3524227772 wait complete. 2024-05-08T23:32:32.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16ffff700 1 Processor -- start 2024-05-08T23:32:32.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16ffff700 1 -- start start 2024-05-08T23:32:32.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff17006acc0 0x7ff1701f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1700b3de0 0x7ff1700c3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff170161610 0x7ff17014caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16ffff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17006b170 con 0x7ff170161610 2024-05-08T23:32:32.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16ffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff170162110 con 0x7ff17006acc0 2024-05-08T23:32:32.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17009af60 con 0x7ff1700b3de0 2024-05-08T23:32:32.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff170161610 0x7ff17014caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff17006acc0 0x7ff1701f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16e7fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1700b3de0 0x7ff1700c3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff17006acc0 0x7ff1701f7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:57632/0 (socket says 172.21.15.81:57632) 2024-05-08T23:32:32.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.559+0000 7ff16effd700 1 -- 172.21.15.81:0/179272438 learned_addr learned my addr 172.21.15.81:0/179272438 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:32.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.563+0000 7ff16effd700 1 -- 172.21.15.81:0/179272438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1700b3de0 msgr2=0x7ff1700c3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.563+0000 7ff16effd700 1 --2- 172.21.15.81:0/179272438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1700b3de0 0x7ff1700c3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.563+0000 7ff16effd700 1 -- 172.21.15.81:0/179272438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff170161610 msgr2=0x7ff17014caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.563+0000 7ff16effd700 1 --2- 172.21.15.81:0/179272438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff170161610 0x7ff17014caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.563+0000 7ff16effd700 1 -- 172.21.15.81:0/179272438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff160009a70 con 0x7ff17006acc0 2024-05-08T23:32:32.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.563+0000 7ff16f7fe700 1 --2- 172.21.15.81:0/179272438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff170161610 0x7ff17014caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:32.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.563+0000 7ff16effd700 1 --2- 172.21.15.81:0/179272438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff17006acc0 0x7ff1701f7b60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c4d0 tx=0x7ff15800c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:32.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.563+0000 7ff167fff700 1 -- 172.21.15.81:0/179272438 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158005d90 con 0x7ff17006acc0 2024-05-08T23:32:32.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.563+0000 7ff167fff700 1 -- 172.21.15.81:0/179272438 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1580070d0 con 0x7ff17006acc0 2024-05-08T23:32:32.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.563+0000 7ff167fff700 1 -- 172.21.15.81:0/179272438 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158007480 con 0x7ff17006acc0 2024-05-08T23:32:32.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.563+0000 7ff16ffff700 1 -- 172.21.15.81:0/179272438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1700c4200 con 0x7ff17006acc0 2024-05-08T23:32:32.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.563+0000 7ff16ffff700 1 -- 172.21.15.81:0/179272438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff170161b70 con 0x7ff17006acc0 2024-05-08T23:32:32.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.563+0000 7ff16ffff700 1 -- 172.21.15.81:0/179272438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1700677c0 con 0x7ff17006acc0 2024-05-08T23:32:32.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.563+0000 7ff167fff700 1 -- 172.21.15.81:0/179272438 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff1580075e0 con 0x7ff17006acc0 2024-05-08T23:32:32.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.563+0000 7ff167fff700 1 --2- 172.21.15.81:0/179272438 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff150066730 0x7ff150068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.567+0000 7ff167fff700 1 -- 172.21.15.81:0/179272438 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff158083320 con 0x7ff17006acc0 2024-05-08T23:32:32.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.567+0000 7ff16e7fc700 1 --2- 172.21.15.81:0/179272438 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff150066730 0x7ff150068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.567+0000 7ff16e7fc700 1 --2- 172.21.15.81:0/179272438 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff150066730 0x7ff150068bf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff160014040 tx=0x7ff16001e040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:32.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.575+0000 7ff167fff700 1 -- 172.21.15.81:0/179272438 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff158050900 con 0x7ff17006acc0 2024-05-08T23:32:32.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.751+0000 7ff16ffff700 1 -- 172.21.15.81:0/179272438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff17006a980 con 0x7ff17006acc0 2024-05-08T23:32:32.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.763+0000 7ff167fff700 1 -- 172.21.15.81:0/179272438 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff158053f20 con 0x7ff17006acc0 2024-05-08T23:32:32.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.763+0000 7ff16ffff700 1 -- 172.21.15.81:0/179272438 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff150066730 msgr2=0x7ff150068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.763+0000 7ff16ffff700 1 --2- 172.21.15.81:0/179272438 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff150066730 0x7ff150068bf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff160014040 tx=0x7ff16001e040 comp rx=0 tx=0).stop 2024-05-08T23:32:32.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.763+0000 7ff16ffff700 1 -- 172.21.15.81:0/179272438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff17006acc0 msgr2=0x7ff1701f7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.763+0000 7ff16ffff700 1 --2- 172.21.15.81:0/179272438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff17006acc0 0x7ff1701f7b60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c4d0 tx=0x7ff15800c7e0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.763+0000 7ff16ffff700 1 -- 172.21.15.81:0/179272438 shutdown_connections 2024-05-08T23:32:32.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.763+0000 7ff16ffff700 1 --2- 172.21.15.81:0/179272438 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff150066730 0x7ff150068bf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.763+0000 7ff16ffff700 1 --2- 172.21.15.81:0/179272438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff170161610 0x7ff17014caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.763+0000 7ff16ffff700 1 --2- 172.21.15.81:0/179272438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1700b3de0 0x7ff1700c3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.763+0000 7ff16ffff700 1 --2- 172.21.15.81:0/179272438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff17006acc0 0x7ff1701f7b60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.763+0000 7ff16ffff700 1 -- 172.21.15.81:0/179272438 >> 172.21.15.81:0/179272438 conn(0x7ff1700b0e00 msgr2=0x7ff17006dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:32.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.763+0000 7ff16ffff700 1 -- 172.21.15.81:0/179272438 shutdown_connections 2024-05-08T23:32:32.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.763+0000 7ff16ffff700 1 -- 172.21.15.81:0/179272438 wait complete. 2024-05-08T23:32:32.780 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2024-05-08T23:32:32.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43b5c9d700 1 Processor -- start 2024-05-08T23:32:32.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43b5c9d700 1 -- start start 2024-05-08T23:32:32.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43b5c9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b006ba80 0x7f43b00ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43b5c9d700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b00ae1f0 con 0x7f43b015d190 2024-05-08T23:32:32.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43b5c9d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b00ad5e0 con 0x7f43b00b2610 2024-05-08T23:32:32.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43b5c9d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00ad8c0 con 0x7f43b006ba80 2024-05-08T23:32:32.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43b4c9b700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f43b015d190 0x7f43b006b370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55460/0 (socket says 172.21.15.81:55460) 2024-05-08T23:32:32.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43b4c9b700 1 -- 172.21.15.81:0/9719718 learned_addr learned my addr 172.21.15.81:0/9719718 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:32.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43affff700 1 --2- 172.21.15.81:0/9719718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b006ba80 0x7f43b00ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3994905243 0 0) 0x7f43b00ae1f0 con 0x7f43b015d190 2024-05-08T23:32:32.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f439c0035d0 con 0x7f43b015d190 2024-05-08T23:32:32.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595866449 0 0) 0x7f43b00ad5e0 con 0x7f43b00b2610 2024-05-08T23:32:32.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43b00ae1f0 con 0x7f43b00b2610 2024-05-08T23:32:32.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4285459148 0 0) 0x7f439c0035d0 con 0x7f43b015d190 2024-05-08T23:32:32.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b006ba80 msgr2=0x7f43b00ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:32.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43af7fe700 1 --2- 172.21.15.81:0/9719718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b006ba80 0x7f43b00ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 >> v1:172.21.15.117:6789/0 conn(0x7f43b00b2610 legacy=0x7f43b006c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b00adba0 con 0x7f43b015d190 2024-05-08T23:32:32.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.895+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f43a80019f0 con 0x7f43b015d190 2024-05-08T23:32:32.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 >> v1:172.21.15.81:6789/0 conn(0x7f43b015d190 legacy=0x7f43b006b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43af7fe700 1 --2- 172.21.15.81:0/9719718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f439c003be0 0x7f439c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43af7fe700 1 --2- 172.21.15.81:0/9719718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f439c004870 0x7f439c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43af7fe700 1 --2- 172.21.15.81:0/9719718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b006ba80 0x7f439c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00ad8c0 con 0x7f439c003be0 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c00bae0 con 0x7f439c004870 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f439c00bca0 con 0x7f43b006ba80 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b4c9b700 1 --2- 172.21.15.81:0/9719718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f439c003be0 0x7f439c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43affff700 1 --2- 172.21.15.81:0/9719718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f439c004870 0x7f439c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b549c700 1 --2- 172.21.15.81:0/9719718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b006ba80 0x7f439c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b4c9b700 1 -- 172.21.15.81:0/9719718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b006ba80 msgr2=0x7f439c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b4c9b700 1 --2- 172.21.15.81:0/9719718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b006ba80 0x7f439c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b4c9b700 1 -- 172.21.15.81:0/9719718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f439c004870 msgr2=0x7f439c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b4c9b700 1 --2- 172.21.15.81:0/9719718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f439c004870 0x7f439c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b4c9b700 1 -- 172.21.15.81:0/9719718 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c0035d0 con 0x7f439c003be0 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43affff700 1 --2- 172.21.15.81:0/9719718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f439c004870 0x7f439c004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b549c700 1 --2- 172.21.15.81:0/9719718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b006ba80 0x7f439c005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b4c9b700 1 --2- 172.21.15.81:0/9719718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f439c003be0 0x7f439c003fe0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f43a800a7a0 tx=0x7f439c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c426a7a07da987 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a80072d0 con 0x7f439c003be0 2024-05-08T23:32:32.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a8007430 con 0x7f439c003be0 2024-05-08T23:32:32.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43af7fe700 1 -- 172.21.15.81:0/9719718 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a80077e0 con 0x7f439c003be0 2024-05-08T23:32:32.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b5c9d700 1 -- 172.21.15.81:0/9719718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f439c003be0 msgr2=0x7f439c003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b5c9d700 1 --2- 172.21.15.81:0/9719718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f439c003be0 0x7f439c003fe0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f43a800a7a0 tx=0x7f439c0058a0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b549c700 1 -- 172.21.15.81:0/9719718 reap_dead start 2024-05-08T23:32:32.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b5c9d700 1 -- 172.21.15.81:0/9719718 shutdown_connections 2024-05-08T23:32:32.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b5c9d700 1 -- 172.21.15.81:0/9719718 >> 172.21.15.81:0/9719718 conn(0x7f43b0070960 msgr2=0x7f43b0070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:32.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b5c9d700 1 -- 172.21.15.81:0/9719718 shutdown_connections 2024-05-08T23:32:32.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b5c9d700 1 -- 172.21.15.81:0/9719718 wait complete. 2024-05-08T23:32:32.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b5c9d700 1 Processor -- start 2024-05-08T23:32:32.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.899+0000 7f43b5c9d700 1 -- start start 2024-05-08T23:32:32.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b5c9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f439c003be0 0x7f43b01407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b5c9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43b0140fa0 0x7f43b015bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b5c9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43b0141420 0x7f43b015c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b5c9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b0140d20 con 0x7f43b0141420 2024-05-08T23:32:32.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b5c9d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b015b960 con 0x7f43b0140fa0 2024-05-08T23:32:32.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b5c9d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b015b0a0 con 0x7f439c003be0 2024-05-08T23:32:32.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b549c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43b0141420 0x7f43b015c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b549c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43b0141420 0x7f43b015c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39838/0 (socket says 172.21.15.81:39838) 2024-05-08T23:32:32.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b549c700 1 -- 172.21.15.81:0/2399604524 learned_addr learned my addr 172.21.15.81:0/2399604524 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:32.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43affff700 1 --2- 172.21.15.81:0/2399604524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43b0140fa0 0x7f43b015bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b4c9b700 1 --2- 172.21.15.81:0/2399604524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f439c003be0 0x7f43b01407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b549c700 1 -- 172.21.15.81:0/2399604524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f439c003be0 msgr2=0x7f43b01407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b549c700 1 --2- 172.21.15.81:0/2399604524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f439c003be0 0x7f43b01407e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b549c700 1 -- 172.21.15.81:0/2399604524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43b0140fa0 msgr2=0x7f43b015bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b549c700 1 --2- 172.21.15.81:0/2399604524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43b0140fa0 0x7f43b015bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b549c700 1 -- 172.21.15.81:0/2399604524 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b0159aa0 con 0x7f43b0141420 2024-05-08T23:32:32.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b549c700 1 --2- 172.21.15.81:0/2399604524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43b0141420 0x7f43b015c330 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f43b0070560 tx=0x7f43a000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:32.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43ad7fa700 1 -- 172.21.15.81:0/2399604524 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a000b070 con 0x7f43b0141420 2024-05-08T23:32:32.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43ad7fa700 1 -- 172.21.15.81:0/2399604524 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a00052f0 con 0x7f43b0141420 2024-05-08T23:32:32.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b5c9d700 1 -- 172.21.15.81:0/2399604524 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b015b310 con 0x7f43b0141420 2024-05-08T23:32:32.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43ad7fa700 1 -- 172.21.15.81:0/2399604524 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0006880 con 0x7f43b0141420 2024-05-08T23:32:32.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b5c9d700 1 -- 172.21.15.81:0/2399604524 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b0143eb0 con 0x7f43b0141420 2024-05-08T23:32:32.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.903+0000 7f43b5c9d700 1 -- 172.21.15.81:0/2399604524 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b00672a0 con 0x7f43b0141420 2024-05-08T23:32:32.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.907+0000 7f43ad7fa700 1 -- 172.21.15.81:0/2399604524 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f43a00069e0 con 0x7f43b0141420 2024-05-08T23:32:32.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.907+0000 7f43ad7fa700 1 --2- 172.21.15.81:0/2399604524 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4390066730 0x7f4390068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.907+0000 7f43ad7fa700 1 -- 172.21.15.81:0/2399604524 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f43a0082f90 con 0x7f43b0141420 2024-05-08T23:32:32.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.907+0000 7f43b4c9b700 1 --2- 172.21.15.81:0/2399604524 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4390066730 0x7f4390068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.907+0000 7f43b4c9b700 1 --2- 172.21.15.81:0/2399604524 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4390066730 0x7f4390068bf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f43a8016040 tx=0x7f43a8009070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:32.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:32.915+0000 7f43ad7fa700 1 -- 172.21.15.81:0/2399604524 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f43a0050600 con 0x7f43b0141420 2024-05-08T23:32:33.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.147+0000 7f43b5c9d700 1 -- 172.21.15.81:0/2399604524 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f43b0071590 con 0x7f43b0141420 2024-05-08T23:32:33.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.427+0000 7f43ad7fa700 1 -- 172.21.15.81:0/2399604524 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v122) v1 ==== 147+0+0 (secure 0 0 0) 0x7f43a00180d0 con 0x7f43b0141420 2024-05-08T23:32:33.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.435+0000 7f43b5c9d700 1 -- 172.21.15.81:0/2399604524 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4390066730 msgr2=0x7f4390068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.435+0000 7f43b5c9d700 1 --2- 172.21.15.81:0/2399604524 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4390066730 0x7f4390068bf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f43a8016040 tx=0x7f43a8009070 comp rx=0 tx=0).stop 2024-05-08T23:32:33.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.435+0000 7f43b5c9d700 1 -- 172.21.15.81:0/2399604524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43b0141420 msgr2=0x7f43b015c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.435+0000 7f43b5c9d700 1 --2- 172.21.15.81:0/2399604524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43b0141420 0x7f43b015c330 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f43b0070560 tx=0x7f43a000d710 comp rx=0 tx=0).stop 2024-05-08T23:32:33.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.435+0000 7f43b5c9d700 1 -- 172.21.15.81:0/2399604524 shutdown_connections 2024-05-08T23:32:33.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.435+0000 7f43b5c9d700 1 --2- 172.21.15.81:0/2399604524 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4390066730 0x7f4390068bf0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.435+0000 7f43b5c9d700 1 --2- 172.21.15.81:0/2399604524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43b0141420 0x7f43b015c330 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.435+0000 7f43b5c9d700 1 --2- 172.21.15.81:0/2399604524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43b0140fa0 0x7f43b015bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.435+0000 7f43b5c9d700 1 --2- 172.21.15.81:0/2399604524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f439c003be0 0x7f43b01407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.435+0000 7f43b5c9d700 1 -- 172.21.15.81:0/2399604524 >> 172.21.15.81:0/2399604524 conn(0x7f43b0070960 msgr2=0x7f43b00b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:33.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.435+0000 7f43b5c9d700 1 -- 172.21.15.81:0/2399604524 shutdown_connections 2024-05-08T23:32:33.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.435+0000 7f43b5c9d700 1 -- 172.21.15.81:0/2399604524 wait complete. 2024-05-08T23:32:33.453 INFO:tasks.cephfs_test_runner:That a client who does not have the capability for setting ... ok 2024-05-08T23:32:33.453 INFO:tasks.cephfs_test_runner:Starting test: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-05-08T23:32:33.454 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:33.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e7cd9700 1 Processor -- start 2024-05-08T23:32:33.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e7cd9700 1 -- start start 2024-05-08T23:32:33.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e7cd9700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb3e0160a30 0x7fb3e00d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e7cd9700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e00b66d0 con 0x7fb3e00d3c50 2024-05-08T23:32:33.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e7cd9700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e00d11c0 con 0x7fb3e00d1e90 2024-05-08T23:32:33.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e7cd9700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e00d14a0 con 0x7fb3e0160a30 2024-05-08T23:32:33.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e5a75700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb3e0160a30 0x7fb3e00d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e5274700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb3e00d3c50 0x7fb3e00d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55468/0 (socket says 172.21.15.81:55468) 2024-05-08T23:32:33.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e5274700 1 -- 172.21.15.81:0/1288385554 learned_addr learned my addr 172.21.15.81:0/1288385554 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:33.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e5a75700 1 -- 172.21.15.81:0/1288385554 >> v1:172.21.15.117:6789/0 conn(0x7fb3e00d1e90 legacy=0x7fb3e00b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e5a75700 1 -- 172.21.15.81:0/1288385554 >> v1:172.21.15.81:6789/0 conn(0x7fb3e00d3c50 legacy=0x7fb3e00d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e5a75700 1 -- 172.21.15.81:0/1288385554 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e00b6080 con 0x7fb3e0160a30 2024-05-08T23:32:33.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e5a75700 1 --2- 172.21.15.81:0/1288385554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3e0160a30 0x7fb3e00d0480 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009d90 tx=0x7fb3d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2303ddad998e83de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:33.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e4a73700 1 -- 172.21.15.81:0/1288385554 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d400e070 con 0x7fb3e0160a30 2024-05-08T23:32:33.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e4a73700 1 -- 172.21.15.81:0/1288385554 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d400bbd0 con 0x7fb3e0160a30 2024-05-08T23:32:33.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e4a73700 1 -- 172.21.15.81:0/1288385554 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4012430 con 0x7fb3e0160a30 2024-05-08T23:32:33.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e7cd9700 1 -- 172.21.15.81:0/1288385554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3e0160a30 msgr2=0x7fb3e00d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e7cd9700 1 --2- 172.21.15.81:0/1288385554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3e0160a30 0x7fb3e00d0480 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009d90 tx=0x7fb3d400b330 comp rx=0 tx=0).stop 2024-05-08T23:32:33.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e7cd9700 1 -- 172.21.15.81:0/1288385554 shutdown_connections 2024-05-08T23:32:33.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e7cd9700 1 --2- 172.21.15.81:0/1288385554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3e0160a30 0x7fb3e00d0480 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e7cd9700 1 -- 172.21.15.81:0/1288385554 >> 172.21.15.81:0/1288385554 conn(0x7fb3e009c500 msgr2=0x7fb3e00d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:33.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e7cd9700 1 -- 172.21.15.81:0/1288385554 shutdown_connections 2024-05-08T23:32:33.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.571+0000 7fb3e7cd9700 1 -- 172.21.15.81:0/1288385554 wait complete. 2024-05-08T23:32:33.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e7cd9700 1 Processor -- start 2024-05-08T23:32:33.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e7cd9700 1 -- start start 2024-05-08T23:32:33.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e7cd9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3e00d1e90 0x7fb3e014c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e7cd9700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3e00d3c50 0x7fb3e0159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e7cd9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3e0160a30 0x7fb3e00ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e7cd9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00cc9b0 con 0x7fb3e0160a30 2024-05-08T23:32:33.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e7cd9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e015c330 con 0x7fb3e00d1e90 2024-05-08T23:32:33.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e7cd9700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00cbd30 con 0x7fb3e00d3c50 2024-05-08T23:32:33.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e6276700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3e0160a30 0x7fb3e00ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e6276700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3e0160a30 0x7fb3e00ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39864/0 (socket says 172.21.15.81:39864) 2024-05-08T23:32:33.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e5a75700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3e00d1e90 0x7fb3e014c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e6276700 1 -- 172.21.15.81:0/660386768 learned_addr learned my addr 172.21.15.81:0/660386768 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:33.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e5274700 1 --2- 172.21.15.81:0/660386768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3e00d3c50 0x7fb3e0159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e6276700 1 -- 172.21.15.81:0/660386768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3e00d3c50 msgr2=0x7fb3e0159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e6276700 1 --2- 172.21.15.81:0/660386768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3e00d3c50 0x7fb3e0159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e6276700 1 -- 172.21.15.81:0/660386768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3e00d1e90 msgr2=0x7fb3e014c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e6276700 1 --2- 172.21.15.81:0/660386768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3e00d1e90 0x7fb3e014c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e6276700 1 -- 172.21.15.81:0/660386768 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d4009a70 con 0x7fb3e0160a30 2024-05-08T23:32:33.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e6276700 1 --2- 172.21.15.81:0/660386768 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3e0160a30 0x7fb3e00ccdd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800c480 tx=0x7fb3d800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:33.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3de7fc700 1 -- 172.21.15.81:0/660386768 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d80052d0 con 0x7fb3e0160a30 2024-05-08T23:32:33.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e7cd9700 1 -- 172.21.15.81:0/660386768 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e00cc010 con 0x7fb3e0160a30 2024-05-08T23:32:33.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3e7cd9700 1 -- 172.21.15.81:0/660386768 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e00cc5d0 con 0x7fb3e0160a30 2024-05-08T23:32:33.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3de7fc700 1 -- 172.21.15.81:0/660386768 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d8005430 con 0x7fb3e0160a30 2024-05-08T23:32:33.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.575+0000 7fb3de7fc700 1 -- 172.21.15.81:0/660386768 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d80055a0 con 0x7fb3e0160a30 2024-05-08T23:32:33.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.579+0000 7fb3e7cd9700 1 -- 172.21.15.81:0/660386768 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e00ccaf0 con 0x7fb3e0160a30 2024-05-08T23:32:33.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.579+0000 7fb3de7fc700 1 -- 172.21.15.81:0/660386768 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb3d80038a0 con 0x7fb3e0160a30 2024-05-08T23:32:33.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.579+0000 7fb3de7fc700 1 --2- 172.21.15.81:0/660386768 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb3c4066730 0x7fb3c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.579+0000 7fb3de7fc700 1 -- 172.21.15.81:0/660386768 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb3d8083130 con 0x7fb3e0160a30 2024-05-08T23:32:33.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.579+0000 7fb3e5a75700 1 --2- 172.21.15.81:0/660386768 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb3c4066730 0x7fb3c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.579+0000 7fb3e5a75700 1 --2- 172.21.15.81:0/660386768 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb3c4066730 0x7fb3c4068bf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4014040 tx=0x7fb3d4011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:33.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.587+0000 7fb3de7fc700 1 -- 172.21.15.81:0/660386768 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb3e00ccaf0 con 0x7fb3e0160a30 2024-05-08T23:32:33.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.759+0000 7fb3e7cd9700 1 -- 172.21.15.81:0/660386768 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3e00ccaf0 con 0x7fb3e0160a30 2024-05-08T23:32:33.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.763+0000 7fb3de7fc700 1 -- 172.21.15.81:0/660386768 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+12520 (secure 0 0 0) 0x7fb3e00ccaf0 con 0x7fb3e0160a30 2024-05-08T23:32:33.766 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:33.766 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":43,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:32:32.407335+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6835/3091184157":"2024-05-09T23:32:28.379033+0000","172.21.15.117:6832/3091184157":"2024-05-09T23:32:28.379033+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:33.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.767+0000 7fb3e7cd9700 1 -- 172.21.15.81:0/660386768 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb3c4066730 msgr2=0x7fb3c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.767+0000 7fb3e7cd9700 1 --2- 172.21.15.81:0/660386768 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb3c4066730 0x7fb3c4068bf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4014040 tx=0x7fb3d4011000 comp rx=0 tx=0).stop 2024-05-08T23:32:33.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.767+0000 7fb3e7cd9700 1 -- 172.21.15.81:0/660386768 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3e0160a30 msgr2=0x7fb3e00ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.767+0000 7fb3e7cd9700 1 --2- 172.21.15.81:0/660386768 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3e0160a30 0x7fb3e00ccdd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800c480 tx=0x7fb3d800c790 comp rx=0 tx=0).stop 2024-05-08T23:32:33.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.767+0000 7fb3e7cd9700 1 -- 172.21.15.81:0/660386768 shutdown_connections 2024-05-08T23:32:33.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.767+0000 7fb3e7cd9700 1 --2- 172.21.15.81:0/660386768 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb3c4066730 0x7fb3c4068bf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.767+0000 7fb3e7cd9700 1 --2- 172.21.15.81:0/660386768 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb3e0160a30 0x7fb3e00ccdd0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.767+0000 7fb3e7cd9700 1 --2- 172.21.15.81:0/660386768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3e00d3c50 0x7fb3e0159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.767+0000 7fb3e7cd9700 1 --2- 172.21.15.81:0/660386768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3e00d1e90 0x7fb3e014c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.767+0000 7fb3e7cd9700 1 -- 172.21.15.81:0/660386768 >> 172.21.15.81:0/660386768 conn(0x7fb3e009c500 msgr2=0x7fb3e015e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:33.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.767+0000 7fb3e7cd9700 1 -- 172.21.15.81:0/660386768 shutdown_connections 2024-05-08T23:32:33.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.767+0000 7fb3e7cd9700 1 -- 172.21.15.81:0/660386768 wait complete. 2024-05-08T23:32:33.781 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:32:33.782 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:33.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.899+0000 7fb1c6b31700 1 Processor -- start 2024-05-08T23:32:33.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.899+0000 7fb1c6b31700 1 -- start start 2024-05-08T23:32:33.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.899+0000 7fb1c6b31700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb1c0160920 0x7fb1c00ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.899+0000 7fb1c6b31700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb1c00d0050 0x7fb1c00b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.899+0000 7fb1c6b31700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c00d3d40 con 0x7fb1c00d1da0 2024-05-08T23:32:33.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.899+0000 7fb1c6b31700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c00d0d00 con 0x7fb1c00d0050 2024-05-08T23:32:33.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.899+0000 7fb1c6b31700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c00d0fe0 con 0x7fb1c0160920 2024-05-08T23:32:33.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.899+0000 7fb1c6330700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb1c00d0050 0x7fb1c00b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.899+0000 7fb1c6330700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb1c00d0050 0x7fb1c00b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39876/0 (socket says 172.21.15.81:39876) 2024-05-08T23:32:33.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.899+0000 7fb1c6330700 1 -- 172.21.15.81:0/2250848347 learned_addr learned my addr 172.21.15.81:0/2250848347 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:33.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.899+0000 7fb1c5b2f700 1 --2- 172.21.15.81:0/2250848347 >> v2:172.21.15.117:3300/0 conn(0x7fb1c0160920 0x7fb1c00ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c6330700 1 -- 172.21.15.81:0/2250848347 >> v2:172.21.15.117:3300/0 conn(0x7fb1c0160920 msgr2=0x7fb1c00ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:33.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c6330700 1 --2- 172.21.15.81:0/2250848347 >> v2:172.21.15.117:3300/0 conn(0x7fb1c0160920 0x7fb1c00ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c6330700 1 -- 172.21.15.81:0/2250848347 >> v1:172.21.15.117:6789/0 conn(0x7fb1c00d1da0 legacy=0x7fb1c00cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:33.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c6330700 1 -- 172.21.15.81:0/2250848347 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c00d12c0 con 0x7fb1c00d0050 2024-05-08T23:32:33.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c5b2f700 1 --2- 172.21.15.81:0/2250848347 >> v2:172.21.15.117:3300/0 conn(0x7fb1c0160920 0x7fb1c00ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:33.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c6330700 1 --2- 172.21.15.81:0/2250848347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1c00d0050 0x7fb1c00b5e50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb1c008c490 tx=0x7fb1b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53a4968d66d0c80 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:33.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c4b2d700 1 -- 172.21.15.81:0/2250848347 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b800e070 con 0x7fb1c00d0050 2024-05-08T23:32:33.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c4b2d700 1 -- 172.21.15.81:0/2250848347 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b8003a40 con 0x7fb1c00d0050 2024-05-08T23:32:33.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c4b2d700 1 -- 172.21.15.81:0/2250848347 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8003df0 con 0x7fb1c00d0050 2024-05-08T23:32:33.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c6b31700 1 -- 172.21.15.81:0/2250848347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1c00d0050 msgr2=0x7fb1c00b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c6b31700 1 --2- 172.21.15.81:0/2250848347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1c00d0050 0x7fb1c00b5e50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb1c008c490 tx=0x7fb1b800b330 comp rx=0 tx=0).stop 2024-05-08T23:32:33.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c6b31700 1 -- 172.21.15.81:0/2250848347 shutdown_connections 2024-05-08T23:32:33.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c6b31700 1 --2- 172.21.15.81:0/2250848347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1c00d0050 0x7fb1c00b5e50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c6b31700 1 --2- 172.21.15.81:0/2250848347 >> v2:172.21.15.117:3300/0 conn(0x7fb1c0160920 0x7fb1c00ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c6b31700 1 -- 172.21.15.81:0/2250848347 >> 172.21.15.81:0/2250848347 conn(0x7fb1c009c340 msgr2=0x7fb1c00d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:33.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c6b31700 1 -- 172.21.15.81:0/2250848347 shutdown_connections 2024-05-08T23:32:33.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c6b31700 1 -- 172.21.15.81:0/2250848347 wait complete. 2024-05-08T23:32:33.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c6b31700 1 Processor -- start 2024-05-08T23:32:33.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.903+0000 7fb1c6b31700 1 -- start start 2024-05-08T23:32:33.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c6b31700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1c00d0050 0x7fb1c015d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c6b31700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1c00d1da0 0x7fb1c014e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c6b31700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c0160920 0x7fb1c014b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c6b31700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c014f230 con 0x7fb1c00d0050 2024-05-08T23:32:33.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c6b31700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c014c6c0 con 0x7fb1c0160920 2024-05-08T23:32:33.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c6b31700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c014ec70 con 0x7fb1c00d1da0 2024-05-08T23:32:33.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c5b2f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1c00d0050 0x7fb1c015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c5b2f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1c00d0050 0x7fb1c015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39886/0 (socket says 172.21.15.81:39886) 2024-05-08T23:32:33.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c5b2f700 1 -- 172.21.15.81:0/4109126251 learned_addr learned my addr 172.21.15.81:0/4109126251 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:33.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c6330700 1 --2- 172.21.15.81:0/4109126251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c0160920 0x7fb1c014b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c532e700 1 --2- 172.21.15.81:0/4109126251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1c00d1da0 0x7fb1c014e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c5b2f700 1 -- 172.21.15.81:0/4109126251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1c00d1da0 msgr2=0x7fb1c014e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c5b2f700 1 --2- 172.21.15.81:0/4109126251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1c00d1da0 0x7fb1c014e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c5b2f700 1 -- 172.21.15.81:0/4109126251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c0160920 msgr2=0x7fb1c014b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c5b2f700 1 --2- 172.21.15.81:0/4109126251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c0160920 0x7fb1c014b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c5b2f700 1 -- 172.21.15.81:0/4109126251 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b8009b30 con 0x7fb1c00d0050 2024-05-08T23:32:33.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c5b2f700 1 --2- 172.21.15.81:0/4109126251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1c00d0050 0x7fb1c015d120 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400c7f0 tx=0x7fb1b400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:33.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1b27fc700 1 -- 172.21.15.81:0/4109126251 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4011800 con 0x7fb1c00d0050 2024-05-08T23:32:33.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1b27fc700 1 -- 172.21.15.81:0/4109126251 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b4011960 con 0x7fb1c00d0050 2024-05-08T23:32:33.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c6b31700 1 -- 172.21.15.81:0/4109126251 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c0160e80 con 0x7fb1c00d0050 2024-05-08T23:32:33.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1b27fc700 1 -- 172.21.15.81:0/4109126251 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4011cf0 con 0x7fb1c00d0050 2024-05-08T23:32:33.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.907+0000 7fb1c6b31700 1 -- 172.21.15.81:0/4109126251 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c014c350 con 0x7fb1c00d0050 2024-05-08T23:32:33.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.911+0000 7fb1c6b31700 1 -- 172.21.15.81:0/4109126251 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c014c1b0 con 0x7fb1c00d0050 2024-05-08T23:32:33.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.911+0000 7fb1b27fc700 1 -- 172.21.15.81:0/4109126251 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb1b4021350 con 0x7fb1c00d0050 2024-05-08T23:32:33.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.911+0000 7fb1b27fc700 1 --2- 172.21.15.81:0/4109126251 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb1a4066730 0x7fb1a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.911+0000 7fb1b27fc700 1 -- 172.21.15.81:0/4109126251 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb1b4083c90 con 0x7fb1c00d0050 2024-05-08T23:32:33.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.911+0000 7fb1c532e700 1 --2- 172.21.15.81:0/4109126251 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb1a4066730 0x7fb1a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.911+0000 7fb1c532e700 1 --2- 172.21.15.81:0/4109126251 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb1a4066730 0x7fb1a4068bf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb1c015bd10 tx=0x7fb1bc0090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:33.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:33.923+0000 7fb1b27fc700 1 -- 172.21.15.81:0/4109126251 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb1c014c1b0 con 0x7fb1c00d0050 2024-05-08T23:32:34.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.091+0000 7fb1c6b31700 1 -- 172.21.15.81:0/4109126251 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1c014c1b0 con 0x7fb1c00d0050 2024-05-08T23:32:34.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.095+0000 7fb1b27fc700 1 -- 172.21.15.81:0/4109126251 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1c014c1b0 con 0x7fb1c00d0050 2024-05-08T23:32:34.095 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:32:34.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.095+0000 7fb1c6b31700 1 -- 172.21.15.81:0/4109126251 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb1a4066730 msgr2=0x7fb1a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.095+0000 7fb1c6b31700 1 --2- 172.21.15.81:0/4109126251 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb1a4066730 0x7fb1a4068bf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb1c015bd10 tx=0x7fb1bc0090e0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.095+0000 7fb1c6b31700 1 -- 172.21.15.81:0/4109126251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1c00d0050 msgr2=0x7fb1c015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.095+0000 7fb1c6b31700 1 --2- 172.21.15.81:0/4109126251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1c00d0050 0x7fb1c015d120 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400c7f0 tx=0x7fb1b400cb00 comp rx=0 tx=0).stop 2024-05-08T23:32:34.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.095+0000 7fb1c6b31700 1 -- 172.21.15.81:0/4109126251 shutdown_connections 2024-05-08T23:32:34.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.095+0000 7fb1c6b31700 1 --2- 172.21.15.81:0/4109126251 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb1a4066730 0x7fb1a4068bf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.095+0000 7fb1c6b31700 1 --2- 172.21.15.81:0/4109126251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c0160920 0x7fb1c014b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.095+0000 7fb1c6b31700 1 --2- 172.21.15.81:0/4109126251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1c00d1da0 0x7fb1c014e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.095+0000 7fb1c6b31700 1 --2- 172.21.15.81:0/4109126251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1c00d0050 0x7fb1c015d120 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.095+0000 7fb1c6b31700 1 -- 172.21.15.81:0/4109126251 >> 172.21.15.81:0/4109126251 conn(0x7fb1c009c340 msgr2=0x7fb1c00b7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:34.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.095+0000 7fb1c6b31700 1 -- 172.21.15.81:0/4109126251 shutdown_connections 2024-05-08T23:32:34.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.095+0000 7fb1c6b31700 1 -- 172.21.15.81:0/4109126251 wait complete. 2024-05-08T23:32:34.110 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2024-05-08T23:32:34.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.215+0000 7fcdd0866700 1 Processor -- start 2024-05-08T23:32:34.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.215+0000 7fcdd0866700 1 -- start start 2024-05-08T23:32:34.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdd0866700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdcc0ce990 0x7fcdcc15f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdd0866700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdcc15eda0 con 0x7fcdcc15f770 2024-05-08T23:32:34.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdd0866700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdcc15ea70 con 0x7fcdcc15afa0 2024-05-08T23:32:34.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdd0866700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc15ba90 con 0x7fcdcc0ce990 2024-05-08T23:32:34.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdcbfff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcdcc15f770 0x7fcdcc0b4270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55480/0 (socket says 172.21.15.81:55480) 2024-05-08T23:32:34.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdcbfff700 1 -- 172.21.15.81:0/2569074587 learned_addr learned my addr 172.21.15.81:0/2569074587 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:34.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdcaffd700 1 --2- 172.21.15.81:0/2569074587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdcc0ce990 0x7fcdcc15f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdca7fc700 1 -- 172.21.15.81:0/2569074587 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773869013 0 0) 0x7fcdcc15eda0 con 0x7fcdcc15f770 2024-05-08T23:32:34.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdca7fc700 1 -- 172.21.15.81:0/2569074587 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdb40035d0 con 0x7fcdcc15f770 2024-05-08T23:32:34.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdca7fc700 1 -- 172.21.15.81:0/2569074587 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586699 0 0) 0x7fcdcc15ea70 con 0x7fcdcc15afa0 2024-05-08T23:32:34.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdca7fc700 1 -- 172.21.15.81:0/2569074587 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdcc15eda0 con 0x7fcdcc15afa0 2024-05-08T23:32:34.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdcaffd700 1 -- 172.21.15.81:0/2569074587 >> v1:172.21.15.117:6789/0 conn(0x7fcdcc15afa0 legacy=0x7fcdcc15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdcaffd700 1 -- 172.21.15.81:0/2569074587 >> v1:172.21.15.81:6789/0 conn(0x7fcdcc15f770 legacy=0x7fcdcc0b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdcaffd700 1 -- 172.21.15.81:0/2569074587 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc15bd40 con 0x7fcdcc0ce990 2024-05-08T23:32:34.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdcaffd700 1 --2- 172.21.15.81:0/2569074587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdcc0ce990 0x7fcdcc15f230 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009880 tx=0x7fcdc0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d866a4fd141c9698 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:34.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdca7fc700 1 -- 172.21.15.81:0/2569074587 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0011830 con 0x7fcdcc0ce990 2024-05-08T23:32:34.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdca7fc700 1 -- 172.21.15.81:0/2569074587 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc0011990 con 0x7fcdcc0ce990 2024-05-08T23:32:34.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdca7fc700 1 -- 172.21.15.81:0/2569074587 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0011d20 con 0x7fcdcc0ce990 2024-05-08T23:32:34.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdd0866700 1 -- 172.21.15.81:0/2569074587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdcc0ce990 msgr2=0x7fcdcc15f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdd0866700 1 --2- 172.21.15.81:0/2569074587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdcc0ce990 0x7fcdcc15f230 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009880 tx=0x7fcdc0009b90 comp rx=0 tx=0).stop 2024-05-08T23:32:34.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdd0866700 1 -- 172.21.15.81:0/2569074587 shutdown_connections 2024-05-08T23:32:34.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdd0866700 1 --2- 172.21.15.81:0/2569074587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdcc0ce990 0x7fcdcc15f230 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdd0866700 1 -- 172.21.15.81:0/2569074587 >> 172.21.15.81:0/2569074587 conn(0x7fcdcc06c330 msgr2=0x7fcdcc06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:34.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdd0866700 1 -- 172.21.15.81:0/2569074587 shutdown_connections 2024-05-08T23:32:34.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.219+0000 7fcdd0866700 1 -- 172.21.15.81:0/2569074587 wait complete. 2024-05-08T23:32:34.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdd0866700 1 Processor -- start 2024-05-08T23:32:34.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdd0866700 1 -- start start 2024-05-08T23:32:34.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdd0866700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdcc0ce990 0x7fcdcc0ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdd0866700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdcc15afa0 0x7fcdcc1f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdd0866700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdcc15f770 0x7fcdcc159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdd0866700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc158e90 con 0x7fcdcc0ce990 2024-05-08T23:32:34.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdd0866700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0b3110 con 0x7fcdcc15afa0 2024-05-08T23:32:34.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdd0866700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0ade00 con 0x7fcdcc15f770 2024-05-08T23:32:34.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdcb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdcc0ce990 0x7fcdcc0ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdcb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdcc0ce990 0x7fcdcc0ad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39896/0 (socket says 172.21.15.81:39896) 2024-05-08T23:32:34.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdcb7fe700 1 -- 172.21.15.81:0/1115778805 learned_addr learned my addr 172.21.15.81:0/1115778805 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:34.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdcaffd700 1 --2- 172.21.15.81:0/1115778805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdcc15afa0 0x7fcdcc1f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdcbfff700 1 --2- 172.21.15.81:0/1115778805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdcc15f770 0x7fcdcc159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdcb7fe700 1 -- 172.21.15.81:0/1115778805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdcc15f770 msgr2=0x7fcdcc159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdcb7fe700 1 --2- 172.21.15.81:0/1115778805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdcc15f770 0x7fcdcc159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdcb7fe700 1 -- 172.21.15.81:0/1115778805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdcc15afa0 msgr2=0x7fcdcc1f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdcb7fe700 1 --2- 172.21.15.81:0/1115778805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdcc15afa0 0x7fcdcc1f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdcb7fe700 1 -- 172.21.15.81:0/1115778805 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc00094b0 con 0x7fcdcc0ce990 2024-05-08T23:32:34.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdcbfff700 1 --2- 172.21.15.81:0/1115778805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdcc15f770 0x7fcdcc159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:34.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.223+0000 7fcdcb7fe700 1 --2- 172.21.15.81:0/1115778805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdcc0ce990 0x7fcdcc0ad8c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4001f00 tx=0x7fcdc400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:34.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.227+0000 7fcdb3fff700 1 -- 172.21.15.81:0/1115778805 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4003c20 con 0x7fcdcc0ce990 2024-05-08T23:32:34.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.227+0000 7fcdb3fff700 1 -- 172.21.15.81:0/1115778805 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc4003d80 con 0x7fcdcc0ce990 2024-05-08T23:32:34.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.227+0000 7fcdd0866700 1 -- 172.21.15.81:0/1115778805 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdcc0b33b0 con 0x7fcdcc0ce990 2024-05-08T23:32:34.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.227+0000 7fcdb3fff700 1 -- 172.21.15.81:0/1115778805 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc400ac60 con 0x7fcdcc0ce990 2024-05-08T23:32:34.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.227+0000 7fcdd0866700 1 -- 172.21.15.81:0/1115778805 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdcc0b4010 con 0x7fcdcc0ce990 2024-05-08T23:32:34.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.227+0000 7fcdd0866700 1 -- 172.21.15.81:0/1115778805 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdcc0684b0 con 0x7fcdcc0ce990 2024-05-08T23:32:34.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.227+0000 7fcdb3fff700 1 -- 172.21.15.81:0/1115778805 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcdc40050d0 con 0x7fcdcc0ce990 2024-05-08T23:32:34.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.227+0000 7fcdb3fff700 1 --2- 172.21.15.81:0/1115778805 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcdb8066730 0x7fcdb8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.227+0000 7fcdb3fff700 1 -- 172.21.15.81:0/1115778805 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcdc40840a0 con 0x7fcdcc0ce990 2024-05-08T23:32:34.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.227+0000 7fcdcaffd700 1 --2- 172.21.15.81:0/1115778805 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcdb8066730 0x7fcdb8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.231+0000 7fcdcaffd700 1 --2- 172.21.15.81:0/1115778805 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcdb8066730 0x7fcdb8068bf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00090b0 tx=0x7fcdc001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:34.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.235+0000 7fcdb3fff700 1 -- 172.21.15.81:0/1115778805 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcdc4051690 con 0x7fcdcc0ce990 2024-05-08T23:32:34.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:34.467+0000 7fcdd0866700 1 -- 172.21.15.81:0/1115778805 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7fcdcc06e220 con 0x7fcdcc0ce990 2024-05-08T23:32:35.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.439+0000 7fcdb3fff700 1 -- 172.21.15.81:0/1115778805 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v124) v1 ==== 137+0+0 (secure 0 0 0) 0x7fcdc4054cb0 con 0x7fcdcc0ce990 2024-05-08T23:32:35.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.443+0000 7fcdd0866700 1 -- 172.21.15.81:0/1115778805 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcdb8066730 msgr2=0x7fcdb8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.443+0000 7fcdd0866700 1 --2- 172.21.15.81:0/1115778805 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcdb8066730 0x7fcdb8068bf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00090b0 tx=0x7fcdc001c040 comp rx=0 tx=0).stop 2024-05-08T23:32:35.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.443+0000 7fcdd0866700 1 -- 172.21.15.81:0/1115778805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdcc0ce990 msgr2=0x7fcdcc0ad8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.443+0000 7fcdd0866700 1 --2- 172.21.15.81:0/1115778805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdcc0ce990 0x7fcdcc0ad8c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4001f00 tx=0x7fcdc400a060 comp rx=0 tx=0).stop 2024-05-08T23:32:35.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.443+0000 7fcdd0866700 1 -- 172.21.15.81:0/1115778805 shutdown_connections 2024-05-08T23:32:35.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.443+0000 7fcdd0866700 1 --2- 172.21.15.81:0/1115778805 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcdb8066730 0x7fcdb8068bf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.443+0000 7fcdd0866700 1 --2- 172.21.15.81:0/1115778805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdcc15f770 0x7fcdcc159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.443+0000 7fcdd0866700 1 --2- 172.21.15.81:0/1115778805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdcc15afa0 0x7fcdcc1f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.443+0000 7fcdd0866700 1 --2- 172.21.15.81:0/1115778805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdcc0ce990 0x7fcdcc0ad8c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.443+0000 7fcdd0866700 1 -- 172.21.15.81:0/1115778805 >> 172.21.15.81:0/1115778805 conn(0x7fcdcc06c330 msgr2=0x7fcdcc15a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:35.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.443+0000 7fcdd0866700 1 -- 172.21.15.81:0/1115778805 shutdown_connections 2024-05-08T23:32:35.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.447+0000 7fcdd0866700 1 -- 172.21.15.81:0/1115778805 wait complete. 2024-05-08T23:32:35.464 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T23:32:35.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.583+0000 7fa47b59e700 1 Processor -- start 2024-05-08T23:32:35.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.583+0000 7fa47b59e700 1 -- start start 2024-05-08T23:32:35.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.583+0000 7fa47b59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa47c15ae10 0x7fa47c0ac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.583+0000 7fa47b59e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c06c7d0 con 0x7fa47c0ce6c0 2024-05-08T23:32:35.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.583+0000 7fa47b59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c06c4d0 con 0x7fa47c15ead0 2024-05-08T23:32:35.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.583+0000 7fa47b59e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47c0acd90 con 0x7fa47c15ae10 2024-05-08T23:32:35.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.583+0000 7fa47a59c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa47c15ae10 0x7fa47c0ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.583+0000 7fa47ad9d700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa47c0ce6c0 0x7fa47c06cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55492/0 (socket says 172.21.15.81:55492) 2024-05-08T23:32:35.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.583+0000 7fa47ad9d700 1 -- 172.21.15.81:0/1493591078 learned_addr learned my addr 172.21.15.81:0/1493591078 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:35.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47959a700 1 -- 172.21.15.81:0/1493591078 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411471016 0 0) 0x7fa47c06c7d0 con 0x7fa47c0ce6c0 2024-05-08T23:32:35.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47959a700 1 -- 172.21.15.81:0/1493591078 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4680035d0 con 0x7fa47c0ce6c0 2024-05-08T23:32:35.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47a59c700 1 -- 172.21.15.81:0/1493591078 >> v1:172.21.15.117:6789/0 conn(0x7fa47c15ead0 legacy=0x7fa47c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47a59c700 1 -- 172.21.15.81:0/1493591078 >> v1:172.21.15.81:6789/0 conn(0x7fa47c0ce6c0 legacy=0x7fa47c06cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47a59c700 1 -- 172.21.15.81:0/1493591078 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c0ad070 con 0x7fa47c15ae10 2024-05-08T23:32:35.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47a59c700 1 --2- 172.21.15.81:0/1493591078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa47c15ae10 0x7fa47c0ac420 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa464009880 tx=0x7fa464009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a31a1e6d833547a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47959a700 1 -- 172.21.15.81:0/1493591078 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464011830 con 0x7fa47c15ae10 2024-05-08T23:32:35.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47959a700 1 -- 172.21.15.81:0/1493591078 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa464011990 con 0x7fa47c15ae10 2024-05-08T23:32:35.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47959a700 1 -- 172.21.15.81:0/1493591078 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464011d20 con 0x7fa47c15ae10 2024-05-08T23:32:35.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47b59e700 1 -- 172.21.15.81:0/1493591078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa47c15ae10 msgr2=0x7fa47c0ac420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47b59e700 1 --2- 172.21.15.81:0/1493591078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa47c15ae10 0x7fa47c0ac420 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa464009880 tx=0x7fa464009b90 comp rx=0 tx=0).stop 2024-05-08T23:32:35.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47b59e700 1 -- 172.21.15.81:0/1493591078 shutdown_connections 2024-05-08T23:32:35.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47b59e700 1 --2- 172.21.15.81:0/1493591078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa47c15ae10 0x7fa47c0ac420 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47b59e700 1 -- 172.21.15.81:0/1493591078 >> 172.21.15.81:0/1493591078 conn(0x7fa47c0b0de0 msgr2=0x7fa47c0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:35.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47b59e700 1 -- 172.21.15.81:0/1493591078 shutdown_connections 2024-05-08T23:32:35.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.587+0000 7fa47b59e700 1 -- 172.21.15.81:0/1493591078 wait complete. 2024-05-08T23:32:35.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa47b59e700 1 Processor -- start 2024-05-08T23:32:35.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa47b59e700 1 -- start start 2024-05-08T23:32:35.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa47b59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa47c0ce6c0 0x7fa47c159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa47b59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa47c15ae10 0x7fa47c14ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa47b59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa47c15ead0 0x7fa47c1555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa47b59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0710f0 con 0x7fa47c15ae10 2024-05-08T23:32:35.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa47b59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0700e0 con 0x7fa47c0ce6c0 2024-05-08T23:32:35.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa47b59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c159970 con 0x7fa47c15ead0 2024-05-08T23:32:35.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa479d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa47c15ae10 0x7fa47c14ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa479d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa47c15ae10 0x7fa47c14ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39920/0 (socket says 172.21.15.81:39920) 2024-05-08T23:32:35.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa479d9b700 1 -- 172.21.15.81:0/2070628825 learned_addr learned my addr 172.21.15.81:0/2070628825 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:35.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa47a59c700 1 --2- 172.21.15.81:0/2070628825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa47c0ce6c0 0x7fa47c159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa47ad9d700 1 --2- 172.21.15.81:0/2070628825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa47c15ead0 0x7fa47c1555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa479d9b700 1 -- 172.21.15.81:0/2070628825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa47c15ead0 msgr2=0x7fa47c1555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa479d9b700 1 --2- 172.21.15.81:0/2070628825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa47c15ead0 0x7fa47c1555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa479d9b700 1 -- 172.21.15.81:0/2070628825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa47c0ce6c0 msgr2=0x7fa47c159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa479d9b700 1 --2- 172.21.15.81:0/2070628825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa47c0ce6c0 0x7fa47c159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa479d9b700 1 -- 172.21.15.81:0/2070628825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4640094b0 con 0x7fa47c15ae10 2024-05-08T23:32:35.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa479d9b700 1 --2- 172.21.15.81:0/2070628825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa47c15ae10 0x7fa47c14ddd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00c4d0 tx=0x7fa46c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa472ffd700 1 -- 172.21.15.81:0/2070628825 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c0052d0 con 0x7fa47c15ae10 2024-05-08T23:32:35.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa472ffd700 1 -- 172.21.15.81:0/2070628825 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c005430 con 0x7fa47c15ae10 2024-05-08T23:32:35.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa47b59e700 1 -- 172.21.15.81:0/2070628825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47c070330 con 0x7fa47c15ae10 2024-05-08T23:32:35.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa472ffd700 1 -- 172.21.15.81:0/2070628825 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c0055a0 con 0x7fa47c15ae10 2024-05-08T23:32:35.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.591+0000 7fa47b59e700 1 -- 172.21.15.81:0/2070628825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47c0708f0 con 0x7fa47c15ae10 2024-05-08T23:32:35.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.595+0000 7fa47b59e700 1 -- 172.21.15.81:0/2070628825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47c0684b0 con 0x7fa47c15ae10 2024-05-08T23:32:35.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.595+0000 7fa472ffd700 1 -- 172.21.15.81:0/2070628825 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa46c0038c0 con 0x7fa47c15ae10 2024-05-08T23:32:35.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.595+0000 7fa472ffd700 1 --2- 172.21.15.81:0/2070628825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa45c066760 0x7fa45c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.595+0000 7fa47a59c700 1 --2- 172.21.15.81:0/2070628825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa45c066760 0x7fa45c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.595+0000 7fa472ffd700 1 -- 172.21.15.81:0/2070628825 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa46c083130 con 0x7fa47c15ae10 2024-05-08T23:32:35.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.595+0000 7fa47a59c700 1 --2- 172.21.15.81:0/2070628825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa45c066760 0x7fa45c068c20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa4640090b0 tx=0x7fa464008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.599+0000 7fa472ffd700 1 -- 172.21.15.81:0/2070628825 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa46c0507a0 con 0x7fa47c15ae10 2024-05-08T23:32:35.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.771+0000 7fa47b59e700 1 -- 172.21.15.81:0/2070628825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa47c0b2660 con 0x7fa47c15ae10 2024-05-08T23:32:35.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.775+0000 7fa472ffd700 1 -- 172.21.15.81:0/2070628825 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa46c053dc0 con 0x7fa47c15ae10 2024-05-08T23:32:35.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.779+0000 7fa47b59e700 1 -- 172.21.15.81:0/2070628825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa45c066760 msgr2=0x7fa45c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.779+0000 7fa47b59e700 1 --2- 172.21.15.81:0/2070628825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa45c066760 0x7fa45c068c20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa4640090b0 tx=0x7fa464008030 comp rx=0 tx=0).stop 2024-05-08T23:32:35.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.779+0000 7fa47b59e700 1 -- 172.21.15.81:0/2070628825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa47c15ae10 msgr2=0x7fa47c14ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.779+0000 7fa47b59e700 1 --2- 172.21.15.81:0/2070628825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa47c15ae10 0x7fa47c14ddd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00c4d0 tx=0x7fa46c00c7e0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.779+0000 7fa47b59e700 1 -- 172.21.15.81:0/2070628825 shutdown_connections 2024-05-08T23:32:35.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.779+0000 7fa47b59e700 1 --2- 172.21.15.81:0/2070628825 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa45c066760 0x7fa45c068c20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.779+0000 7fa47b59e700 1 --2- 172.21.15.81:0/2070628825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa47c15ead0 0x7fa47c1555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.779+0000 7fa47b59e700 1 --2- 172.21.15.81:0/2070628825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa47c15ae10 0x7fa47c14ddd0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.779+0000 7fa47b59e700 1 --2- 172.21.15.81:0/2070628825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa47c0ce6c0 0x7fa47c159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.779+0000 7fa47b59e700 1 -- 172.21.15.81:0/2070628825 >> 172.21.15.81:0/2070628825 conn(0x7fa47c0b0de0 msgr2=0x7fa47c159ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:35.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.779+0000 7fa47b59e700 1 -- 172.21.15.81:0/2070628825 shutdown_connections 2024-05-08T23:32:35.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:35.779+0000 7fa47b59e700 1 -- 172.21.15.81:0/2070628825 wait complete. 2024-05-08T23:32:35.796 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:32:35.796 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:32:35.816 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:32:35.835 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:32:35.836 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:32:35.836 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:35.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346332k,nr_inodes=4086583,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14716 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T23:32:35.843 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:32:35.843 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:35.885 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346332k,nr_inodes=4086583,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17609 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T23:32:35.885 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:32:35.885 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:35.893 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346332k,nr_inodes=4086583,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14716 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T23:32:35.894 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:32:35.894 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:35.901 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346332k,nr_inodes=4086583,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17609 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T23:32:35.901 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T23:32:35.902 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:36.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.055+0000 7f4d2b59e700 1 Processor -- start 2024-05-08T23:32:36.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.055+0000 7f4d2b59e700 1 -- start start 2024-05-08T23:32:36.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2b59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4d2c071ab0 0x7f4d2c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:36.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2b59e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d2c0cea50 con 0x7f4d2c15ac70 2024-05-08T23:32:36.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2b59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d2c0ce080 con 0x7f4d2c071e90 2024-05-08T23:32:36.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2b59e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d2c0cd7c0 con 0x7f4d2c071ab0 2024-05-08T23:32:36.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2a59c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4d2c071ab0 0x7f4d2c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:36.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2ad9d700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f4d2c15ac70 0x7f4d2c13d500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35226/0 (socket says 172.21.15.81:35226) 2024-05-08T23:32:36.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2ad9d700 1 -- 172.21.15.81:0/2149682390 learned_addr learned my addr 172.21.15.81:0/2149682390 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:36.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2a59c700 1 -- 172.21.15.81:0/2149682390 >> v1:172.21.15.117:6789/0 conn(0x7f4d2c071e90 legacy=0x7f4d2c15a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:36.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2a59c700 1 -- 172.21.15.81:0/2149682390 >> v1:172.21.15.81:6789/0 conn(0x7f4d2c15ac70 legacy=0x7f4d2c13d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:36.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2a59c700 1 -- 172.21.15.81:0/2149682390 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c0cda30 con 0x7f4d2c071ab0 2024-05-08T23:32:36.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2a59c700 1 --2- 172.21.15.81:0/2149682390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2c071ab0 0x7f4d2c0ce510 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009850 tx=0x7f4d1c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f4edd46e844b5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:36.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d29d9b700 1 -- 172.21.15.81:0/2149682390 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c011830 con 0x7f4d2c071ab0 2024-05-08T23:32:36.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d29d9b700 1 -- 172.21.15.81:0/2149682390 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1c011990 con 0x7f4d2c071ab0 2024-05-08T23:32:36.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d29d9b700 1 -- 172.21.15.81:0/2149682390 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c011d20 con 0x7f4d2c071ab0 2024-05-08T23:32:36.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2b59e700 1 -- 172.21.15.81:0/2149682390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2c071ab0 msgr2=0x7f4d2c0ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:36.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2b59e700 1 --2- 172.21.15.81:0/2149682390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2c071ab0 0x7f4d2c0ce510 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009850 tx=0x7f4d1c009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:36.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2b59e700 1 -- 172.21.15.81:0/2149682390 shutdown_connections 2024-05-08T23:32:36.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2b59e700 1 --2- 172.21.15.81:0/2149682390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2c071ab0 0x7f4d2c0ce510 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2b59e700 1 -- 172.21.15.81:0/2149682390 >> 172.21.15.81:0/2149682390 conn(0x7f4d2c06c330 msgr2=0x7f4d2c06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:36.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2b59e700 1 -- 172.21.15.81:0/2149682390 shutdown_connections 2024-05-08T23:32:36.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.059+0000 7f4d2b59e700 1 -- 172.21.15.81:0/2149682390 wait complete. 2024-05-08T23:32:36.064 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2b59e700 1 Processor -- start 2024-05-08T23:32:36.064 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2b59e700 1 -- start start 2024-05-08T23:32:36.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2b59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d2c071ab0 0x7f4d2c12be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:36.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2b59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d2c071e90 0x7f4d2c0b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:36.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2b59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2c15ac70 0x7f4d2c1f0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:36.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2b59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c0b4800 con 0x7f4d2c15ac70 2024-05-08T23:32:36.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2b59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c0b3990 con 0x7f4d2c071ab0 2024-05-08T23:32:36.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2b59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c1f0dc0 con 0x7f4d2c071e90 2024-05-08T23:32:36.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2c15ac70 0x7f4d2c1f0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:36.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2c15ac70 0x7f4d2c1f0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34106/0 (socket says 172.21.15.81:34106) 2024-05-08T23:32:36.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2ad9d700 1 -- 172.21.15.81:0/766406541 learned_addr learned my addr 172.21.15.81:0/766406541 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:36.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2a59c700 1 --2- 172.21.15.81:0/766406541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d2c071ab0 0x7f4d2c12be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:36.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d21d9b700 1 --2- 172.21.15.81:0/766406541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d2c071e90 0x7f4d2c0b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:36.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2ad9d700 1 -- 172.21.15.81:0/766406541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d2c071e90 msgr2=0x7f4d2c0b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:36.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2ad9d700 1 --2- 172.21.15.81:0/766406541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d2c071e90 0x7f4d2c0b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2ad9d700 1 -- 172.21.15.81:0/766406541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d2c071ab0 msgr2=0x7f4d2c12be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:36.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2ad9d700 1 --2- 172.21.15.81:0/766406541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d2c071ab0 0x7f4d2c12be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2ad9d700 1 -- 172.21.15.81:0/766406541 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c0094b0 con 0x7f4d2c15ac70 2024-05-08T23:32:36.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d21d9b700 1 --2- 172.21.15.81:0/766406541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d2c071e90 0x7f4d2c0b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:36.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.063+0000 7f4d2ad9d700 1 --2- 172.21.15.81:0/766406541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2c15ac70 0x7f4d2c1f0880 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400cbc0 tx=0x7f4d2400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:36.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.067+0000 7f4d237fe700 1 -- 172.21.15.81:0/766406541 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2400a7b0 con 0x7f4d2c15ac70 2024-05-08T23:32:36.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.067+0000 7f4d237fe700 1 -- 172.21.15.81:0/766406541 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2400a910 con 0x7f4d2c15ac70 2024-05-08T23:32:36.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.067+0000 7f4d2b59e700 1 -- 172.21.15.81:0/766406541 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2c0b3bc0 con 0x7f4d2c15ac70 2024-05-08T23:32:36.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.067+0000 7f4d237fe700 1 -- 172.21.15.81:0/766406541 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2400ac80 con 0x7f4d2c15ac70 2024-05-08T23:32:36.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.067+0000 7f4d2b59e700 1 -- 172.21.15.81:0/766406541 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2c0b4140 con 0x7f4d2c15ac70 2024-05-08T23:32:36.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.067+0000 7f4d2b59e700 1 -- 172.21.15.81:0/766406541 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2c0b4940 con 0x7f4d2c15ac70 2024-05-08T23:32:36.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.067+0000 7f4d237fe700 1 -- 172.21.15.81:0/766406541 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4d24012400 con 0x7f4d2c15ac70 2024-05-08T23:32:36.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.067+0000 7f4d237fe700 1 --2- 172.21.15.81:0/766406541 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4d10066730 0x7f4d10068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:36.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.067+0000 7f4d237fe700 1 -- 172.21.15.81:0/766406541 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4d24083510 con 0x7f4d2c15ac70 2024-05-08T23:32:36.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.067+0000 7f4d2a59c700 1 --2- 172.21.15.81:0/766406541 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4d10066730 0x7f4d10068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:36.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.067+0000 7f4d2a59c700 1 --2- 172.21.15.81:0/766406541 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4d10066730 0x7f4d10068bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c014040 tx=0x7f4d1c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:36.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.075+0000 7f4d237fe700 1 -- 172.21.15.81:0/766406541 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4d2c0b4940 con 0x7f4d2c15ac70 2024-05-08T23:32:36.200 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:32:36.202 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:32:36.288 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.287+0000 7f4d2b59e700 1 -- 172.21.15.81:0/766406541 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d2c0cea50 con 0x7f4d2c15ac70 2024-05-08T23:32:36.289 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.287+0000 7f4d237fe700 1 -- 172.21.15.81:0/766406541 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+3536 (secure 0 0 0) 0x7f4d24017de0 con 0x7f4d2c15ac70 2024-05-08T23:32:36.289 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:36.289 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":17,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1243037934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1243037934},{"type":"v1","addr":"172.21.15.81:6835","nonce":1243037934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3124115182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3124115182},{"type":"v1","addr":"172.21.15.81:6837","nonce":3124115182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1423349691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":1423349691},{"type":"v1","addr":"172.21.15.117:6834","nonce":1423349691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":15251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3105055764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3105055764},{"type":"v1","addr":"172.21.15.117:6835","nonce":3105055764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[]} 2024-05-08T23:32:36.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.287+0000 7f4d2b59e700 1 -- 172.21.15.81:0/766406541 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4d10066730 msgr2=0x7f4d10068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:36.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.287+0000 7f4d2b59e700 1 --2- 172.21.15.81:0/766406541 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4d10066730 0x7f4d10068bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c014040 tx=0x7f4d1c01c040 comp rx=0 tx=0).stop 2024-05-08T23:32:36.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.287+0000 7f4d2b59e700 1 -- 172.21.15.81:0/766406541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2c15ac70 msgr2=0x7f4d2c1f0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:36.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.287+0000 7f4d2b59e700 1 --2- 172.21.15.81:0/766406541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2c15ac70 0x7f4d2c1f0880 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400cbc0 tx=0x7f4d2400cbf0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.291+0000 7f4d2b59e700 1 -- 172.21.15.81:0/766406541 shutdown_connections 2024-05-08T23:32:36.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.291+0000 7f4d2b59e700 1 --2- 172.21.15.81:0/766406541 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4d10066730 0x7f4d10068bf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.291+0000 7f4d2b59e700 1 --2- 172.21.15.81:0/766406541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d2c15ac70 0x7f4d2c1f0880 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.291+0000 7f4d2b59e700 1 --2- 172.21.15.81:0/766406541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d2c071e90 0x7f4d2c0b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.291+0000 7f4d2b59e700 1 --2- 172.21.15.81:0/766406541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d2c071ab0 0x7f4d2c12be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.291+0000 7f4d2b59e700 1 -- 172.21.15.81:0/766406541 >> 172.21.15.81:0/766406541 conn(0x7f4d2c06c330 msgr2=0x7f4d2c13d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:36.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.291+0000 7f4d2b59e700 1 -- 172.21.15.81:0/766406541 shutdown_connections 2024-05-08T23:32:36.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.291+0000 7f4d2b59e700 1 -- 172.21.15.81:0/766406541 wait complete. 2024-05-08T23:32:36.293 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 17 2024-05-08T23:32:36.304 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T23:32:36.304 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T23:32:36.304 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T23:32:36.305 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:32:36.305 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T23:32:36.305 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T23:32:36.305 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T23:32:36.305 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:32:36.305 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T23:32:36.305 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T23:32:36.306 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T23:32:36.306 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:32:36.306 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T23:32:36.306 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T23:32:36.306 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T23:32:36.306 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:32:36.316 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T23:32:36.316 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T23:32:36.318 INFO:tasks.ceph.mds.a:Started 2024-05-08T23:32:36.319 DEBUG:teuthology.parallel:result is None 2024-05-08T23:32:36.321 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T23:32:36.321 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T23:32:36.322 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T23:32:36.322 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T23:32:36.324 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T23:32:36.324 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T23:32:36.325 INFO:tasks.ceph.mds.d:Started 2024-05-08T23:32:36.325 DEBUG:teuthology.parallel:result is None 2024-05-08T23:32:36.326 INFO:tasks.ceph.mds.c:Started 2024-05-08T23:32:36.326 DEBUG:teuthology.parallel:result is None 2024-05-08T23:32:36.327 INFO:tasks.ceph.mds.b:Started 2024-05-08T23:32:36.327 DEBUG:teuthology.parallel:result is None 2024-05-08T23:32:36.327 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T23:32:36.394 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2024-05-08T23:32:36.406 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-05-08T23:32:36.406 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-05-08T23:32:36.413 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2024-05-08T23:32:36.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.439+0000 7fde1c879700 1 Processor -- start 2024-05-08T23:32:36.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.439+0000 7fde1c879700 1 -- start start 2024-05-08T23:32:36.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde1c879700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fde181596a0 0x7fde180b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:36.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde1c879700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fde180b4e50 0x7fde18159a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:36.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde1c879700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde180b3590 con 0x7fde18159fc0 2024-05-08T23:32:36.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde1c879700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde180b24f0 con 0x7fde180b4e50 2024-05-08T23:32:36.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde1659c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fde180b4e50 0x7fde18159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:36.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde1c879700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde180b27d0 con 0x7fde181596a0 2024-05-08T23:32:36.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde1659c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fde180b4e50 0x7fde18159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34126/0 (socket says 172.21.15.81:34126) 2024-05-08T23:32:36.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde1659c700 1 -- 172.21.15.81:0/3038531009 learned_addr learned my addr 172.21.15.81:0/3038531009 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:36.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde16d9d700 1 --2- 172.21.15.81:0/3038531009 >> v2:172.21.15.117:3300/0 conn(0x7fde181596a0 0x7fde180b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:36.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde1659c700 1 -- 172.21.15.81:0/3038531009 >> v2:172.21.15.117:3300/0 conn(0x7fde181596a0 msgr2=0x7fde180b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:36.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde1659c700 1 --2- 172.21.15.81:0/3038531009 >> v2:172.21.15.117:3300/0 conn(0x7fde181596a0 0x7fde180b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde1659c700 1 -- 172.21.15.81:0/3038531009 >> v1:172.21.15.117:6789/0 conn(0x7fde18159fc0 legacy=0x7fde1815d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:36.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde1659c700 1 -- 172.21.15.81:0/3038531009 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde180b2ab0 con 0x7fde180b4e50 2024-05-08T23:32:36.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde16d9d700 1 --2- 172.21.15.81:0/3038531009 >> v2:172.21.15.117:3300/0 conn(0x7fde181596a0 0x7fde180b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:36.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde1659c700 1 --2- 172.21.15.81:0/3038531009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde180b4e50 0x7fde18159a80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fde180cee40 tx=0x7fde0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a1c0de3075afd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:36.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde15d9b700 1 -- 172.21.15.81:0/3038531009 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0800e070 con 0x7fde180b4e50 2024-05-08T23:32:36.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde15d9b700 1 -- 172.21.15.81:0/3038531009 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0800b950 con 0x7fde180b4e50 2024-05-08T23:32:36.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.443+0000 7fde15d9b700 1 -- 172.21.15.81:0/3038531009 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0800bd00 con 0x7fde180b4e50 2024-05-08T23:32:36.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 -- 172.21.15.81:0/3038531009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde180b4e50 msgr2=0x7fde18159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:36.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 --2- 172.21.15.81:0/3038531009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde180b4e50 0x7fde18159a80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fde180cee40 tx=0x7fde0800b330 comp rx=0 tx=0).stop 2024-05-08T23:32:36.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 -- 172.21.15.81:0/3038531009 shutdown_connections 2024-05-08T23:32:36.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 --2- 172.21.15.81:0/3038531009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde180b4e50 0x7fde18159a80 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 --2- 172.21.15.81:0/3038531009 >> v2:172.21.15.117:3300/0 conn(0x7fde181596a0 0x7fde180b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 -- 172.21.15.81:0/3038531009 >> 172.21.15.81:0/3038531009 conn(0x7fde1806c330 msgr2=0x7fde1806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:36.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 -- 172.21.15.81:0/3038531009 shutdown_connections 2024-05-08T23:32:36.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 -- 172.21.15.81:0/3038531009 wait complete. 2024-05-08T23:32:36.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 Processor -- start 2024-05-08T23:32:36.450 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 -- start start 2024-05-08T23:32:36.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde180b4e50 0x7fde18150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:36.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde181596a0 0x7fde180ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:36.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde18159fc0 0x7fde180afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:36.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde18150e80 con 0x7fde180b4e50 2024-05-08T23:32:36.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde18155890 con 0x7fde18159fc0 2024-05-08T23:32:36.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1c879700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde18155b70 con 0x7fde181596a0 2024-05-08T23:32:36.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde16d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde180b4e50 0x7fde18150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:36.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde16d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde180b4e50 0x7fde18150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34132/0 (socket says 172.21.15.81:34132) 2024-05-08T23:32:36.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde16d9d700 1 -- 172.21.15.81:0/416618690 learned_addr learned my addr 172.21.15.81:0/416618690 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:36.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1759e700 1 --2- 172.21.15.81:0/416618690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde18159fc0 0x7fde180afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:36.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.447+0000 7fde1659c700 1 --2- 172.21.15.81:0/416618690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde181596a0 0x7fde180ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:36.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde16d9d700 1 -- 172.21.15.81:0/416618690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde181596a0 msgr2=0x7fde180ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:36.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde16d9d700 1 --2- 172.21.15.81:0/416618690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde181596a0 0x7fde180ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde16d9d700 1 -- 172.21.15.81:0/416618690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde18159fc0 msgr2=0x7fde180afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:36.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde16d9d700 1 --2- 172.21.15.81:0/416618690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde18159fc0 0x7fde180afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde16d9d700 1 -- 172.21.15.81:0/416618690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde08009b30 con 0x7fde180b4e50 2024-05-08T23:32:36.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde1659c700 1 --2- 172.21.15.81:0/416618690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde181596a0 0x7fde180ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:36.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde16d9d700 1 --2- 172.21.15.81:0/416618690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde180b4e50 0x7fde18150940 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fde1000e810 tx=0x7fde1000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:36.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde0f7fe700 1 -- 172.21.15.81:0/416618690 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10013610 con 0x7fde180b4e50 2024-05-08T23:32:36.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde1c879700 1 -- 172.21.15.81:0/416618690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde18155e50 con 0x7fde180b4e50 2024-05-08T23:32:36.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde0f7fe700 1 -- 172.21.15.81:0/416618690 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde10013770 con 0x7fde180b4e50 2024-05-08T23:32:36.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde0f7fe700 1 -- 172.21.15.81:0/416618690 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10013b00 con 0x7fde180b4e50 2024-05-08T23:32:36.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde1c879700 1 -- 172.21.15.81:0/416618690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde18156410 con 0x7fde180b4e50 2024-05-08T23:32:36.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde1c879700 1 -- 172.21.15.81:0/416618690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde180684b0 con 0x7fde180b4e50 2024-05-08T23:32:36.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde0f7fe700 1 -- 172.21.15.81:0/416618690 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fde10013c60 con 0x7fde180b4e50 2024-05-08T23:32:36.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde0f7fe700 1 --2- 172.21.15.81:0/416618690 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fddf8066730 0x7fddf8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:36.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde1659c700 1 --2- 172.21.15.81:0/416618690 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fddf8066730 0x7fddf8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:36.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde0f7fe700 1 -- 172.21.15.81:0/416618690 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fde10083ac0 con 0x7fde180b4e50 2024-05-08T23:32:36.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.451+0000 7fde1659c700 1 --2- 172.21.15.81:0/416618690 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fddf8066730 0x7fddf8068bf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fde180b0c30 tx=0x7fde08009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:36.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.459+0000 7fde0f7fe700 1 -- 172.21.15.81:0/416618690 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fde10051140 con 0x7fde180b4e50 2024-05-08T23:32:36.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:36.639+0000 7fde1c879700 1 -- 172.21.15.81:0/416618690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fde180ceb50 con 0x7fde180b4e50 2024-05-08T23:32:37.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.459+0000 7fde0f7fe700 1 -- 172.21.15.81:0/416618690 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) v1 ==== 97+0+0 (secure 0 0 0) 0x7fde10054760 con 0x7fde180b4e50 2024-05-08T23:32:37.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.463+0000 7fde1c879700 1 -- 172.21.15.81:0/416618690 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fddf8066730 msgr2=0x7fddf8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:37.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.463+0000 7fde1c879700 1 --2- 172.21.15.81:0/416618690 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fddf8066730 0x7fddf8068bf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fde180b0c30 tx=0x7fde08009000 comp rx=0 tx=0).stop 2024-05-08T23:32:37.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.463+0000 7fde1c879700 1 -- 172.21.15.81:0/416618690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde180b4e50 msgr2=0x7fde18150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:37.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.463+0000 7fde1c879700 1 --2- 172.21.15.81:0/416618690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde180b4e50 0x7fde18150940 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fde1000e810 tx=0x7fde1000ebd0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.463+0000 7fde1c879700 1 -- 172.21.15.81:0/416618690 shutdown_connections 2024-05-08T23:32:37.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.463+0000 7fde1c879700 1 --2- 172.21.15.81:0/416618690 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fddf8066730 0x7fddf8068bf0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.463+0000 7fde1c879700 1 --2- 172.21.15.81:0/416618690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde18159fc0 0x7fde180afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.463+0000 7fde1c879700 1 --2- 172.21.15.81:0/416618690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde181596a0 0x7fde180ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.463+0000 7fde1c879700 1 --2- 172.21.15.81:0/416618690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde180b4e50 0x7fde18150940 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.463+0000 7fde1c879700 1 -- 172.21.15.81:0/416618690 >> 172.21.15.81:0/416618690 conn(0x7fde1806c330 msgr2=0x7fde180b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:37.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.463+0000 7fde1c879700 1 -- 172.21.15.81:0/416618690 shutdown_connections 2024-05-08T23:32:37.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.463+0000 7fde1c879700 1 -- 172.21.15.81:0/416618690 wait complete. 2024-05-08T23:32:37.469 INFO:teuthology.orchestra.run.smithi081.stderr: removed all blocklist entries 2024-05-08T23:32:37.483 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T23:32:37.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.599+0000 7f8509531700 1 Processor -- start 2024-05-08T23:32:37.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.599+0000 7f8509531700 1 -- start start 2024-05-08T23:32:37.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.599+0000 7f8509531700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f85040d1690 0x7f85040d37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:37.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.599+0000 7f8509531700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f85040d0f20 0x7f85040d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:37.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.599+0000 7f8509531700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85040d1fb0 con 0x7f85040cf6e0 2024-05-08T23:32:37.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.599+0000 7f8509531700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85040d3e50 con 0x7f85040d0f20 2024-05-08T23:32:37.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.599+0000 7f8509531700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85040ce870 con 0x7f85040d1690 2024-05-08T23:32:37.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.599+0000 7f85037fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f85040d0f20 0x7f85040d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:37.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.599+0000 7f85037fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f85040d0f20 0x7f85040d1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34142/0 (socket says 172.21.15.81:34142) 2024-05-08T23:32:37.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.599+0000 7f85037fe700 1 -- 172.21.15.81:0/2353163991 learned_addr learned my addr 172.21.15.81:0/2353163991 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:37.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.599+0000 7f8503fff700 1 --2- 172.21.15.81:0/2353163991 >> v2:172.21.15.117:3300/0 conn(0x7f85040d1690 0x7f85040d37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:37.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.599+0000 7f85037fe700 1 -- 172.21.15.81:0/2353163991 >> v2:172.21.15.117:3300/0 conn(0x7f85040d1690 msgr2=0x7f85040d37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:37.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f85037fe700 1 --2- 172.21.15.81:0/2353163991 >> v2:172.21.15.117:3300/0 conn(0x7f85040d1690 0x7f85040d37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f85037fe700 1 -- 172.21.15.81:0/2353163991 >> v1:172.21.15.117:6789/0 conn(0x7f85040cf6e0 legacy=0x7f85040cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:37.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f85037fe700 1 -- 172.21.15.81:0/2353163991 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85040ceb50 con 0x7f85040d0f20 2024-05-08T23:32:37.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f8503fff700 1 --2- 172.21.15.81:0/2353163991 >> v2:172.21.15.117:3300/0 conn(0x7f85040d1690 0x7f85040d37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:37.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f85037fe700 1 --2- 172.21.15.81:0/2353163991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85040d0f20 0x7f85040d1a70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f85040b5ca0 tx=0x7f84f4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aeb04b0eb584834 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:37.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f8502ffd700 1 -- 172.21.15.81:0/2353163991 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f400b8a0 con 0x7f85040d0f20 2024-05-08T23:32:37.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f8502ffd700 1 -- 172.21.15.81:0/2353163991 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f400ba00 con 0x7f85040d0f20 2024-05-08T23:32:37.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f8502ffd700 1 -- 172.21.15.81:0/2353163991 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f400bd90 con 0x7f85040d0f20 2024-05-08T23:32:37.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f8509531700 1 -- 172.21.15.81:0/2353163991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85040d0f20 msgr2=0x7f85040d1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:37.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f8509531700 1 --2- 172.21.15.81:0/2353163991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85040d0f20 0x7f85040d1a70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f85040b5ca0 tx=0x7f84f4009bf0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f8509531700 1 -- 172.21.15.81:0/2353163991 shutdown_connections 2024-05-08T23:32:37.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f8509531700 1 --2- 172.21.15.81:0/2353163991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85040d0f20 0x7f85040d1a70 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f8509531700 1 --2- 172.21.15.81:0/2353163991 >> v2:172.21.15.117:3300/0 conn(0x7f85040d1690 0x7f85040d37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f8509531700 1 -- 172.21.15.81:0/2353163991 >> 172.21.15.81:0/2353163991 conn(0x7f850409c3c0 msgr2=0x7f85040d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:37.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f8509531700 1 -- 172.21.15.81:0/2353163991 shutdown_connections 2024-05-08T23:32:37.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f8509531700 1 -- 172.21.15.81:0/2353163991 wait complete. 2024-05-08T23:32:37.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f8509531700 1 Processor -- start 2024-05-08T23:32:37.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.603+0000 7f8509531700 1 -- start start 2024-05-08T23:32:37.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8509531700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85040cf6e0 0x7f8504151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:37.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8509531700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85040d0f20 0x7f85041f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:37.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8509531700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85040d1690 0x7f850415b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:37.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8509531700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85040b7520 con 0x7f85040d1690 2024-05-08T23:32:37.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8509531700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85041f3a90 con 0x7f85040d0f20 2024-05-08T23:32:37.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8509531700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85040b68a0 con 0x7f85040cf6e0 2024-05-08T23:32:37.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8508d30700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85040d1690 0x7f850415b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:37.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8503fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85040cf6e0 0x7f8504151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:37.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f85037fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85040d0f20 0x7f85041f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:37.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8508d30700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85040d1690 0x7f850415b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34146/0 (socket says 172.21.15.81:34146) 2024-05-08T23:32:37.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8503fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85040cf6e0 0x7f8504151ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:49830/0 (socket says 172.21.15.81:49830) 2024-05-08T23:32:37.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8508d30700 1 -- 172.21.15.81:0/565272325 learned_addr learned my addr 172.21.15.81:0/565272325 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:37.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8508d30700 1 -- 172.21.15.81:0/565272325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85040cf6e0 msgr2=0x7f8504151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:37.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8508d30700 1 --2- 172.21.15.81:0/565272325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85040cf6e0 0x7f8504151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8508d30700 1 -- 172.21.15.81:0/565272325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85040d0f20 msgr2=0x7f85041f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:37.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8508d30700 1 --2- 172.21.15.81:0/565272325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85040d0f20 0x7f85041f3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8508d30700 1 -- 172.21.15.81:0/565272325 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f4009570 con 0x7f85040d1690 2024-05-08T23:32:37.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8508d30700 1 --2- 172.21.15.81:0/565272325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85040d1690 0x7f850415b170 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00cbc0 tx=0x7f84fc00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:37.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8500ff9700 1 -- 172.21.15.81:0/565272325 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc003c20 con 0x7f85040d1690 2024-05-08T23:32:37.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8500ff9700 1 -- 172.21.15.81:0/565272325 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc003d80 con 0x7f85040d1690 2024-05-08T23:32:37.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8509531700 1 -- 172.21.15.81:0/565272325 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85040b6b80 con 0x7f85040d1690 2024-05-08T23:32:37.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8500ff9700 1 -- 172.21.15.81:0/565272325 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc00aa20 con 0x7f85040d1690 2024-05-08T23:32:37.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.607+0000 7f8509531700 1 -- 172.21.15.81:0/565272325 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85040b7140 con 0x7f85040d1690 2024-05-08T23:32:37.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.611+0000 7f8509531700 1 -- 172.21.15.81:0/565272325 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850415ed90 con 0x7f85040d1690 2024-05-08T23:32:37.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.611+0000 7f8500ff9700 1 -- 172.21.15.81:0/565272325 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f84fc00ab80 con 0x7f85040d1690 2024-05-08T23:32:37.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.611+0000 7f8500ff9700 1 --2- 172.21.15.81:0/565272325 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f84e4066730 0x7f84e4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:37.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.611+0000 7f8503fff700 1 --2- 172.21.15.81:0/565272325 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f84e4066730 0x7f84e4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:37.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.611+0000 7f8500ff9700 1 -- 172.21.15.81:0/565272325 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f84fc083cd0 con 0x7f85040d1690 2024-05-08T23:32:37.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.611+0000 7f8503fff700 1 --2- 172.21.15.81:0/565272325 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f84e4066730 0x7f84e4068bf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00c960 tx=0x7f84ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:37.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.619+0000 7f8500ff9700 1 -- 172.21.15.81:0/565272325 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f850415ed90 con 0x7f85040d1690 2024-05-08T23:32:37.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.851+0000 7f8509531700 1 -- 172.21.15.81:0/565272325 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f850415ed90 con 0x7f85040d1690 2024-05-08T23:32:37.852 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.851+0000 7f8500ff9700 1 -- 172.21.15.81:0/565272325 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f850415ed90 con 0x7f85040d1690 2024-05-08T23:32:37.852 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:37.852 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:32:37.852 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_dump": [ 2024-05-08T23:32:37.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.a", 2024-05-08T23:32:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmmtpFBBAAO8W7v1ZRGwo4hFj/fYPECw==", 2024-05-08T23:32:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:32:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2024-05-08T23:32:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2024-05-08T23:32:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:32:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.b", 2024-05-08T23:32:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxms5vJCxAA7vARVP0vvogEEps6DJFK1g==", 2024-05-08T23:32:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:32:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2024-05-08T23:32:37.854 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2024-05-08T23:32:37.854 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:32:37.854 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.854 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.c", 2024-05-08T23:32:37.854 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmXD37BxAALgQN2Yixer9fPYj2YdLW/w==", 2024-05-08T23:32:37.854 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.854 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:32:37.854 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2024-05-08T23:32:37.854 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2024-05-08T23:32:37.854 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:32:37.854 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.855 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.d", 2024-05-08T23:32:37.855 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmpIKFDxAACxYpOsx1laPsl0QNG+NFdw==", 2024-05-08T23:32:37.855 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.855 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:32:37.855 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2024-05-08T23:32:37.855 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2024-05-08T23:32:37.855 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:32:37.855 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.855 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.0", 2024-05-08T23:32:37.855 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDkCjxmUOfLCBAAc0gYW9BV8BE+h7Qhhc01IQ==", 2024-05-08T23:32:37.855 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.855 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:32:37.855 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:32:37.856 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:32:37.856 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.856 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.1", 2024-05-08T23:32:37.856 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDnCjxmKcSHDBAA5dF6MWsChaZUZQskZYtrKg==", 2024-05-08T23:32:37.856 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.856 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:32:37.856 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:32:37.856 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:32:37.856 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.856 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.2", 2024-05-08T23:32:37.856 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDqCjxmeYjnEhAAAVMSJ8jGR+dThJL51VsQxA==", 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.3", 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDtCjxmlPycFhAAx+amZqgwyvwydi2d5dBZmg==", 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.858 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.4", 2024-05-08T23:32:37.858 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDyCjxmcnu+NxAAJ5cn9LG2uTjAZzW2Qv2y5A==", 2024-05-08T23:32:37.858 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.858 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:32:37.858 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:32:37.858 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:32:37.858 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.858 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.5", 2024-05-08T23:32:37.858 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQD2CjxmQNJoARAAfINQxQDKESMqyuxmEFwwJA==", 2024-05-08T23:32:37.858 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.858 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:32:37.858 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:32:37.858 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:32:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.6", 2024-05-08T23:32:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQD5CjxmGcKCCRAApkAe4V2JvttFUqcwVa7PbA==", 2024-05-08T23:32:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:32:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:32:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:32:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.7", 2024-05-08T23:32:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQD8CjxmIbmICxAAqF8ZE7Gxj1pLKuaCvqOIRw==", 2024-05-08T23:32:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.0", 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmQWD3EhAAkS5P2FRmiLBI9qjk3JjiyA==", 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow r", 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rw pool=cephfs_data" 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.860 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.1", 2024-05-08T23:32:37.861 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmuQBJGRAAA7KQ8CtfMfkVgM89n1QBMA==", 2024-05-08T23:32:37.861 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.861 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:32:37.861 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2024-05-08T23:32:37.861 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2024-05-08T23:32:37.861 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2024-05-08T23:32:37.861 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.861 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.861 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.2", 2024-05-08T23:32:37.861 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmhpPVFRAA2gIDHl4KehKcgXKX0Rs9tg==", 2024-05-08T23:32:37.861 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.861 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:32:37.861 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2024-05-08T23:32:37.861 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.3", 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmf6gIHxAARsUHPVNk3HzYs3lQyqt2Cg==", 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.admin", 2024-05-08T23:32:37.862 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDgCjxmh+ZhMBAA8ifuuRdCV2SQaSOLTWhPsw==", 2024-05-08T23:32:37.863 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.863 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2024-05-08T23:32:37.863 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2024-05-08T23:32:37.863 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2024-05-08T23:32:37.863 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:32:37.863 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.863 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mgr.x", 2024-05-08T23:32:37.863 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDhCjxmUtdWABAARBlm/xpvB6mlamPG1+thYg==", 2024-05-08T23:32:37.863 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.863 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2024-05-08T23:32:37.863 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile mgr", 2024-05-08T23:32:37.863 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:32:37.863 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:37.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:37.864 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mgr.y", 2024-05-08T23:32:37.864 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQDgCjxmgG3AOBAAK6T63BGID2I5zE0Z3VR4TQ==", 2024-05-08T23:32:37.864 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2024-05-08T23:32:37.864 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2024-05-08T23:32:37.864 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile mgr", 2024-05-08T23:32:37.864 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2024-05-08T23:32:37.864 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.864 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:37.864 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-05-08T23:32:37.864 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:32:37.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.851+0000 7f8509531700 1 -- 172.21.15.81:0/565272325 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f84e4066730 msgr2=0x7f84e4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:37.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.851+0000 7f8509531700 1 --2- 172.21.15.81:0/565272325 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f84e4066730 0x7f84e4068bf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00c960 tx=0x7f84ec008040 comp rx=0 tx=0).stop 2024-05-08T23:32:37.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.851+0000 7f8509531700 1 -- 172.21.15.81:0/565272325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85040d1690 msgr2=0x7f850415b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:37.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.851+0000 7f8509531700 1 --2- 172.21.15.81:0/565272325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85040d1690 0x7f850415b170 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00cbc0 tx=0x7f84fc00ced0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.851+0000 7f8509531700 1 -- 172.21.15.81:0/565272325 shutdown_connections 2024-05-08T23:32:37.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.851+0000 7f8509531700 1 --2- 172.21.15.81:0/565272325 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f84e4066730 0x7f84e4068bf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.851+0000 7f8509531700 1 --2- 172.21.15.81:0/565272325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85040d1690 0x7f850415b170 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.851+0000 7f8509531700 1 --2- 172.21.15.81:0/565272325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85040d0f20 0x7f85041f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.851+0000 7f8509531700 1 --2- 172.21.15.81:0/565272325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85040cf6e0 0x7f8504151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.851+0000 7f8509531700 1 -- 172.21.15.81:0/565272325 >> 172.21.15.81:0/565272325 conn(0x7f850409c3c0 msgr2=0x7f85040b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:37.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.851+0000 7f8509531700 1 -- 172.21.15.81:0/565272325 shutdown_connections 2024-05-08T23:32:37.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.851+0000 7f8509531700 1 -- 172.21.15.81:0/565272325 wait complete. 2024-05-08T23:32:37.868 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:37.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe161bb6700 1 Processor -- start 2024-05-08T23:32:37.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe161bb6700 1 -- start start 2024-05-08T23:32:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe161bb6700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe15c071ab0 0x7fe15c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe161bb6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe15c15ac70 0x7fe15c13d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe161bb6700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe15c0cea50 con 0x7fe15c071e90 2024-05-08T23:32:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe161bb6700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15c0ce080 con 0x7fe15c071ab0 2024-05-08T23:32:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe161bb6700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c0cd7c0 con 0x7fe15c15ac70 2024-05-08T23:32:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe15bfff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe15c071e90 0x7fe15c15a560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35258/0 (socket says 172.21.15.81:35258) 2024-05-08T23:32:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe15bfff700 1 -- 172.21.15.81:0/428195864 learned_addr learned my addr 172.21.15.81:0/428195864 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe160bb4700 1 --2- 172.21.15.81:0/428195864 >> v2:172.21.15.81:3300/0 conn(0x7fe15c071ab0 0x7fe15c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe1613b5700 1 --2- 172.21.15.81:0/428195864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe15c15ac70 0x7fe15c13d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe15b7fe700 1 -- 172.21.15.81:0/428195864 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631376157 0 0) 0x7fe144001110 con 0x7fe15c071e90 2024-05-08T23:32:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe15b7fe700 1 -- 172.21.15.81:0/428195864 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe13c0035d0 con 0x7fe15c071e90 2024-05-08T23:32:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe160bb4700 1 -- 172.21.15.81:0/428195864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe15c15ac70 msgr2=0x7fe15c13d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:37.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe160bb4700 1 --2- 172.21.15.81:0/428195864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe15c15ac70 0x7fe15c13d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe160bb4700 1 -- 172.21.15.81:0/428195864 >> v1:172.21.15.81:6789/0 conn(0x7fe15c071e90 legacy=0x7fe15c15a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:37.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe160bb4700 1 -- 172.21.15.81:0/428195864 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c0cda30 con 0x7fe15c071ab0 2024-05-08T23:32:37.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe160bb4700 1 --2- 172.21.15.81:0/428195864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe15c071ab0 0x7fe15c0ce510 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe154009d90 tx=0x7fe15400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94a04634fa6a874 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:37.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe15b7fe700 1 -- 172.21.15.81:0/428195864 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15400e070 con 0x7fe15c071ab0 2024-05-08T23:32:37.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe15b7fe700 1 -- 172.21.15.81:0/428195864 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15400bb20 con 0x7fe15c071ab0 2024-05-08T23:32:37.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.975+0000 7fe15b7fe700 1 -- 172.21.15.81:0/428195864 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15400be90 con 0x7fe15c071ab0 2024-05-08T23:32:37.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.979+0000 7fe161bb6700 1 -- 172.21.15.81:0/428195864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe15c071ab0 msgr2=0x7fe15c0ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:37.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.979+0000 7fe161bb6700 1 --2- 172.21.15.81:0/428195864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe15c071ab0 0x7fe15c0ce510 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe154009d90 tx=0x7fe15400b330 comp rx=0 tx=0).stop 2024-05-08T23:32:37.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.979+0000 7fe161bb6700 1 -- 172.21.15.81:0/428195864 shutdown_connections 2024-05-08T23:32:37.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.979+0000 7fe161bb6700 1 --2- 172.21.15.81:0/428195864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe15c15ac70 0x7fe15c13d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.979+0000 7fe161bb6700 1 --2- 172.21.15.81:0/428195864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe15c071ab0 0x7fe15c0ce510 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.979+0000 7fe161bb6700 1 -- 172.21.15.81:0/428195864 >> 172.21.15.81:0/428195864 conn(0x7fe15c06c330 msgr2=0x7fe15c06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:37.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.979+0000 7fe161bb6700 1 -- 172.21.15.81:0/428195864 shutdown_connections 2024-05-08T23:32:37.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.979+0000 7fe161bb6700 1 -- 172.21.15.81:0/428195864 wait complete. 2024-05-08T23:32:37.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.979+0000 7fe161bb6700 1 Processor -- start 2024-05-08T23:32:37.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.979+0000 7fe161bb6700 1 -- start start 2024-05-08T23:32:37.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe161bb6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe15c071ab0 0x7fe15c0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe161bb6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe15c071e90 0x7fe15c1f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe161bb6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe15c15ac70 0x7fe15c1f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe161bb6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c1f6740 con 0x7fe15c071ab0 2024-05-08T23:32:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe161bb6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c1f71d0 con 0x7fe15c15ac70 2024-05-08T23:32:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe161bb6700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c1f2c10 con 0x7fe15c071e90 2024-05-08T23:32:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe160bb4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe15c071ab0 0x7fe15c0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe160bb4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe15c071ab0 0x7fe15c0b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34152/0 (socket says 172.21.15.81:34152) 2024-05-08T23:32:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe160bb4700 1 -- 172.21.15.81:0/1869944449 learned_addr learned my addr 172.21.15.81:0/1869944449 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe15bfff700 1 --2- 172.21.15.81:0/1869944449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe15c071e90 0x7fe15c1f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe1613b5700 1 --2- 172.21.15.81:0/1869944449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe15c15ac70 0x7fe15c1f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe160bb4700 1 -- 172.21.15.81:0/1869944449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe15c071e90 msgr2=0x7fe15c1f6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe160bb4700 1 --2- 172.21.15.81:0/1869944449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe15c071e90 0x7fe15c1f6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe160bb4700 1 -- 172.21.15.81:0/1869944449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe15c15ac70 msgr2=0x7fe15c1f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe160bb4700 1 --2- 172.21.15.81:0/1869944449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe15c15ac70 0x7fe15c1f0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe160bb4700 1 -- 172.21.15.81:0/1869944449 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe154009a70 con 0x7fe15c071ab0 2024-05-08T23:32:37.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe15bfff700 1 --2- 172.21.15.81:0/1869944449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe15c071e90 0x7fe15c1f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:37.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe160bb4700 1 --2- 172.21.15.81:0/1869944449 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe15c071ab0 0x7fe15c0b4b10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe15400b370 tx=0x7fe154004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:37.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe1597fa700 1 -- 172.21.15.81:0/1869944449 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154007000 con 0x7fe15c071ab0 2024-05-08T23:32:37.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe1597fa700 1 -- 172.21.15.81:0/1869944449 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe154020070 con 0x7fe15c071ab0 2024-05-08T23:32:37.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe161bb6700 1 -- 172.21.15.81:0/1869944449 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15c1f6930 con 0x7fe15c071ab0 2024-05-08T23:32:37.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe1597fa700 1 -- 172.21.15.81:0/1869944449 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1540276c0 con 0x7fe15c071ab0 2024-05-08T23:32:37.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe161bb6700 1 -- 172.21.15.81:0/1869944449 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15c1f6ef0 con 0x7fe15c071ab0 2024-05-08T23:32:37.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe161bb6700 1 -- 172.21.15.81:0/1869944449 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15c0684b0 con 0x7fe15c071ab0 2024-05-08T23:32:37.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe1597fa700 1 -- 172.21.15.81:0/1869944449 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe154027820 con 0x7fe15c071ab0 2024-05-08T23:32:37.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe1597fa700 1 --2- 172.21.15.81:0/1869944449 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe140066730 0x7fe140068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:37.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe1597fa700 1 -- 172.21.15.81:0/1869944449 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe154022070 con 0x7fe15c071ab0 2024-05-08T23:32:37.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.983+0000 7fe15bfff700 1 --2- 172.21.15.81:0/1869944449 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe140066730 0x7fe140068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:37.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.987+0000 7fe15bfff700 1 --2- 172.21.15.81:0/1869944449 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe140066730 0x7fe140068bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe144007fd0 tx=0x7fe144009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:37.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:37.991+0000 7fe1597fa700 1 -- 172.21.15.81:0/1869944449 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe154053970 con 0x7fe15c071ab0 2024-05-08T23:32:38.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.155+0000 7fe161bb6700 1 -- 172.21.15.81:0/1869944449 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe15c070110 con 0x7fe15c071ab0 2024-05-08T23:32:38.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.155+0000 7fe1597fa700 1 -- 172.21.15.81:0/1869944449 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+12389 (secure 0 0 0) 0x7fe154056f90 con 0x7fe15c071ab0 2024-05-08T23:32:38.159 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:38.159 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":48,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:32:37.450302+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:38.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.159+0000 7fe161bb6700 1 -- 172.21.15.81:0/1869944449 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe140066730 msgr2=0x7fe140068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.159+0000 7fe161bb6700 1 --2- 172.21.15.81:0/1869944449 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe140066730 0x7fe140068bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe144007fd0 tx=0x7fe144009040 comp rx=0 tx=0).stop 2024-05-08T23:32:38.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.159+0000 7fe161bb6700 1 -- 172.21.15.81:0/1869944449 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe15c071ab0 msgr2=0x7fe15c0b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.159+0000 7fe161bb6700 1 --2- 172.21.15.81:0/1869944449 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe15c071ab0 0x7fe15c0b4b10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe15400b370 tx=0x7fe154004af0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.159+0000 7fe161bb6700 1 -- 172.21.15.81:0/1869944449 shutdown_connections 2024-05-08T23:32:38.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.159+0000 7fe161bb6700 1 --2- 172.21.15.81:0/1869944449 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe140066730 0x7fe140068bf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.159+0000 7fe161bb6700 1 --2- 172.21.15.81:0/1869944449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe15c15ac70 0x7fe15c1f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.159+0000 7fe161bb6700 1 --2- 172.21.15.81:0/1869944449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe15c071e90 0x7fe15c1f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.159+0000 7fe161bb6700 1 --2- 172.21.15.81:0/1869944449 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe15c071ab0 0x7fe15c0b4b10 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.159+0000 7fe161bb6700 1 -- 172.21.15.81:0/1869944449 >> 172.21.15.81:0/1869944449 conn(0x7fe15c06c330 msgr2=0x7fe15c06d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:38.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.159+0000 7fe161bb6700 1 -- 172.21.15.81:0/1869944449 shutdown_connections 2024-05-08T23:32:38.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.159+0000 7fe161bb6700 1 -- 172.21.15.81:0/1869944449 wait complete. 2024-05-08T23:32:38.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:32:38.173 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:38.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e50d0b700 1 Processor -- start 2024-05-08T23:32:38.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e50d0b700 1 -- start start 2024-05-08T23:32:38.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e50d0b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e4c15ae10 0x7f8e4c0ac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e50d0b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8e4c15ead0 0x7f8e4c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e50d0b700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4c06c7d0 con 0x7f8e4c0ce4f0 2024-05-08T23:32:38.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e50d0b700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4c0ce9a0 con 0x7f8e4c15ead0 2024-05-08T23:32:38.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e50d0b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c06c4d0 con 0x7f8e4c15ae10 2024-05-08T23:32:38.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e4b7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e4c15ae10 0x7f8e4c0ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e4affd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8e4c15ead0 0x7f8e4c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e4bfff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f8e4c0ce4f0 0x7f8e4c0b2dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:46912/0 (socket says 172.21.15.81:46912) 2024-05-08T23:32:38.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e4bfff700 1 -- 172.21.15.81:0/1005462709 learned_addr learned my addr 172.21.15.81:0/1005462709 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:38.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e4b7fe700 1 -- 172.21.15.81:0/1005462709 >> v2:172.21.15.117:3300/0 conn(0x7f8e4c15ead0 msgr2=0x7f8e4c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:38.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e4b7fe700 1 --2- 172.21.15.81:0/1005462709 >> v2:172.21.15.117:3300/0 conn(0x7f8e4c15ead0 0x7f8e4c0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e4b7fe700 1 -- 172.21.15.81:0/1005462709 >> v1:172.21.15.117:6789/0 conn(0x7f8e4c0ce4f0 legacy=0x7f8e4c0b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e4b7fe700 1 -- 172.21.15.81:0/1005462709 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c0acd90 con 0x7f8e4c15ae10 2024-05-08T23:32:38.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.295+0000 7f8e4affd700 1 --2- 172.21.15.81:0/1005462709 >> v2:172.21.15.117:3300/0 conn(0x7f8e4c15ead0 0x7f8e4c0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:38.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.299+0000 7f8e4b7fe700 1 --2- 172.21.15.81:0/1005462709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e4c15ae10 0x7f8e4c0ac420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009850 tx=0x7f8e34009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dcace3aa4299755 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:38.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.299+0000 7f8e4a7fc700 1 -- 172.21.15.81:0/1005462709 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34011830 con 0x7f8e4c15ae10 2024-05-08T23:32:38.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.299+0000 7f8e4a7fc700 1 -- 172.21.15.81:0/1005462709 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e34011990 con 0x7f8e4c15ae10 2024-05-08T23:32:38.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.299+0000 7f8e4a7fc700 1 -- 172.21.15.81:0/1005462709 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34011d40 con 0x7f8e4c15ae10 2024-05-08T23:32:38.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.299+0000 7f8e50d0b700 1 -- 172.21.15.81:0/1005462709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e4c15ae10 msgr2=0x7f8e4c0ac420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.299+0000 7f8e50d0b700 1 --2- 172.21.15.81:0/1005462709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e4c15ae10 0x7f8e4c0ac420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009850 tx=0x7f8e34009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:38.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.299+0000 7f8e50d0b700 1 -- 172.21.15.81:0/1005462709 shutdown_connections 2024-05-08T23:32:38.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.299+0000 7f8e50d0b700 1 --2- 172.21.15.81:0/1005462709 >> v2:172.21.15.117:3300/0 conn(0x7f8e4c15ead0 0x7f8e4c0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.299+0000 7f8e50d0b700 1 --2- 172.21.15.81:0/1005462709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e4c15ae10 0x7f8e4c0ac420 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.299+0000 7f8e50d0b700 1 -- 172.21.15.81:0/1005462709 >> 172.21.15.81:0/1005462709 conn(0x7f8e4c0b0de0 msgr2=0x7f8e4c0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:38.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.299+0000 7f8e50d0b700 1 -- 172.21.15.81:0/1005462709 shutdown_connections 2024-05-08T23:32:38.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.299+0000 7f8e50d0b700 1 -- 172.21.15.81:0/1005462709 wait complete. 2024-05-08T23:32:38.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.299+0000 7f8e50d0b700 1 Processor -- start 2024-05-08T23:32:38.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.299+0000 7f8e50d0b700 1 -- start start 2024-05-08T23:32:38.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e50d0b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e4c0ce4f0 0x7f8e4c1f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e50d0b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e4c15ae10 0x7f8e4c14d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e50d0b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c15ead0 0x7f8e4c14d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e50d0b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c1f45e0 con 0x7f8e4c15ead0 2024-05-08T23:32:38.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e50d0b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c1e4db0 con 0x7f8e4c0ce4f0 2024-05-08T23:32:38.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e50d0b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c1e5090 con 0x7f8e4c15ae10 2024-05-08T23:32:38.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e4bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c15ead0 0x7f8e4c14d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e4bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c15ead0 0x7f8e4c14d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34156/0 (socket says 172.21.15.81:34156) 2024-05-08T23:32:38.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e4bfff700 1 -- 172.21.15.81:0/3030119461 learned_addr learned my addr 172.21.15.81:0/3030119461 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:38.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e4b7fe700 1 --2- 172.21.15.81:0/3030119461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e4c0ce4f0 0x7f8e4c1f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e4affd700 1 --2- 172.21.15.81:0/3030119461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e4c15ae10 0x7f8e4c14d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e4bfff700 1 -- 172.21.15.81:0/3030119461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e4c15ae10 msgr2=0x7f8e4c14d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e4bfff700 1 --2- 172.21.15.81:0/3030119461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e4c15ae10 0x7f8e4c14d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e4bfff700 1 -- 172.21.15.81:0/3030119461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e4c0ce4f0 msgr2=0x7f8e4c1f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e4bfff700 1 --2- 172.21.15.81:0/3030119461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e4c0ce4f0 0x7f8e4c1f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e4bfff700 1 -- 172.21.15.81:0/3030119461 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c0094b0 con 0x7f8e4c15ead0 2024-05-08T23:32:38.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e4bfff700 1 --2- 172.21.15.81:0/3030119461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c15ead0 0x7f8e4c14d600 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400cbc0 tx=0x7f8e4400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:38.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e43fff700 1 -- 172.21.15.81:0/3030119461 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44007480 con 0x7f8e4c15ead0 2024-05-08T23:32:38.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e43fff700 1 -- 172.21.15.81:0/3030119461 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e440075e0 con 0x7f8e4c15ead0 2024-05-08T23:32:38.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e50d0b700 1 -- 172.21.15.81:0/3030119461 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e340094b0 con 0x7f8e4c15ead0 2024-05-08T23:32:38.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e43fff700 1 -- 172.21.15.81:0/3030119461 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44007950 con 0x7f8e4c15ead0 2024-05-08T23:32:38.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.303+0000 7f8e50d0b700 1 -- 172.21.15.81:0/3030119461 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c14ff70 con 0x7f8e4c15ead0 2024-05-08T23:32:38.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.307+0000 7f8e43fff700 1 -- 172.21.15.81:0/3030119461 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8e44012400 con 0x7f8e4c15ead0 2024-05-08T23:32:38.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.307+0000 7f8e50d0b700 1 -- 172.21.15.81:0/3030119461 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4c1e5200 con 0x7f8e4c15ead0 2024-05-08T23:32:38.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.307+0000 7f8e43fff700 1 --2- 172.21.15.81:0/3030119461 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8e2c066730 0x7f8e2c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.307+0000 7f8e43fff700 1 -- 172.21.15.81:0/3030119461 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8e44083670 con 0x7f8e4c15ead0 2024-05-08T23:32:38.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.307+0000 7f8e4b7fe700 1 --2- 172.21.15.81:0/3030119461 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8e2c066730 0x7f8e2c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.307+0000 7f8e4b7fe700 1 --2- 172.21.15.81:0/3030119461 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8e2c066730 0x7f8e2c068bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8e340090b0 tx=0x7f8e34008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:38.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.315+0000 7f8e43fff700 1 -- 172.21.15.81:0/3030119461 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8e44050d60 con 0x7f8e4c15ead0 2024-05-08T23:32:38.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.479+0000 7f8e50d0b700 1 -- 172.21.15.81:0/3030119461 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e4c0684b0 con 0x7f8e4c15ead0 2024-05-08T23:32:38.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.479+0000 7f8e43fff700 1 -- 172.21.15.81:0/3030119461 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e44017020 con 0x7f8e4c15ead0 2024-05-08T23:32:38.482 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:32:38.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.483+0000 7f8e50d0b700 1 -- 172.21.15.81:0/3030119461 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8e2c066730 msgr2=0x7f8e2c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.483+0000 7f8e50d0b700 1 --2- 172.21.15.81:0/3030119461 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8e2c066730 0x7f8e2c068bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8e340090b0 tx=0x7f8e34008030 comp rx=0 tx=0).stop 2024-05-08T23:32:38.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.483+0000 7f8e50d0b700 1 -- 172.21.15.81:0/3030119461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c15ead0 msgr2=0x7f8e4c14d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.483+0000 7f8e50d0b700 1 --2- 172.21.15.81:0/3030119461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c15ead0 0x7f8e4c14d600 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400cbc0 tx=0x7f8e4400cbf0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.483+0000 7f8e50d0b700 1 -- 172.21.15.81:0/3030119461 shutdown_connections 2024-05-08T23:32:38.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.483+0000 7f8e50d0b700 1 --2- 172.21.15.81:0/3030119461 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8e2c066730 0x7f8e2c068bf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.483+0000 7f8e50d0b700 1 --2- 172.21.15.81:0/3030119461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c15ead0 0x7f8e4c14d600 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.483+0000 7f8e50d0b700 1 --2- 172.21.15.81:0/3030119461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e4c15ae10 0x7f8e4c14d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.483+0000 7f8e50d0b700 1 --2- 172.21.15.81:0/3030119461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e4c0ce4f0 0x7f8e4c1f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.483+0000 7f8e50d0b700 1 -- 172.21.15.81:0/3030119461 >> 172.21.15.81:0/3030119461 conn(0x7f8e4c0b0de0 msgr2=0x7f8e4c0ae830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:38.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.483+0000 7f8e50d0b700 1 -- 172.21.15.81:0/3030119461 shutdown_connections 2024-05-08T23:32:38.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.483+0000 7f8e50d0b700 1 -- 172.21.15.81:0/3030119461 wait complete. 2024-05-08T23:32:38.497 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T23:32:38.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.603+0000 7f2de2984700 1 Processor -- start 2024-05-08T23:32:38.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.603+0000 7f2de2984700 1 -- start start 2024-05-08T23:32:38.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.603+0000 7f2de2984700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2ddc1624b0 0x7f2ddc070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.603+0000 7f2de2984700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ddc158960 con 0x7f2ddc071e70 2024-05-08T23:32:38.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.603+0000 7f2de2984700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ddc0ae990 con 0x7f2ddc0b3c20 2024-05-08T23:32:38.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.603+0000 7f2de2984700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ddc0b1f60 con 0x7f2ddc1624b0 2024-05-08T23:32:38.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.603+0000 7f2de1181700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2ddc071e70 0x7f2ddc070110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35274/0 (socket says 172.21.15.81:35274) 2024-05-08T23:32:38.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.603+0000 7f2de1181700 1 -- 172.21.15.81:0/2844441213 learned_addr learned my addr 172.21.15.81:0/2844441213 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:38.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.603+0000 7f2de2183700 1 --2- 172.21.15.81:0/2844441213 >> v2:172.21.15.81:3300/0 conn(0x7f2ddc1624b0 0x7f2ddc070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.603+0000 7f2de0980700 1 -- 172.21.15.81:0/2844441213 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3829290962 0 0) 0x7f2ddc158960 con 0x7f2ddc071e70 2024-05-08T23:32:38.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 -- 172.21.15.81:0/2844441213 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dc80035d0 con 0x7f2ddc071e70 2024-05-08T23:32:38.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 -- 172.21.15.81:0/2844441213 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3139136803 0 0) 0x7f2dc80035d0 con 0x7f2ddc071e70 2024-05-08T23:32:38.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 -- 172.21.15.81:0/2844441213 >> v2:172.21.15.81:3300/0 conn(0x7f2ddc1624b0 msgr2=0x7f2ddc070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:38.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 --2- 172.21.15.81:0/2844441213 >> v2:172.21.15.81:3300/0 conn(0x7f2ddc1624b0 0x7f2ddc070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 -- 172.21.15.81:0/2844441213 >> v1:172.21.15.117:6789/0 conn(0x7f2ddc0b3c20 legacy=0x7f2ddc0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 -- 172.21.15.81:0/2844441213 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc0b2170 con 0x7f2ddc071e70 2024-05-08T23:32:38.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 -- 172.21.15.81:0/2844441213 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f2dd8001670 con 0x7f2ddc071e70 2024-05-08T23:32:38.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 -- 172.21.15.81:0/2844441213 >> v1:172.21.15.81:6789/0 conn(0x7f2ddc071e70 legacy=0x7f2ddc070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 --2- 172.21.15.81:0/2844441213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2dc8003a20 0x7f2dc8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 --2- 172.21.15.81:0/2844441213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2dc8004870 0x7f2dc8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 --2- 172.21.15.81:0/2844441213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dc8005230 0x7f2dc800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 -- 172.21.15.81:0/2844441213 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc0b1f60 con 0x7f2dc8004870 2024-05-08T23:32:38.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 -- 172.21.15.81:0/2844441213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc800c1d0 con 0x7f2dc8005230 2024-05-08T23:32:38.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 -- 172.21.15.81:0/2844441213 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc800c390 con 0x7f2dc8003a20 2024-05-08T23:32:38.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de1181700 1 --2- 172.21.15.81:0/2844441213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2dc8004870 0x7f2dc8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de2183700 1 --2- 172.21.15.81:0/2844441213 >> v2:172.21.15.81:3300/0 conn(0x7f2ddc1624b0 0x7f2ddc070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:38.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de1982700 1 --2- 172.21.15.81:0/2844441213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2dc8003a20 0x7f2dc8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de2183700 1 --2- 172.21.15.81:0/2844441213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dc8005230 0x7f2dc800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de1181700 1 -- 172.21.15.81:0/2844441213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2dc8003a20 msgr2=0x7f2dc8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:38.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de1181700 1 --2- 172.21.15.81:0/2844441213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2dc8003a20 0x7f2dc8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de1181700 1 -- 172.21.15.81:0/2844441213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dc8005230 msgr2=0x7f2dc800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:38.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de1181700 1 --2- 172.21.15.81:0/2844441213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dc8005230 0x7f2dc800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de1181700 1 -- 172.21.15.81:0/2844441213 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc80035d0 con 0x7f2dc8004870 2024-05-08T23:32:38.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de2183700 1 --2- 172.21.15.81:0/2844441213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dc8005230 0x7f2dc800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:38.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de2183700 1 -- 172.21.15.81:0/2844441213 reap_dead start 2024-05-08T23:32:38.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de1181700 1 --2- 172.21.15.81:0/2844441213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2dc8004870 0x7f2dc8004cf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800c320 tx=0x7f2dc80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adf24ccf457f4a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:38.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 -- 172.21.15.81:0/2844441213 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8007c30 con 0x7f2dc8004870 2024-05-08T23:32:38.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 -- 172.21.15.81:0/2844441213 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd800ce80 con 0x7f2dc8004870 2024-05-08T23:32:38.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de0980700 1 -- 172.21.15.81:0/2844441213 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8005bc0 con 0x7f2dc8004870 2024-05-08T23:32:38.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de2984700 1 -- 172.21.15.81:0/2844441213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2dc8004870 msgr2=0x7f2dc8004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de2984700 1 --2- 172.21.15.81:0/2844441213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2dc8004870 0x7f2dc8004cf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800c320 tx=0x7f2dc80058a0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de2984700 1 -- 172.21.15.81:0/2844441213 shutdown_connections 2024-05-08T23:32:38.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de2984700 1 --2- 172.21.15.81:0/2844441213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2dc8004870 0x7f2dc8004cf0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de2984700 1 -- 172.21.15.81:0/2844441213 >> 172.21.15.81:0/2844441213 conn(0x7f2ddc06c330 msgr2=0x7f2ddc06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:38.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de2984700 1 -- 172.21.15.81:0/2844441213 shutdown_connections 2024-05-08T23:32:38.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.607+0000 7f2de2984700 1 -- 172.21.15.81:0/2844441213 wait complete. 2024-05-08T23:32:38.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.611+0000 7f2de2984700 1 Processor -- start 2024-05-08T23:32:38.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.611+0000 7f2de2984700 1 -- start start 2024-05-08T23:32:38.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de2984700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dc8004870 0x7f2ddc142e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de2984700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ddc162940 0x7f2ddc14ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de2984700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ddc14f2d0 0x7f2ddc14f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de2984700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc143370 con 0x7f2ddc162940 2024-05-08T23:32:38.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de2984700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc14d880 con 0x7f2dc8004870 2024-05-08T23:32:38.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de2984700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc165830 con 0x7f2ddc14f2d0 2024-05-08T23:32:38.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de1181700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ddc162940 0x7f2ddc14ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de1181700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ddc162940 0x7f2ddc14ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34172/0 (socket says 172.21.15.81:34172) 2024-05-08T23:32:38.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de1181700 1 -- 172.21.15.81:0/2033409688 learned_addr learned my addr 172.21.15.81:0/2033409688 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:38.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de1982700 1 --2- 172.21.15.81:0/2033409688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dc8004870 0x7f2ddc142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de2183700 1 --2- 172.21.15.81:0/2033409688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ddc14f2d0 0x7f2ddc14f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de1181700 1 -- 172.21.15.81:0/2033409688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ddc14f2d0 msgr2=0x7f2ddc14f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de1181700 1 --2- 172.21.15.81:0/2033409688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ddc14f2d0 0x7f2ddc14f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de1181700 1 -- 172.21.15.81:0/2033409688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dc8004870 msgr2=0x7f2ddc142e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de1181700 1 --2- 172.21.15.81:0/2033409688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dc8004870 0x7f2ddc142e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de1181700 1 -- 172.21.15.81:0/2033409688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc165b10 con 0x7f2ddc162940 2024-05-08T23:32:38.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de1181700 1 --2- 172.21.15.81:0/2033409688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ddc162940 0x7f2ddc14ed90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800c1a0 tx=0x7f2dd8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:38.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2dd27fc700 1 -- 172.21.15.81:0/2033409688 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8004d60 con 0x7f2ddc162940 2024-05-08T23:32:38.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2dd27fc700 1 -- 172.21.15.81:0/2033409688 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd80070d0 con 0x7f2ddc162940 2024-05-08T23:32:38.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de2984700 1 -- 172.21.15.81:0/2033409688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ddc143370 con 0x7f2ddc162940 2024-05-08T23:32:38.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2dd27fc700 1 -- 172.21.15.81:0/2033409688 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8007480 con 0x7f2ddc162940 2024-05-08T23:32:38.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de2984700 1 -- 172.21.15.81:0/2033409688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ddc151bb0 con 0x7f2ddc162940 2024-05-08T23:32:38.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.615+0000 7f2de2984700 1 -- 172.21.15.81:0/2033409688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ddc151d70 con 0x7f2ddc162940 2024-05-08T23:32:38.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.619+0000 7f2dd27fc700 1 -- 172.21.15.81:0/2033409688 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2dd80075e0 con 0x7f2ddc162940 2024-05-08T23:32:38.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.619+0000 7f2dd27fc700 1 --2- 172.21.15.81:0/2033409688 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2dc0066730 0x7f2dc0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.619+0000 7f2dd27fc700 1 -- 172.21.15.81:0/2033409688 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2dd8084d00 con 0x7f2ddc162940 2024-05-08T23:32:38.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.619+0000 7f2de1982700 1 --2- 172.21.15.81:0/2033409688 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2dc0066730 0x7f2dc0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.619+0000 7f2de1982700 1 --2- 172.21.15.81:0/2033409688 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2dc0066730 0x7f2dc0068bf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc000c00 tx=0x7f2dcc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:38.625 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.623+0000 7f2dd27fc700 1 -- 172.21.15.81:0/2033409688 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2dd8052450 con 0x7f2ddc162940 2024-05-08T23:32:38.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.803+0000 7f2de2984700 1 -- 172.21.15.81:0/2033409688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2ddc0684b0 con 0x7f2ddc162940 2024-05-08T23:32:38.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.803+0000 7f2dd27fc700 1 -- 172.21.15.81:0/2033409688 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2dd805ae60 con 0x7f2ddc162940 2024-05-08T23:32:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:38.806 INFO:teuthology.orchestra.run.smithi081.stdout:[ 2024-05-08T23:32:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-05-08T23:32:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: "poolnum": 1, 2024-05-08T23:32:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: "poolname": ".mgr" 2024-05-08T23:32:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:38.806 INFO:teuthology.orchestra.run.smithi081.stdout:] 2024-05-08T23:32:38.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.807+0000 7f2de2984700 1 -- 172.21.15.81:0/2033409688 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2dc0066730 msgr2=0x7f2dc0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.807+0000 7f2de2984700 1 --2- 172.21.15.81:0/2033409688 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2dc0066730 0x7f2dc0068bf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc000c00 tx=0x7f2dcc009040 comp rx=0 tx=0).stop 2024-05-08T23:32:38.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.807+0000 7f2de2984700 1 -- 172.21.15.81:0/2033409688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ddc162940 msgr2=0x7f2ddc14ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.807+0000 7f2de2984700 1 --2- 172.21.15.81:0/2033409688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ddc162940 0x7f2ddc14ed90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800c1a0 tx=0x7f2dd8001a00 comp rx=0 tx=0).stop 2024-05-08T23:32:38.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.807+0000 7f2de2984700 1 -- 172.21.15.81:0/2033409688 shutdown_connections 2024-05-08T23:32:38.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.807+0000 7f2de2984700 1 --2- 172.21.15.81:0/2033409688 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2dc0066730 0x7f2dc0068bf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.807+0000 7f2de2984700 1 --2- 172.21.15.81:0/2033409688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ddc14f2d0 0x7f2ddc14f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.807+0000 7f2de2984700 1 --2- 172.21.15.81:0/2033409688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ddc162940 0x7f2ddc14ed90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.807+0000 7f2de2984700 1 --2- 172.21.15.81:0/2033409688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2dc8004870 0x7f2ddc142e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.807+0000 7f2de2984700 1 -- 172.21.15.81:0/2033409688 >> 172.21.15.81:0/2033409688 conn(0x7f2ddc06c330 msgr2=0x7f2ddc0cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:38.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.807+0000 7f2de2984700 1 -- 172.21.15.81:0/2033409688 shutdown_connections 2024-05-08T23:32:38.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.807+0000 7f2de2984700 1 -- 172.21.15.81:0/2033409688 wait complete. 2024-05-08T23:32:38.821 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T23:32:38.821 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T23:32:38.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 Processor -- start 2024-05-08T23:32:38.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 -- start start 2024-05-08T23:32:38.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd844161600 0x7fd8440b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd844071850 0x7fd84406c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd84406c870 0x7fd844165270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd84406cce0 con 0x7fd84406c870 2024-05-08T23:32:38.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8440b2930 con 0x7fd844071850 2024-05-08T23:32:38.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd844071d10 con 0x7fd844161600 2024-05-08T23:32:38.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd8488d5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd84406c870 0x7fd844165270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd8488d5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd84406c870 0x7fd844165270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34184/0 (socket says 172.21.15.81:34184) 2024-05-08T23:32:38.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd8488d5700 1 -- 172.21.15.81:0/744421972 learned_addr learned my addr 172.21.15.81:0/744421972 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:38.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd8437fe700 1 --2- 172.21.15.81:0/744421972 >> v2:172.21.15.117:3300/0 conn(0x7fd844071850 0x7fd84406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd843fff700 1 --2- 172.21.15.81:0/744421972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd844161600 0x7fd8440b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd8488d5700 1 -- 172.21.15.81:0/744421972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd844161600 msgr2=0x7fd8440b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:38.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd8488d5700 1 --2- 172.21.15.81:0/744421972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd844161600 0x7fd8440b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd8488d5700 1 -- 172.21.15.81:0/744421972 >> v2:172.21.15.117:3300/0 conn(0x7fd844071850 msgr2=0x7fd84406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:38.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd8488d5700 1 --2- 172.21.15.81:0/744421972 >> v2:172.21.15.117:3300/0 conn(0x7fd844071850 0x7fd84406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd8488d5700 1 -- 172.21.15.81:0/744421972 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8440b2000 con 0x7fd84406c870 2024-05-08T23:32:38.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd8437fe700 1 --2- 172.21.15.81:0/744421972 >> v2:172.21.15.117:3300/0 conn(0x7fd844071850 0x7fd84406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:38.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd843fff700 1 --2- 172.21.15.81:0/744421972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd844161600 0x7fd8440b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:38.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd8488d5700 1 --2- 172.21.15.81:0/744421972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84406c870 0x7fd844165270 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd844066b60 tx=0x7fd83c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87b4b16ab13e832 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:38.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd842ffd700 1 -- 172.21.15.81:0/744421972 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00e070 con 0x7fd84406c870 2024-05-08T23:32:38.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd842ffd700 1 -- 172.21.15.81:0/744421972 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c00b9c0 con 0x7fd84406c870 2024-05-08T23:32:38.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd842ffd700 1 -- 172.21.15.81:0/744421972 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00bd70 con 0x7fd84406c870 2024-05-08T23:32:38.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 -- 172.21.15.81:0/744421972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84406c870 msgr2=0x7fd844165270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 --2- 172.21.15.81:0/744421972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84406c870 0x7fd844165270 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd844066b60 tx=0x7fd83c00b330 comp rx=0 tx=0).stop 2024-05-08T23:32:38.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 -- 172.21.15.81:0/744421972 shutdown_connections 2024-05-08T23:32:38.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 --2- 172.21.15.81:0/744421972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84406c870 0x7fd844165270 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 --2- 172.21.15.81:0/744421972 >> v2:172.21.15.117:3300/0 conn(0x7fd844071850 0x7fd84406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 --2- 172.21.15.81:0/744421972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd844161600 0x7fd8440b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 -- 172.21.15.81:0/744421972 >> 172.21.15.81:0/744421972 conn(0x7fd84406a360 msgr2=0x7fd84406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:38.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 -- 172.21.15.81:0/744421972 shutdown_connections 2024-05-08T23:32:38.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.935+0000 7fd84a338700 1 -- 172.21.15.81:0/744421972 wait complete. 2024-05-08T23:32:38.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd84a338700 1 Processor -- start 2024-05-08T23:32:38.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd84a338700 1 -- start start 2024-05-08T23:32:38.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd84a338700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84406c870 0x7fd8441f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd84a338700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd844071850 0x7fd8440a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd84a338700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd844161600 0x7fd8440a9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd84a338700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440a8bc0 con 0x7fd84406c870 2024-05-08T23:32:38.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd84a338700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8441f6f20 con 0x7fd844161600 2024-05-08T23:32:38.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd84a338700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440aa4a0 con 0x7fd844071850 2024-05-08T23:32:38.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd843fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84406c870 0x7fd8441f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd843fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84406c870 0x7fd8441f69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34192/0 (socket says 172.21.15.81:34192) 2024-05-08T23:32:38.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd843fff700 1 -- 172.21.15.81:0/909776069 learned_addr learned my addr 172.21.15.81:0/909776069 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:38.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd8488d5700 1 --2- 172.21.15.81:0/909776069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd844161600 0x7fd8440a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd8437fe700 1 --2- 172.21.15.81:0/909776069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd844071850 0x7fd8440a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd843fff700 1 -- 172.21.15.81:0/909776069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd844071850 msgr2=0x7fd8440a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd843fff700 1 --2- 172.21.15.81:0/909776069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd844071850 0x7fd8440a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd843fff700 1 -- 172.21.15.81:0/909776069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd844161600 msgr2=0x7fd8440a9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:38.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd843fff700 1 --2- 172.21.15.81:0/909776069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd844161600 0x7fd8440a9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:38.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd843fff700 1 -- 172.21.15.81:0/909776069 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c009b30 con 0x7fd84406c870 2024-05-08T23:32:38.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd843fff700 1 --2- 172.21.15.81:0/909776069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84406c870 0x7fd8441f69e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c7f0 tx=0x7fd83800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:38.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd840ff9700 1 -- 172.21.15.81:0/909776069 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd838011800 con 0x7fd84406c870 2024-05-08T23:32:38.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd840ff9700 1 -- 172.21.15.81:0/909776069 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd838011960 con 0x7fd84406c870 2024-05-08T23:32:38.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd84a338700 1 -- 172.21.15.81:0/909776069 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd844161b60 con 0x7fd84406c870 2024-05-08T23:32:38.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd840ff9700 1 -- 172.21.15.81:0/909776069 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd838011cf0 con 0x7fd84406c870 2024-05-08T23:32:38.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.939+0000 7fd84a338700 1 -- 172.21.15.81:0/909776069 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8440a8570 con 0x7fd84406c870 2024-05-08T23:32:38.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.943+0000 7fd84a338700 1 -- 172.21.15.81:0/909776069 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd844068630 con 0x7fd84406c870 2024-05-08T23:32:38.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.943+0000 7fd840ff9700 1 -- 172.21.15.81:0/909776069 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd838023480 con 0x7fd84406c870 2024-05-08T23:32:38.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.943+0000 7fd840ff9700 1 --2- 172.21.15.81:0/909776069 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd830066730 0x7fd830068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:38.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.943+0000 7fd8437fe700 1 --2- 172.21.15.81:0/909776069 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd830066730 0x7fd830068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:38.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.943+0000 7fd840ff9700 1 -- 172.21.15.81:0/909776069 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd8380841e0 con 0x7fd84406c870 2024-05-08T23:32:38.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.943+0000 7fd8437fe700 1 --2- 172.21.15.81:0/909776069 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd830066730 0x7fd830068bf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd8441fa550 tx=0x7fd834008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:38.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:38.951+0000 7fd840ff9700 1 -- 172.21.15.81:0/909776069 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd838051890 con 0x7fd84406c870 2024-05-08T23:32:39.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.115+0000 7fd84a338700 1 -- 172.21.15.81:0/909776069 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd84406d080 con 0x7fd84406c870 2024-05-08T23:32:39.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.471+0000 7fd840ff9700 1 -- 172.21.15.81:0/909776069 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v49) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd838054eb0 con 0x7fd84406c870 2024-05-08T23:32:39.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.479+0000 7fd84a338700 1 -- 172.21.15.81:0/909776069 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd830066730 msgr2=0x7fd830068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:39.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.479+0000 7fd84a338700 1 --2- 172.21.15.81:0/909776069 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd830066730 0x7fd830068bf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd8441fa550 tx=0x7fd834008040 comp rx=0 tx=0).stop 2024-05-08T23:32:39.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.479+0000 7fd84a338700 1 -- 172.21.15.81:0/909776069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84406c870 msgr2=0x7fd8441f69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:39.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.479+0000 7fd84a338700 1 --2- 172.21.15.81:0/909776069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84406c870 0x7fd8441f69e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c7f0 tx=0x7fd83800cb00 comp rx=0 tx=0).stop 2024-05-08T23:32:39.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.483+0000 7fd84a338700 1 -- 172.21.15.81:0/909776069 shutdown_connections 2024-05-08T23:32:39.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.483+0000 7fd84a338700 1 --2- 172.21.15.81:0/909776069 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd830066730 0x7fd830068bf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:39.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.483+0000 7fd84a338700 1 --2- 172.21.15.81:0/909776069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd844161600 0x7fd8440a9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:39.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.483+0000 7fd84a338700 1 --2- 172.21.15.81:0/909776069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd844071850 0x7fd8440a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:39.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.483+0000 7fd84a338700 1 --2- 172.21.15.81:0/909776069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84406c870 0x7fd8441f69e0 secure :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c7f0 tx=0x7fd83800cb00 comp rx=0 tx=0).stop 2024-05-08T23:32:39.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.483+0000 7fd84a338700 1 -- 172.21.15.81:0/909776069 >> 172.21.15.81:0/909776069 conn(0x7fd84406a360 msgr2=0x7fd8440b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:39.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.483+0000 7fd84a338700 1 -- 172.21.15.81:0/909776069 shutdown_connections 2024-05-08T23:32:39.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.483+0000 7fd84a338700 1 -- 172.21.15.81:0/909776069 wait complete. 2024-05-08T23:32:39.485 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' created 2024-05-08T23:32:39.497 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T23:32:39.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.615+0000 7f2ebee0b700 1 Processor -- start 2024-05-08T23:32:39.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.615+0000 7f2ebee0b700 1 -- start start 2024-05-08T23:32:39.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebee0b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2eb813d050 0x7f2eb813f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:39.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebee0b700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb815a6d0 con 0x7f2eb80b44e0 2024-05-08T23:32:39.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebee0b700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb80b3940 con 0x7f2eb813feb0 2024-05-08T23:32:39.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebee0b700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb80b3010 con 0x7f2eb813d050 2024-05-08T23:32:39.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebcba7700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2eb813d050 0x7f2eb813f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:39.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebcba7700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2eb813d050 0x7f2eb813f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34204/0 (socket says 172.21.15.81:34204) 2024-05-08T23:32:39.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebcba7700 1 -- 172.21.15.81:0/2370203133 learned_addr learned my addr 172.21.15.81:0/2370203133 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:39.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebd3a8700 1 --1- 172.21.15.81:0/2370203133 >> v1:172.21.15.81:6789/0 conn(0x7f2eb80b44e0 0x7f2eb815dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35286/0 (socket says 172.21.15.81:35286) 2024-05-08T23:32:39.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebcba7700 1 -- 172.21.15.81:0/2370203133 >> v1:172.21.15.117:6789/0 conn(0x7f2eb813feb0 legacy=0x7f2eb80b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:39.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebcba7700 1 -- 172.21.15.81:0/2370203133 >> v1:172.21.15.81:6789/0 conn(0x7f2eb80b44e0 legacy=0x7f2eb815dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:39.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebcba7700 1 -- 172.21.15.81:0/2370203133 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb80b32f0 con 0x7f2eb813d050 2024-05-08T23:32:39.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebcba7700 1 --2- 172.21.15.81:0/2370203133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2eb813d050 0x7f2eb813f970 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800d150 tx=0x7f2ea800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a565e7ac7504f436 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:39.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2eb77fe700 1 -- 172.21.15.81:0/2370203133 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8010070 con 0x7f2eb813d050 2024-05-08T23:32:39.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2eb77fe700 1 -- 172.21.15.81:0/2370203133 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea800ddc0 con 0x7f2eb813d050 2024-05-08T23:32:39.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2eb77fe700 1 -- 172.21.15.81:0/2370203133 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800b310 con 0x7f2eb813d050 2024-05-08T23:32:39.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebee0b700 1 -- 172.21.15.81:0/2370203133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2eb813d050 msgr2=0x7f2eb813f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:39.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebee0b700 1 --2- 172.21.15.81:0/2370203133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2eb813d050 0x7f2eb813f970 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800d150 tx=0x7f2ea800d460 comp rx=0 tx=0).stop 2024-05-08T23:32:39.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebee0b700 1 -- 172.21.15.81:0/2370203133 shutdown_connections 2024-05-08T23:32:39.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebee0b700 1 --2- 172.21.15.81:0/2370203133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2eb813d050 0x7f2eb813f970 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:39.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebee0b700 1 -- 172.21.15.81:0/2370203133 >> 172.21.15.81:0/2370203133 conn(0x7f2eb806b470 msgr2=0x7f2eb806b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:39.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebee0b700 1 -- 172.21.15.81:0/2370203133 shutdown_connections 2024-05-08T23:32:39.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.619+0000 7f2ebee0b700 1 -- 172.21.15.81:0/2370203133 wait complete. 2024-05-08T23:32:39.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebee0b700 1 Processor -- start 2024-05-08T23:32:39.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebee0b700 1 -- start start 2024-05-08T23:32:39.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebee0b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2eb80b44e0 0x7f2eb81ef280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:39.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebee0b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2eb813d050 0x7f2eb8139640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:39.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebee0b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2eb813feb0 0x7f2eb815b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:39.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebee0b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb8149740 con 0x7f2eb813feb0 2024-05-08T23:32:39.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebee0b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb815bc90 con 0x7f2eb813d050 2024-05-08T23:32:39.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebee0b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb815bf70 con 0x7f2eb80b44e0 2024-05-08T23:32:39.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebd3a8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2eb813feb0 0x7f2eb815b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:39.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebd3a8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2eb813feb0 0x7f2eb815b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34220/0 (socket says 172.21.15.81:34220) 2024-05-08T23:32:39.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebd3a8700 1 -- 172.21.15.81:0/3626895744 learned_addr learned my addr 172.21.15.81:0/3626895744 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:39.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2eb7fff700 1 --2- 172.21.15.81:0/3626895744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2eb813d050 0x7f2eb8139640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:39.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebcba7700 1 --2- 172.21.15.81:0/3626895744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2eb80b44e0 0x7f2eb81ef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:39.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebd3a8700 1 -- 172.21.15.81:0/3626895744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2eb80b44e0 msgr2=0x7f2eb81ef280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:39.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebd3a8700 1 --2- 172.21.15.81:0/3626895744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2eb80b44e0 0x7f2eb81ef280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:39.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebd3a8700 1 -- 172.21.15.81:0/3626895744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2eb813d050 msgr2=0x7f2eb8139640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:39.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebd3a8700 1 --2- 172.21.15.81:0/3626895744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2eb813d050 0x7f2eb8139640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:39.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebd3a8700 1 -- 172.21.15.81:0/3626895744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea800bd40 con 0x7f2eb813feb0 2024-05-08T23:32:39.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2eb7fff700 1 --2- 172.21.15.81:0/3626895744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2eb813d050 0x7f2eb8139640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:39.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebd3a8700 1 --2- 172.21.15.81:0/3626895744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2eb813feb0 0x7f2eb815b120 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000c500 tx=0x7f2eb000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:39.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2eb57fa700 1 -- 172.21.15.81:0/3626895744 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0005cd0 con 0x7f2eb813feb0 2024-05-08T23:32:39.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebee0b700 1 -- 172.21.15.81:0/3626895744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb815c250 con 0x7f2eb813feb0 2024-05-08T23:32:39.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2eb57fa700 1 -- 172.21.15.81:0/3626895744 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb0005e30 con 0x7f2eb813feb0 2024-05-08T23:32:39.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2eb57fa700 1 -- 172.21.15.81:0/3626895744 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb00072f0 con 0x7f2eb813feb0 2024-05-08T23:32:39.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.623+0000 7f2ebee0b700 1 -- 172.21.15.81:0/3626895744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb815c810 con 0x7f2eb813feb0 2024-05-08T23:32:39.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.627+0000 7f2ebee0b700 1 -- 172.21.15.81:0/3626895744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb815ca30 con 0x7f2eb813feb0 2024-05-08T23:32:39.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.627+0000 7f2eb57fa700 1 -- 172.21.15.81:0/3626895744 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2eb0007450 con 0x7f2eb813feb0 2024-05-08T23:32:39.630 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.627+0000 7f2eb57fa700 1 --2- 172.21.15.81:0/3626895744 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2ea4066730 0x7f2ea4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:39.630 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.627+0000 7f2eb57fa700 1 -- 172.21.15.81:0/3626895744 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2eb0083430 con 0x7f2eb813feb0 2024-05-08T23:32:39.630 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.627+0000 7f2ebcba7700 1 --2- 172.21.15.81:0/3626895744 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2ea4066730 0x7f2ea4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:39.631 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.627+0000 7f2ebcba7700 1 --2- 172.21.15.81:0/3626895744 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2ea4066730 0x7f2ea4068bf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800d120 tx=0x7f2ea800b000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:39.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.639+0000 7f2eb57fa700 1 -- 172.21.15.81:0/3626895744 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2eb00507d0 con 0x7f2eb813feb0 2024-05-08T23:32:39.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:39.811+0000 7f2ebee0b700 1 -- 172.21.15.81:0/3626895744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2eb815a6d0 con 0x7f2eb813feb0 2024-05-08T23:32:40.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.479+0000 7f2eb57fa700 1 -- 172.21.15.81:0/3626895744 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v50) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2eb815a6d0 con 0x7f2eb813feb0 2024-05-08T23:32:40.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.491+0000 7f2ebee0b700 1 -- 172.21.15.81:0/3626895744 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2ea4066730 msgr2=0x7f2ea4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:40.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.491+0000 7f2ebee0b700 1 --2- 172.21.15.81:0/3626895744 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2ea4066730 0x7f2ea4068bf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800d120 tx=0x7f2ea800b000 comp rx=0 tx=0).stop 2024-05-08T23:32:40.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.491+0000 7f2ebee0b700 1 -- 172.21.15.81:0/3626895744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2eb813feb0 msgr2=0x7f2eb815b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:40.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.495+0000 7f2ebee0b700 1 --2- 172.21.15.81:0/3626895744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2eb813feb0 0x7f2eb815b120 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000c500 tx=0x7f2eb000c8c0 comp rx=0 tx=0).stop 2024-05-08T23:32:40.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.499+0000 7f2ebee0b700 1 -- 172.21.15.81:0/3626895744 shutdown_connections 2024-05-08T23:32:40.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.499+0000 7f2ebee0b700 1 --2- 172.21.15.81:0/3626895744 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2ea4066730 0x7f2ea4068bf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:40.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.499+0000 7f2ebee0b700 1 --2- 172.21.15.81:0/3626895744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2eb813feb0 0x7f2eb815b120 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:40.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.499+0000 7f2ebee0b700 1 --2- 172.21.15.81:0/3626895744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2eb813d050 0x7f2eb8139640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:40.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.499+0000 7f2ebee0b700 1 --2- 172.21.15.81:0/3626895744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2eb80b44e0 0x7f2eb81ef280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:40.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.499+0000 7f2ebee0b700 1 -- 172.21.15.81:0/3626895744 >> 172.21.15.81:0/3626895744 conn(0x7f2eb806b470 msgr2=0x7f2eb806dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:40.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.499+0000 7f2ebee0b700 1 -- 172.21.15.81:0/3626895744 shutdown_connections 2024-05-08T23:32:40.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.499+0000 7f2ebee0b700 1 -- 172.21.15.81:0/3626895744 wait complete. 2024-05-08T23:32:40.504 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' created 2024-05-08T23:32:40.516 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T23:32:40.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d71d0b700 1 Processor -- start 2024-05-08T23:32:40.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d71d0b700 1 -- start start 2024-05-08T23:32:40.631 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d71d0b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d6c0b4c40 0x7f2d6c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:40.631 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d71d0b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2d6c15d780 0x7f2d6c15dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:40.631 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d71d0b700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c0ce3c0 con 0x7f2d6c0ae670 2024-05-08T23:32:40.631 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d71d0b700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c15e750 con 0x7f2d6c15d780 2024-05-08T23:32:40.631 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d71d0b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c15e470 con 0x7f2d6c0b4c40 2024-05-08T23:32:40.631 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d6bfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2d6c15d780 0x7f2d6c15dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:40.631 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d6bfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2d6c15d780 0x7f2d6c15dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34234/0 (socket says 172.21.15.81:34234) 2024-05-08T23:32:40.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d6bfff700 1 -- 172.21.15.81:0/4068705881 learned_addr learned my addr 172.21.15.81:0/4068705881 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:40.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d6b7fe700 1 --2- 172.21.15.81:0/4068705881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d6c0b4c40 0x7f2d6c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:40.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d6bfff700 1 -- 172.21.15.81:0/4068705881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d6c0b4c40 msgr2=0x7f2d6c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:40.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d6bfff700 1 --2- 172.21.15.81:0/4068705881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d6c0b4c40 0x7f2d6c0cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:40.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d6bfff700 1 -- 172.21.15.81:0/4068705881 >> v1:172.21.15.117:6789/0 conn(0x7f2d6c0ae670 legacy=0x7f2d6c0aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:40.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d6bfff700 1 -- 172.21.15.81:0/4068705881 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0cd3a0 con 0x7f2d6c15d780 2024-05-08T23:32:40.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d6bfff700 1 --2- 172.21.15.81:0/4068705881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d6c15d780 0x7f2d6c15dbd0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009e50 tx=0x7f2d6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e354996d80fb1d79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:40.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.627+0000 7f2d6a7fc700 1 -- 172.21.15.81:0/4068705881 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400e070 con 0x7f2d6c15d780 2024-05-08T23:32:40.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.631+0000 7f2d6a7fc700 1 -- 172.21.15.81:0/4068705881 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6400b9c0 con 0x7f2d6c15d780 2024-05-08T23:32:40.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.631+0000 7f2d6a7fc700 1 -- 172.21.15.81:0/4068705881 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400bd70 con 0x7f2d6c15d780 2024-05-08T23:32:40.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.631+0000 7f2d71d0b700 1 -- 172.21.15.81:0/4068705881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d6c15d780 msgr2=0x7f2d6c15dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:40.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.631+0000 7f2d71d0b700 1 --2- 172.21.15.81:0/4068705881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d6c15d780 0x7f2d6c15dbd0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009e50 tx=0x7f2d6400b330 comp rx=0 tx=0).stop 2024-05-08T23:32:40.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.631+0000 7f2d71d0b700 1 -- 172.21.15.81:0/4068705881 shutdown_connections 2024-05-08T23:32:40.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.631+0000 7f2d71d0b700 1 --2- 172.21.15.81:0/4068705881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d6c15d780 0x7f2d6c15dbd0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:40.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.631+0000 7f2d71d0b700 1 --2- 172.21.15.81:0/4068705881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d6c0b4c40 0x7f2d6c0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:40.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.631+0000 7f2d71d0b700 1 -- 172.21.15.81:0/4068705881 >> 172.21.15.81:0/4068705881 conn(0x7f2d6c0cf630 msgr2=0x7f2d6c0cfa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:40.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.631+0000 7f2d71d0b700 1 -- 172.21.15.81:0/4068705881 shutdown_connections 2024-05-08T23:32:40.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.631+0000 7f2d71d0b700 1 -- 172.21.15.81:0/4068705881 wait complete. 2024-05-08T23:32:40.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.631+0000 7f2d71d0b700 1 Processor -- start 2024-05-08T23:32:40.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.631+0000 7f2d71d0b700 1 -- start start 2024-05-08T23:32:40.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d71d0b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d6c0ae670 0x7f2d6c15a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:40.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d71d0b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d6c0b4c40 0x7f2d6c15fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:40.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d71d0b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d6c15d780 0x7f2d6c0b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:40.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d71d0b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0b24f0 con 0x7f2d6c0ae670 2024-05-08T23:32:40.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d71d0b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c157ef0 con 0x7f2d6c15d780 2024-05-08T23:32:40.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d71d0b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c1581d0 con 0x7f2d6c0b4c40 2024-05-08T23:32:40.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d6b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d6c0ae670 0x7f2d6c15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:40.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d6b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d6c0ae670 0x7f2d6c15a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34242/0 (socket says 172.21.15.81:34242) 2024-05-08T23:32:40.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d6b7fe700 1 -- 172.21.15.81:0/1805162445 learned_addr learned my addr 172.21.15.81:0/1805162445 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:40.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d6bfff700 1 --2- 172.21.15.81:0/1805162445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d6c15d780 0x7f2d6c0b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:40.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d6affd700 1 --2- 172.21.15.81:0/1805162445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d6c0b4c40 0x7f2d6c15fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:40.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d6b7fe700 1 -- 172.21.15.81:0/1805162445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d6c0b4c40 msgr2=0x7f2d6c15fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:40.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d6b7fe700 1 --2- 172.21.15.81:0/1805162445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d6c0b4c40 0x7f2d6c15fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:40.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d6b7fe700 1 -- 172.21.15.81:0/1805162445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d6c15d780 msgr2=0x7f2d6c0b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:40.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d6b7fe700 1 --2- 172.21.15.81:0/1805162445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d6c15d780 0x7f2d6c0b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:40.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d6b7fe700 1 -- 172.21.15.81:0/1805162445 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d64009b30 con 0x7f2d6c0ae670 2024-05-08T23:32:40.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d6b7fe700 1 --2- 172.21.15.81:0/1805162445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d6c0ae670 0x7f2d6c15a540 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000e760 tx=0x7f2d6000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:40.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d53fff700 1 -- 172.21.15.81:0/1805162445 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60013610 con 0x7f2d6c0ae670 2024-05-08T23:32:40.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d53fff700 1 -- 172.21.15.81:0/1805162445 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d60013770 con 0x7f2d6c0ae670 2024-05-08T23:32:40.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d71d0b700 1 -- 172.21.15.81:0/1805162445 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c0b2790 con 0x7f2d6c0ae670 2024-05-08T23:32:40.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d53fff700 1 -- 172.21.15.81:0/1805162445 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60013ae0 con 0x7f2d6c0ae670 2024-05-08T23:32:40.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d71d0b700 1 -- 172.21.15.81:0/1805162445 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c160090 con 0x7f2d6c0ae670 2024-05-08T23:32:40.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.635+0000 7f2d71d0b700 1 -- 172.21.15.81:0/1805162445 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c0672a0 con 0x7f2d6c0ae670 2024-05-08T23:32:40.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.639+0000 7f2d53fff700 1 -- 172.21.15.81:0/1805162445 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2d60013c40 con 0x7f2d6c0ae670 2024-05-08T23:32:40.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.639+0000 7f2d53fff700 1 --2- 172.21.15.81:0/1805162445 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2d58066730 0x7f2d58068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:40.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.639+0000 7f2d6affd700 1 --2- 172.21.15.81:0/1805162445 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2d58066730 0x7f2d58068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:40.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.639+0000 7f2d53fff700 1 -- 172.21.15.81:0/1805162445 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2d600841c0 con 0x7f2d6c0ae670 2024-05-08T23:32:40.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.639+0000 7f2d6affd700 1 --2- 172.21.15.81:0/1805162445 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2d58066730 0x7f2d58068bf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c15aa80 tx=0x7f2d5c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:40.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.643+0000 7f2d53fff700 1 -- 172.21.15.81:0/1805162445 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2d600514f0 con 0x7f2d6c0ae670 2024-05-08T23:32:40.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.847+0000 7f2d71d0b700 1 -- 172.21.15.81:0/1805162445 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2d6c0742e0 con 0x7f2d6c0ae670 2024-05-08T23:32:40.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.859+0000 7f2d53fff700 1 -- 172.21.15.81:0/1805162445 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v19) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2d60054b10 con 0x7f2d6c0ae670 2024-05-08T23:32:40.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.867+0000 7f2d71d0b700 1 -- 172.21.15.81:0/1805162445 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2d58066730 msgr2=0x7f2d58068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:40.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.867+0000 7f2d71d0b700 1 --2- 172.21.15.81:0/1805162445 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2d58066730 0x7f2d58068bf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c15aa80 tx=0x7f2d5c009040 comp rx=0 tx=0).stop 2024-05-08T23:32:40.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.867+0000 7f2d71d0b700 1 -- 172.21.15.81:0/1805162445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d6c0ae670 msgr2=0x7f2d6c15a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:40.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.867+0000 7f2d71d0b700 1 --2- 172.21.15.81:0/1805162445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d6c0ae670 0x7f2d6c15a540 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000e760 tx=0x7f2d6000eb20 comp rx=0 tx=0).stop 2024-05-08T23:32:40.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.867+0000 7f2d71d0b700 1 -- 172.21.15.81:0/1805162445 shutdown_connections 2024-05-08T23:32:40.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.867+0000 7f2d71d0b700 1 --2- 172.21.15.81:0/1805162445 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2d58066730 0x7f2d58068bf0 secure :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c15aa80 tx=0x7f2d5c009040 comp rx=0 tx=0).stop 2024-05-08T23:32:40.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.867+0000 7f2d71d0b700 1 --2- 172.21.15.81:0/1805162445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d6c15d780 0x7f2d6c0b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:40.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.867+0000 7f2d71d0b700 1 --2- 172.21.15.81:0/1805162445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d6c0b4c40 0x7f2d6c15fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:40.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.867+0000 7f2d71d0b700 1 --2- 172.21.15.81:0/1805162445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d6c0ae670 0x7f2d6c15a540 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:40.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.867+0000 7f2d71d0b700 1 -- 172.21.15.81:0/1805162445 >> 172.21.15.81:0/1805162445 conn(0x7f2d6c0cf630 msgr2=0x7f2d6c0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:40.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.867+0000 7f2d71d0b700 1 -- 172.21.15.81:0/1805162445 shutdown_connections 2024-05-08T23:32:40.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.867+0000 7f2d71d0b700 1 -- 172.21.15.81:0/1805162445 wait complete. 2024-05-08T23:32:40.875 INFO:teuthology.orchestra.run.smithi081.stderr:new fs with metadata pool 7 and data pool 8 2024-05-08T23:32:40.893 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:40.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.991+0000 7fca47094700 1 Processor -- start 2024-05-08T23:32:40.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.991+0000 7fca47094700 1 -- start start 2024-05-08T23:32:40.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca47094700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca40159dc0 0x7fca400b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:40.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca47094700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fca4006b370 0x7fca4006b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:40.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca47094700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca400b2b50 con 0x7fca4015a1a0 2024-05-08T23:32:40.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca47094700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca4015b430 con 0x7fca4006b370 2024-05-08T23:32:40.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca47094700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca400ce2d0 con 0x7fca40159dc0 2024-05-08T23:32:40.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca45891700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fca4006b370 0x7fca4006b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:40.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca46893700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fca4015a1a0 0x7fca4015a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35300/0 (socket says 172.21.15.81:35300) 2024-05-08T23:32:40.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca46893700 1 -- 172.21.15.81:0/915357434 learned_addr learned my addr 172.21.15.81:0/915357434 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:40.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca46092700 1 --2- 172.21.15.81:0/915357434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca40159dc0 0x7fca400b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:40.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca45090700 1 -- 172.21.15.81:0/915357434 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3269883026 0 0) 0x7fca400b2b50 con 0x7fca4015a1a0 2024-05-08T23:32:40.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca45090700 1 -- 172.21.15.81:0/915357434 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca2c0035d0 con 0x7fca4015a1a0 2024-05-08T23:32:40.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca45891700 1 -- 172.21.15.81:0/915357434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca40159dc0 msgr2=0x7fca400b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:40.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca45891700 1 --2- 172.21.15.81:0/915357434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca40159dc0 0x7fca400b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:40.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca45891700 1 -- 172.21.15.81:0/915357434 >> v1:172.21.15.81:6789/0 conn(0x7fca4015a1a0 legacy=0x7fca4015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:40.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca45891700 1 -- 172.21.15.81:0/915357434 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4006bd10 con 0x7fca4006b370 2024-05-08T23:32:40.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca45891700 1 --2- 172.21.15.81:0/915357434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4006b370 0x7fca4006b7d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fca30009850 tx=0x7fca30009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f0acce80030b21 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:40.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca45090700 1 -- 172.21.15.81:0/915357434 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30011830 con 0x7fca4006b370 2024-05-08T23:32:40.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca45090700 1 -- 172.21.15.81:0/915357434 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca30011990 con 0x7fca4006b370 2024-05-08T23:32:40.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca45090700 1 -- 172.21.15.81:0/915357434 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30011d00 con 0x7fca4006b370 2024-05-08T23:32:40.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca47094700 1 -- 172.21.15.81:0/915357434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4006b370 msgr2=0x7fca4006b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:41.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca47094700 1 --2- 172.21.15.81:0/915357434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4006b370 0x7fca4006b7d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fca30009850 tx=0x7fca30009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:41.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca47094700 1 -- 172.21.15.81:0/915357434 shutdown_connections 2024-05-08T23:32:41.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca47094700 1 --2- 172.21.15.81:0/915357434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4006b370 0x7fca4006b7d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca47094700 1 --2- 172.21.15.81:0/915357434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca40159dc0 0x7fca400b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca47094700 1 -- 172.21.15.81:0/915357434 >> 172.21.15.81:0/915357434 conn(0x7fca40070960 msgr2=0x7fca40070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:41.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca47094700 1 -- 172.21.15.81:0/915357434 shutdown_connections 2024-05-08T23:32:41.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.995+0000 7fca47094700 1 -- 172.21.15.81:0/915357434 wait complete. 2024-05-08T23:32:41.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca47094700 1 Processor -- start 2024-05-08T23:32:41.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca47094700 1 -- start start 2024-05-08T23:32:41.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca47094700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4006b370 0x7fca40150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca47094700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca40159dc0 0x7fca4015c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca47094700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca4015a1a0 0x7fca4015cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca47094700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca400aa780 con 0x7fca4006b370 2024-05-08T23:32:41.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca47094700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca400aaa60 con 0x7fca40159dc0 2024-05-08T23:32:41.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca47094700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca400aad40 con 0x7fca4015a1a0 2024-05-08T23:32:41.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca46092700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4006b370 0x7fca40150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca46092700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4006b370 0x7fca40150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34270/0 (socket says 172.21.15.81:34270) 2024-05-08T23:32:41.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca46092700 1 -- 172.21.15.81:0/39341214 learned_addr learned my addr 172.21.15.81:0/39341214 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:41.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca45891700 1 --2- 172.21.15.81:0/39341214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca40159dc0 0x7fca4015c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca46893700 1 --2- 172.21.15.81:0/39341214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca4015a1a0 0x7fca4015cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca46092700 1 -- 172.21.15.81:0/39341214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca4015a1a0 msgr2=0x7fca4015cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:41.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca46092700 1 --2- 172.21.15.81:0/39341214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca4015a1a0 0x7fca4015cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca46092700 1 -- 172.21.15.81:0/39341214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca40159dc0 msgr2=0x7fca4015c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:41.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca46092700 1 --2- 172.21.15.81:0/39341214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca40159dc0 0x7fca4015c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca46092700 1 -- 172.21.15.81:0/39341214 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca300094b0 con 0x7fca4006b370 2024-05-08T23:32:41.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca46092700 1 --2- 172.21.15.81:0/39341214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4006b370 0x7fca40150370 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00e9d0 tx=0x7fca3c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:41.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca36ffd700 1 -- 172.21.15.81:0/39341214 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c009720 con 0x7fca4006b370 2024-05-08T23:32:41.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca36ffd700 1 -- 172.21.15.81:0/39341214 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c00ee70 con 0x7fca4006b370 2024-05-08T23:32:41.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca47094700 1 -- 172.21.15.81:0/39341214 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca400ab020 con 0x7fca4006b370 2024-05-08T23:32:41.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca36ffd700 1 -- 172.21.15.81:0/39341214 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00ca50 con 0x7fca4006b370 2024-05-08T23:32:41.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:40.999+0000 7fca47094700 1 -- 172.21.15.81:0/39341214 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca401508b0 con 0x7fca4006b370 2024-05-08T23:32:41.005 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.003+0000 7fca47094700 1 -- 172.21.15.81:0/39341214 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca400672a0 con 0x7fca4006b370 2024-05-08T23:32:41.005 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.003+0000 7fca36ffd700 1 -- 172.21.15.81:0/39341214 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fca3c00cbb0 con 0x7fca4006b370 2024-05-08T23:32:41.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.003+0000 7fca36ffd700 1 --2- 172.21.15.81:0/39341214 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fca24066730 0x7fca24068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.003+0000 7fca36ffd700 1 -- 172.21.15.81:0/39341214 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca3c084190 con 0x7fca4006b370 2024-05-08T23:32:41.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.003+0000 7fca45891700 1 --2- 172.21.15.81:0/39341214 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fca24066730 0x7fca24068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.003+0000 7fca45891700 1 --2- 172.21.15.81:0/39341214 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fca24066730 0x7fca24068bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fca300090b0 tx=0x7fca30008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:41.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.011+0000 7fca36ffd700 1 -- 172.21.15.81:0/39341214 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fca3c051500 con 0x7fca4006b370 2024-05-08T23:32:41.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.183+0000 7fca47094700 1 -- 172.21.15.81:0/39341214 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca4006cfe0 con 0x7fca4006b370 2024-05-08T23:32:41.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.187+0000 7fca36ffd700 1 -- 172.21.15.81:0/39341214 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16114 (secure 0 0 0) 0x7fca3c00ce60 con 0x7fca4006b370 2024-05-08T23:32:41.188 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:41.188 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":51,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:32:40.852508+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:39.161796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:39.853963+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:41.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.187+0000 7fca47094700 1 -- 172.21.15.81:0/39341214 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fca24066730 msgr2=0x7fca24068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:41.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.187+0000 7fca47094700 1 --2- 172.21.15.81:0/39341214 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fca24066730 0x7fca24068bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fca300090b0 tx=0x7fca30008030 comp rx=0 tx=0).stop 2024-05-08T23:32:41.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.187+0000 7fca47094700 1 -- 172.21.15.81:0/39341214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4006b370 msgr2=0x7fca40150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:41.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.187+0000 7fca47094700 1 --2- 172.21.15.81:0/39341214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4006b370 0x7fca40150370 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00e9d0 tx=0x7fca3c00ece0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.187+0000 7fca47094700 1 -- 172.21.15.81:0/39341214 shutdown_connections 2024-05-08T23:32:41.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.187+0000 7fca47094700 1 --2- 172.21.15.81:0/39341214 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fca24066730 0x7fca24068bf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.187+0000 7fca47094700 1 --2- 172.21.15.81:0/39341214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca4015a1a0 0x7fca4015cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.187+0000 7fca47094700 1 --2- 172.21.15.81:0/39341214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca40159dc0 0x7fca4015c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.187+0000 7fca47094700 1 --2- 172.21.15.81:0/39341214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4006b370 0x7fca40150370 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.187+0000 7fca47094700 1 -- 172.21.15.81:0/39341214 >> 172.21.15.81:0/39341214 conn(0x7fca40070960 msgr2=0x7fca40159090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:41.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.187+0000 7fca47094700 1 -- 172.21.15.81:0/39341214 shutdown_connections 2024-05-08T23:32:41.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.187+0000 7fca47094700 1 -- 172.21.15.81:0/39341214 wait complete. 2024-05-08T23:32:41.203 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:41.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06759e700 1 Processor -- start 2024-05-08T23:32:41.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06759e700 1 -- start start 2024-05-08T23:32:41.323 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd068071ab0 0x7fd0680ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.323 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06759e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd068071e90 0x7fd06815a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.323 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06759e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0680cea50 con 0x7fd06815aaa0 2024-05-08T23:32:41.323 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06759e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0680ce080 con 0x7fd068071e90 2024-05-08T23:32:41.323 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680cd7c0 con 0x7fd068071ab0 2024-05-08T23:32:41.323 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd065d9b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd068071e90 0x7fd06815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.323 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd066d9d700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd06815aaa0 0x7fd06813d500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35312/0 (socket says 172.21.15.81:35312) 2024-05-08T23:32:41.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd065d9b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd068071e90 0x7fd06815a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34286/0 (socket says 172.21.15.81:34286) 2024-05-08T23:32:41.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd066d9d700 1 -- 172.21.15.81:0/3102603468 learned_addr learned my addr 172.21.15.81:0/3102603468 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:41.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06659c700 1 --2- 172.21.15.81:0/3102603468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd068071ab0 0x7fd0680ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd065d9b700 1 -- 172.21.15.81:0/3102603468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd068071ab0 msgr2=0x7fd0680ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:41.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd065d9b700 1 --2- 172.21.15.81:0/3102603468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd068071ab0 0x7fd0680ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd065d9b700 1 -- 172.21.15.81:0/3102603468 >> v1:172.21.15.81:6789/0 conn(0x7fd06815aaa0 legacy=0x7fd06813d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:41.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06559a700 1 -- 172.21.15.81:0/3102603468 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401800583 0 0) 0x7fd060001490 con 0x7fd06815aaa0 2024-05-08T23:32:41.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd065d9b700 1 -- 172.21.15.81:0/3102603468 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0680cda30 con 0x7fd068071e90 2024-05-08T23:32:41.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd065d9b700 1 --2- 172.21.15.81:0/3102603468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd068071e90 0x7fd06815a560 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd058009850 tx=0x7fd058009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1581916a04dd9e5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:41.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06559a700 1 -- 172.21.15.81:0/3102603468 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058011830 con 0x7fd068071e90 2024-05-08T23:32:41.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06559a700 1 -- 172.21.15.81:0/3102603468 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd058011990 con 0x7fd068071e90 2024-05-08T23:32:41.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06559a700 1 -- 172.21.15.81:0/3102603468 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058011d20 con 0x7fd068071e90 2024-05-08T23:32:41.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06759e700 1 -- 172.21.15.81:0/3102603468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd068071e90 msgr2=0x7fd06815a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:41.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06759e700 1 --2- 172.21.15.81:0/3102603468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd068071e90 0x7fd06815a560 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd058009850 tx=0x7fd058009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:41.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06759e700 1 -- 172.21.15.81:0/3102603468 shutdown_connections 2024-05-08T23:32:41.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06759e700 1 --2- 172.21.15.81:0/3102603468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd068071e90 0x7fd06815a560 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06759e700 1 --2- 172.21.15.81:0/3102603468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd068071ab0 0x7fd0680ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06759e700 1 -- 172.21.15.81:0/3102603468 >> 172.21.15.81:0/3102603468 conn(0x7fd06806c330 msgr2=0x7fd06806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:41.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06759e700 1 -- 172.21.15.81:0/3102603468 shutdown_connections 2024-05-08T23:32:41.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.319+0000 7fd06759e700 1 -- 172.21.15.81:0/3102603468 wait complete. 2024-05-08T23:32:41.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.323+0000 7fd06759e700 1 Processor -- start 2024-05-08T23:32:41.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.323+0000 7fd06759e700 1 -- start start 2024-05-08T23:32:41.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.323+0000 7fd06759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd068071ab0 0x7fd068143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.323+0000 7fd06759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd068071e90 0x7fd06813a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.323+0000 7fd06759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06815aaa0 0x7fd0680b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.323+0000 7fd06759e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0681eea90 con 0x7fd068071e90 2024-05-08T23:32:41.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.323+0000 7fd06759e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06813db00 con 0x7fd06815aaa0 2024-05-08T23:32:41.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.323+0000 7fd06759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0681435d0 con 0x7fd068071ab0 2024-05-08T23:32:41.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.323+0000 7fd065d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd068071e90 0x7fd06813a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.323+0000 7fd065d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd068071e90 0x7fd06813a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34298/0 (socket says 172.21.15.81:34298) 2024-05-08T23:32:41.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.323+0000 7fd065d9b700 1 -- 172.21.15.81:0/3227867537 learned_addr learned my addr 172.21.15.81:0/3227867537 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:41.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.323+0000 7fd06659c700 1 --2- 172.21.15.81:0/3227867537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd068071ab0 0x7fd068143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.323+0000 7fd066d9d700 1 --2- 172.21.15.81:0/3227867537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06815aaa0 0x7fd0680b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd065d9b700 1 -- 172.21.15.81:0/3227867537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd068071ab0 msgr2=0x7fd068143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:41.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd065d9b700 1 --2- 172.21.15.81:0/3227867537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd068071ab0 0x7fd068143090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd065d9b700 1 -- 172.21.15.81:0/3227867537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06815aaa0 msgr2=0x7fd0680b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:41.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd065d9b700 1 --2- 172.21.15.81:0/3227867537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06815aaa0 0x7fd0680b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd065d9b700 1 -- 172.21.15.81:0/3227867537 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0580094b0 con 0x7fd068071e90 2024-05-08T23:32:41.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd066d9d700 1 --2- 172.21.15.81:0/3227867537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06815aaa0 0x7fd0680b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:41.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd065d9b700 1 --2- 172.21.15.81:0/3227867537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd068071e90 0x7fd06813a2e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd058000c00 tx=0x7fd058004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:41.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd05effd700 1 -- 172.21.15.81:0/3227867537 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058004590 con 0x7fd068071e90 2024-05-08T23:32:41.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd05effd700 1 -- 172.21.15.81:0/3227867537 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0580036c0 con 0x7fd068071e90 2024-05-08T23:32:41.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd06759e700 1 -- 172.21.15.81:0/3227867537 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06813fb70 con 0x7fd068071e90 2024-05-08T23:32:41.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd05effd700 1 -- 172.21.15.81:0/3227867537 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058003a50 con 0x7fd068071e90 2024-05-08T23:32:41.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd06759e700 1 -- 172.21.15.81:0/3227867537 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd068140130 con 0x7fd068071e90 2024-05-08T23:32:41.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd06759e700 1 -- 172.21.15.81:0/3227867537 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0680cd200 con 0x7fd068071e90 2024-05-08T23:32:41.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd05effd700 1 -- 172.21.15.81:0/3227867537 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd058003bb0 con 0x7fd068071e90 2024-05-08T23:32:41.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd05effd700 1 --2- 172.21.15.81:0/3227867537 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd048066730 0x7fd048068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd05effd700 1 -- 172.21.15.81:0/3227867537 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd058021030 con 0x7fd068071e90 2024-05-08T23:32:41.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.327+0000 7fd06659c700 1 --2- 172.21.15.81:0/3227867537 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd048066730 0x7fd048068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.331+0000 7fd06659c700 1 --2- 172.21.15.81:0/3227867537 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd048066730 0x7fd048068bf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd050009a00 tx=0x7fd050008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:41.337 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.335+0000 7fd05effd700 1 -- 172.21.15.81:0/3227867537 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd058052ad0 con 0x7fd068071e90 2024-05-08T23:32:41.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.511+0000 7fd06759e700 1 -- 172.21.15.81:0/3227867537 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0680684b0 con 0x7fd068071e90 2024-05-08T23:32:41.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.515+0000 7fd05effd700 1 -- 172.21.15.81:0/3227867537 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16114 (secure 0 0 0) 0x7fd058015090 con 0x7fd068071e90 2024-05-08T23:32:41.516 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:41.516 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":51,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:32:40.852508+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:39.161796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:39.853963+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:41.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.515+0000 7fd06759e700 1 -- 172.21.15.81:0/3227867537 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd048066730 msgr2=0x7fd048068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:41.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.515+0000 7fd06759e700 1 --2- 172.21.15.81:0/3227867537 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd048066730 0x7fd048068bf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd050009a00 tx=0x7fd050008040 comp rx=0 tx=0).stop 2024-05-08T23:32:41.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.515+0000 7fd06759e700 1 -- 172.21.15.81:0/3227867537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd068071e90 msgr2=0x7fd06813a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:41.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.515+0000 7fd06759e700 1 --2- 172.21.15.81:0/3227867537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd068071e90 0x7fd06813a2e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd058000c00 tx=0x7fd058004770 comp rx=0 tx=0).stop 2024-05-08T23:32:41.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.515+0000 7fd06759e700 1 -- 172.21.15.81:0/3227867537 shutdown_connections 2024-05-08T23:32:41.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.515+0000 7fd06759e700 1 --2- 172.21.15.81:0/3227867537 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd048066730 0x7fd048068bf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.515+0000 7fd06759e700 1 --2- 172.21.15.81:0/3227867537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06815aaa0 0x7fd0680b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.515+0000 7fd06759e700 1 --2- 172.21.15.81:0/3227867537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd068071e90 0x7fd06813a2e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.515+0000 7fd06759e700 1 --2- 172.21.15.81:0/3227867537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd068071ab0 0x7fd068143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.515+0000 7fd06759e700 1 -- 172.21.15.81:0/3227867537 >> 172.21.15.81:0/3227867537 conn(0x7fd06806c330 msgr2=0x7fd06815c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:41.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.515+0000 7fd06759e700 1 -- 172.21.15.81:0/3227867537 shutdown_connections 2024-05-08T23:32:41.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.515+0000 7fd06759e700 1 -- 172.21.15.81:0/3227867537 wait complete. 2024-05-08T23:32:41.532 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T23:32:41.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.647+0000 7fedcf59e700 1 Processor -- start 2024-05-08T23:32:41.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.647+0000 7fedcf59e700 1 -- start start 2024-05-08T23:32:41.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcf59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedd006f5d0 0x7fedd006a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcf59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fedd006a8a0 0x7fedd00acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcf59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd00ad740 con 0x7fedd00ad2f0 2024-05-08T23:32:41.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcf59e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd00aea50 con 0x7fedd006a8a0 2024-05-08T23:32:41.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcf59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd006ad20 con 0x7fedd006f5d0 2024-05-08T23:32:41.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedc5d9b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fedd006a8a0 0x7fedd00acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedc5d9b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fedd006a8a0 0x7fedd00acdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34306/0 (socket says 172.21.15.81:34306) 2024-05-08T23:32:41.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedc5d9b700 1 -- 172.21.15.81:0/1022249697 learned_addr learned my addr 172.21.15.81:0/1022249697 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:41.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedce59c700 1 --2- 172.21.15.81:0/1022249697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedd006f5d0 0x7fedd006a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedc5d9b700 1 -- 172.21.15.81:0/1022249697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedd006f5d0 msgr2=0x7fedd006a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:41.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedc5d9b700 1 --2- 172.21.15.81:0/1022249697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedd006f5d0 0x7fedd006a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedc5d9b700 1 -- 172.21.15.81:0/1022249697 >> v1:172.21.15.117:6789/0 conn(0x7fedd00ad2f0 legacy=0x7fedd0156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:41.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedc5d9b700 1 -- 172.21.15.81:0/1022249697 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd00af340 con 0x7fedd006a8a0 2024-05-08T23:32:41.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedce59c700 1 --2- 172.21.15.81:0/1022249697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedd006f5d0 0x7fedd006a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:41.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedc5d9b700 1 --2- 172.21.15.81:0/1022249697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedd006a8a0 0x7fedd00acdb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009910 tx=0x7fedb8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97176cf4a9ba3f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:41.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcdd9b700 1 -- 172.21.15.81:0/1022249697 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb800ba60 con 0x7fedd006a8a0 2024-05-08T23:32:41.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcdd9b700 1 -- 172.21.15.81:0/1022249697 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb800bbc0 con 0x7fedd006a8a0 2024-05-08T23:32:41.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcdd9b700 1 -- 172.21.15.81:0/1022249697 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8009e60 con 0x7fedd006a8a0 2024-05-08T23:32:41.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcf59e700 1 -- 172.21.15.81:0/1022249697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedd006a8a0 msgr2=0x7fedd00acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:41.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcf59e700 1 --2- 172.21.15.81:0/1022249697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedd006a8a0 0x7fedd00acdb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009910 tx=0x7fedb8009c20 comp rx=0 tx=0).stop 2024-05-08T23:32:41.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcf59e700 1 -- 172.21.15.81:0/1022249697 shutdown_connections 2024-05-08T23:32:41.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcf59e700 1 --2- 172.21.15.81:0/1022249697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedd006a8a0 0x7fedd00acdb0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcf59e700 1 --2- 172.21.15.81:0/1022249697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedd006f5d0 0x7fedd006a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcf59e700 1 -- 172.21.15.81:0/1022249697 >> 172.21.15.81:0/1022249697 conn(0x7fedd00b1a60 msgr2=0x7fedd00b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:41.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcf59e700 1 -- 172.21.15.81:0/1022249697 shutdown_connections 2024-05-08T23:32:41.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.651+0000 7fedcf59e700 1 -- 172.21.15.81:0/1022249697 wait complete. 2024-05-08T23:32:41.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedcf59e700 1 Processor -- start 2024-05-08T23:32:41.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedcf59e700 1 -- start start 2024-05-08T23:32:41.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedcf59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd006a8a0 0x7fedd0151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedcf59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedd006f5d0 0x7fedd015f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedcf59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedd00ad2f0 0x7fedd015fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedcf59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd0158860 con 0x7fedd00ad2f0 2024-05-08T23:32:41.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedcf59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd0158510 con 0x7fedd006a8a0 2024-05-08T23:32:41.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedcf59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd0153070 con 0x7fedd006f5d0 2024-05-08T23:32:41.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedced9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedd00ad2f0 0x7fedd015fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedced9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedd00ad2f0 0x7fedd015fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34318/0 (socket says 172.21.15.81:34318) 2024-05-08T23:32:41.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedced9d700 1 -- 172.21.15.81:0/331248170 learned_addr learned my addr 172.21.15.81:0/331248170 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:41.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedce59c700 1 --2- 172.21.15.81:0/331248170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd006a8a0 0x7fedd0151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedc5d9b700 1 --2- 172.21.15.81:0/331248170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedd006f5d0 0x7fedd015f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedced9d700 1 -- 172.21.15.81:0/331248170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedd006f5d0 msgr2=0x7fedd015f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:41.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedced9d700 1 --2- 172.21.15.81:0/331248170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedd006f5d0 0x7fedd015f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedced9d700 1 -- 172.21.15.81:0/331248170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd006a8a0 msgr2=0x7fedd0151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:41.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedced9d700 1 --2- 172.21.15.81:0/331248170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd006a8a0 0x7fedd0151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:41.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedced9d700 1 -- 172.21.15.81:0/331248170 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb8009570 con 0x7fedd00ad2f0 2024-05-08T23:32:41.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedced9d700 1 --2- 172.21.15.81:0/331248170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedd00ad2f0 0x7fedd015fc60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fedc800cb50 tx=0x7fedc800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:41.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedc77fe700 1 -- 172.21.15.81:0/331248170 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8003c20 con 0x7fedd00ad2f0 2024-05-08T23:32:41.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedc77fe700 1 -- 172.21.15.81:0/331248170 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc8003d80 con 0x7fedd00ad2f0 2024-05-08T23:32:41.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedcf59e700 1 -- 172.21.15.81:0/331248170 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd0153350 con 0x7fedd00ad2f0 2024-05-08T23:32:41.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedc77fe700 1 -- 172.21.15.81:0/331248170 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc800aa20 con 0x7fedd00ad2f0 2024-05-08T23:32:41.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.655+0000 7fedcf59e700 1 -- 172.21.15.81:0/331248170 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd0151a70 con 0x7fedd00ad2f0 2024-05-08T23:32:41.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.659+0000 7fedcf59e700 1 -- 172.21.15.81:0/331248170 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd01589a0 con 0x7fedd00ad2f0 2024-05-08T23:32:41.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.659+0000 7fedc77fe700 1 -- 172.21.15.81:0/331248170 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fedc800ab80 con 0x7fedd00ad2f0 2024-05-08T23:32:41.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.659+0000 7fedc77fe700 1 --2- 172.21.15.81:0/331248170 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fedb0066760 0x7fedb0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:41.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.659+0000 7fedc77fe700 1 -- 172.21.15.81:0/331248170 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedc80840a0 con 0x7fedd00ad2f0 2024-05-08T23:32:41.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.659+0000 7fedce59c700 1 --2- 172.21.15.81:0/331248170 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fedb0066760 0x7fedb0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:41.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.659+0000 7fedce59c700 1 --2- 172.21.15.81:0/331248170 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fedb0066760 0x7fedb0068c20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fedc00099f0 tx=0x7fedc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:41.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.671+0000 7fedc77fe700 1 -- 172.21.15.81:0/331248170 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fedc80514a0 con 0x7fedd00ad2f0 2024-05-08T23:32:41.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:41.875+0000 7fedcf59e700 1 -- 172.21.15.81:0/331248170 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fedd00672a0 con 0x7fedd00ad2f0 2024-05-08T23:32:42.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:42.867+0000 7fedc77fe700 1 -- 172.21.15.81:0/331248170 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7fedc8054ac0 con 0x7fedd00ad2f0 2024-05-08T23:32:42.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:42.875+0000 7fedcf59e700 1 -- 172.21.15.81:0/331248170 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fedb0066760 msgr2=0x7fedb0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:42.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:42.875+0000 7fedcf59e700 1 --2- 172.21.15.81:0/331248170 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fedb0066760 0x7fedb0068c20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fedc00099f0 tx=0x7fedc0008040 comp rx=0 tx=0).stop 2024-05-08T23:32:42.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:42.875+0000 7fedcf59e700 1 -- 172.21.15.81:0/331248170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedd00ad2f0 msgr2=0x7fedd015fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:42.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:42.875+0000 7fedcf59e700 1 --2- 172.21.15.81:0/331248170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedd00ad2f0 0x7fedd015fc60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fedc800cb50 tx=0x7fedc800ce60 comp rx=0 tx=0).stop 2024-05-08T23:32:42.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:42.875+0000 7fedcf59e700 1 -- 172.21.15.81:0/331248170 shutdown_connections 2024-05-08T23:32:42.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:42.875+0000 7fedcf59e700 1 --2- 172.21.15.81:0/331248170 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fedb0066760 0x7fedb0068c20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:42.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:42.875+0000 7fedcf59e700 1 --2- 172.21.15.81:0/331248170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedd00ad2f0 0x7fedd015fc60 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:42.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:42.875+0000 7fedcf59e700 1 --2- 172.21.15.81:0/331248170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedd006f5d0 0x7fedd015f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:42.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:42.875+0000 7fedcf59e700 1 --2- 172.21.15.81:0/331248170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd006a8a0 0x7fedd0151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:42.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:42.875+0000 7fedcf59e700 1 -- 172.21.15.81:0/331248170 >> 172.21.15.81:0/331248170 conn(0x7fedd00b1a60 msgr2=0x7fedd0071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:42.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:42.875+0000 7fedcf59e700 1 -- 172.21.15.81:0/331248170 shutdown_connections 2024-05-08T23:32:42.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:42.875+0000 7fedcf59e700 1 -- 172.21.15.81:0/331248170 wait complete. 2024-05-08T23:32:42.893 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T23:32:42.893 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T23:32:43.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.015+0000 7fc5cb59e700 1 Processor -- start 2024-05-08T23:32:43.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.015+0000 7fc5cb59e700 1 -- start start 2024-05-08T23:32:43.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.015+0000 7fc5cb59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5cc160920 0x7fc5cc0ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:43.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.015+0000 7fc5cb59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc5cc0d0050 0x7fc5cc0b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:43.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.015+0000 7fc5cb59e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5cc0d3d40 con 0x7fc5cc0d1da0 2024-05-08T23:32:43.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.015+0000 7fc5cb59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5cc0d0d00 con 0x7fc5cc0d0050 2024-05-08T23:32:43.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.015+0000 7fc5cb59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0d0fe0 con 0x7fc5cc160920 2024-05-08T23:32:43.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5c9d9b700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc5cc0d1da0 0x7fc5cc0cf940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35316/0 (socket says 172.21.15.81:35316) 2024-05-08T23:32:43.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5c9d9b700 1 -- 172.21.15.81:0/3360923527 learned_addr learned my addr 172.21.15.81:0/3360923527 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:43.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5cad9d700 1 --2- 172.21.15.81:0/3360923527 >> v2:172.21.15.117:3300/0 conn(0x7fc5cc0d0050 0x7fc5cc0b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:43.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5ca59c700 1 --2- 172.21.15.81:0/3360923527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5cc160920 0x7fc5cc0ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:43.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5c959a700 1 -- 172.21.15.81:0/3360923527 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801126493 0 0) 0x7fc5cc0d3d40 con 0x7fc5cc0d1da0 2024-05-08T23:32:43.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5c959a700 1 -- 172.21.15.81:0/3360923527 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b40035d0 con 0x7fc5cc0d1da0 2024-05-08T23:32:43.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5cad9d700 1 -- 172.21.15.81:0/3360923527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5cc160920 msgr2=0x7fc5cc0ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:43.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5cad9d700 1 --2- 172.21.15.81:0/3360923527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5cc160920 0x7fc5cc0ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:43.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5cad9d700 1 -- 172.21.15.81:0/3360923527 >> v1:172.21.15.81:6789/0 conn(0x7fc5cc0d1da0 legacy=0x7fc5cc0cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:43.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5cad9d700 1 -- 172.21.15.81:0/3360923527 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc0d12c0 con 0x7fc5cc0d0050 2024-05-08T23:32:43.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5cad9d700 1 --2- 172.21.15.81:0/3360923527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc0d0050 0x7fc5cc0b5e50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009d90 tx=0x7fc5c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=828ca4ab34ed6be2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:43.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5c959a700 1 -- 172.21.15.81:0/3360923527 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c400e070 con 0x7fc5cc0d0050 2024-05-08T23:32:43.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5c959a700 1 -- 172.21.15.81:0/3360923527 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c400bb20 con 0x7fc5cc0d0050 2024-05-08T23:32:43.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5c959a700 1 -- 172.21.15.81:0/3360923527 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c400be90 con 0x7fc5cc0d0050 2024-05-08T23:32:43.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5cb59e700 1 -- 172.21.15.81:0/3360923527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc0d0050 msgr2=0x7fc5cc0b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:43.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5cb59e700 1 --2- 172.21.15.81:0/3360923527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc0d0050 0x7fc5cc0b5e50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009d90 tx=0x7fc5c400b330 comp rx=0 tx=0).stop 2024-05-08T23:32:43.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5cb59e700 1 -- 172.21.15.81:0/3360923527 shutdown_connections 2024-05-08T23:32:43.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5cb59e700 1 --2- 172.21.15.81:0/3360923527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc0d0050 0x7fc5cc0b5e50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:43.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5cb59e700 1 --2- 172.21.15.81:0/3360923527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5cc160920 0x7fc5cc0ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:43.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5cb59e700 1 -- 172.21.15.81:0/3360923527 >> 172.21.15.81:0/3360923527 conn(0x7fc5cc09c340 msgr2=0x7fc5cc0d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:43.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5cb59e700 1 -- 172.21.15.81:0/3360923527 shutdown_connections 2024-05-08T23:32:43.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.019+0000 7fc5cb59e700 1 -- 172.21.15.81:0/3360923527 wait complete. 2024-05-08T23:32:43.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.023+0000 7fc5cb59e700 1 Processor -- start 2024-05-08T23:32:43.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.023+0000 7fc5cb59e700 1 -- start start 2024-05-08T23:32:43.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.023+0000 7fc5cb59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc0d0050 0x7fc5cc0cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:43.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.023+0000 7fc5cb59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5cc0d1da0 0x7fc5cc0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:43.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.023+0000 7fc5cb59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5cc160920 0x7fc5cc15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:43.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.023+0000 7fc5cb59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc1f6d90 con 0x7fc5cc160920 2024-05-08T23:32:43.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.023+0000 7fc5cb59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0cd520 con 0x7fc5cc0d0050 2024-05-08T23:32:43.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.023+0000 7fc5cb59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0ca5d0 con 0x7fc5cc0d1da0 2024-05-08T23:32:43.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.023+0000 7fc5cad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5cc160920 0x7fc5cc15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:43.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.023+0000 7fc5cad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5cc160920 0x7fc5cc15a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34324/0 (socket says 172.21.15.81:34324) 2024-05-08T23:32:43.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.023+0000 7fc5ca59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc0d0050 0x7fc5cc0cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:43.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.023+0000 7fc5cad9d700 1 -- 172.21.15.81:0/1659821297 learned_addr learned my addr 172.21.15.81:0/1659821297 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:43.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.023+0000 7fc5c9d9b700 1 --2- 172.21.15.81:0/1659821297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5cc0d1da0 0x7fc5cc0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:43.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.023+0000 7fc5cad9d700 1 -- 172.21.15.81:0/1659821297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5cc0d1da0 msgr2=0x7fc5cc0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:43.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.027+0000 7fc5cad9d700 1 --2- 172.21.15.81:0/1659821297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5cc0d1da0 0x7fc5cc0ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:43.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.027+0000 7fc5cad9d700 1 -- 172.21.15.81:0/1659821297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc0d0050 msgr2=0x7fc5cc0cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:43.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.027+0000 7fc5cad9d700 1 --2- 172.21.15.81:0/1659821297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc0d0050 0x7fc5cc0cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:43.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.027+0000 7fc5cad9d700 1 -- 172.21.15.81:0/1659821297 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c4009a70 con 0x7fc5cc160920 2024-05-08T23:32:43.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.027+0000 7fc5c9d9b700 1 --2- 172.21.15.81:0/1659821297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5cc0d1da0 0x7fc5cc0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:43.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.027+0000 7fc5cad9d700 1 --2- 172.21.15.81:0/1659821297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5cc160920 0x7fc5cc15a9f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4014040 tx=0x7fc5c4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:43.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.027+0000 7fc5baffd700 1 -- 172.21.15.81:0/1659821297 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4004800 con 0x7fc5cc160920 2024-05-08T23:32:43.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.027+0000 7fc5baffd700 1 -- 172.21.15.81:0/1659821297 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c4020070 con 0x7fc5cc160920 2024-05-08T23:32:43.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.027+0000 7fc5baffd700 1 -- 172.21.15.81:0/1659821297 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40276c0 con 0x7fc5cc160920 2024-05-08T23:32:43.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.027+0000 7fc5cb59e700 1 -- 172.21.15.81:0/1659821297 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5cc160e80 con 0x7fc5cc160920 2024-05-08T23:32:43.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.027+0000 7fc5cb59e700 1 -- 172.21.15.81:0/1659821297 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5cc0cab90 con 0x7fc5cc160920 2024-05-08T23:32:43.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.027+0000 7fc5cb59e700 1 -- 172.21.15.81:0/1659821297 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5cc0684b0 con 0x7fc5cc160920 2024-05-08T23:32:43.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.027+0000 7fc5baffd700 1 -- 172.21.15.81:0/1659821297 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc5c4012430 con 0x7fc5cc160920 2024-05-08T23:32:43.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.027+0000 7fc5baffd700 1 --2- 172.21.15.81:0/1659821297 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc5ac066760 0x7fc5ac068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:43.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.031+0000 7fc5ca59c700 1 --2- 172.21.15.81:0/1659821297 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc5ac066760 0x7fc5ac068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:43.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.031+0000 7fc5baffd700 1 -- 172.21.15.81:0/1659821297 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5c4022070 con 0x7fc5cc160920 2024-05-08T23:32:43.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.031+0000 7fc5ca59c700 1 --2- 172.21.15.81:0/1659821297 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc5ac066760 0x7fc5ac068c20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009bf0 tx=0x7fc5c0009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:43.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.035+0000 7fc5baffd700 1 -- 172.21.15.81:0/1659821297 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc5c4053010 con 0x7fc5cc160920 2024-05-08T23:32:43.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.243+0000 7fc5cb59e700 1 -- 172.21.15.81:0/1659821297 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc5cc0cf510 con 0x7fc5cc160920 2024-05-08T23:32:43.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.871+0000 7fc5baffd700 1 -- 172.21.15.81:0/1659821297 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc5c4056630 con 0x7fc5cc160920 2024-05-08T23:32:43.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.875+0000 7fc5cb59e700 1 -- 172.21.15.81:0/1659821297 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc5ac066760 msgr2=0x7fc5ac068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:43.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.875+0000 7fc5cb59e700 1 --2- 172.21.15.81:0/1659821297 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc5ac066760 0x7fc5ac068c20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009bf0 tx=0x7fc5c0009240 comp rx=0 tx=0).stop 2024-05-08T23:32:43.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.875+0000 7fc5cb59e700 1 -- 172.21.15.81:0/1659821297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5cc160920 msgr2=0x7fc5cc15a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:43.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.875+0000 7fc5cb59e700 1 --2- 172.21.15.81:0/1659821297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5cc160920 0x7fc5cc15a9f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4014040 tx=0x7fc5c4004af0 comp rx=0 tx=0).stop 2024-05-08T23:32:43.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.875+0000 7fc5cb59e700 1 -- 172.21.15.81:0/1659821297 shutdown_connections 2024-05-08T23:32:43.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.875+0000 7fc5cb59e700 1 --2- 172.21.15.81:0/1659821297 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc5ac066760 0x7fc5ac068c20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:43.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.875+0000 7fc5cb59e700 1 --2- 172.21.15.81:0/1659821297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5cc160920 0x7fc5cc15a9f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:43.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.875+0000 7fc5cb59e700 1 --2- 172.21.15.81:0/1659821297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5cc0d1da0 0x7fc5cc0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:43.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.875+0000 7fc5cb59e700 1 --2- 172.21.15.81:0/1659821297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5cc0d0050 0x7fc5cc0cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:43.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.875+0000 7fc5cb59e700 1 -- 172.21.15.81:0/1659821297 >> 172.21.15.81:0/1659821297 conn(0x7fc5cc09c340 msgr2=0x7fc5cc15e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:43.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.875+0000 7fc5cb59e700 1 -- 172.21.15.81:0/1659821297 shutdown_connections 2024-05-08T23:32:43.879 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:43.875+0000 7fc5cb59e700 1 -- 172.21.15.81:0/1659821297 wait complete. 2024-05-08T23:32:43.894 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T23:32:43.894 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:44.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 Processor -- start 2024-05-08T23:32:44.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 -- start start 2024-05-08T23:32:44.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd6840b3c20 0x7fd6841624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd684071e70 0x7fd6840b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd684165070 0x7fd684165500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6840b1f60 con 0x7fd6840b3c20 2024-05-08T23:32:44.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd684158960 con 0x7fd684071e70 2024-05-08T23:32:44.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840b4540 con 0x7fd684165070 2024-05-08T23:32:44.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68919f700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd6840b3c20 0x7fd6841624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68919f700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd6840b3c20 0x7fd6841624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34330/0 (socket says 172.21.15.81:34330) 2024-05-08T23:32:44.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68919f700 1 -- 172.21.15.81:0/3187147831 learned_addr learned my addr 172.21.15.81:0/3187147831 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:44.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68899e700 1 --2- 172.21.15.81:0/3187147831 >> v2:172.21.15.117:3300/0 conn(0x7fd684071e70 0x7fd6840b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd6899a0700 1 --2- 172.21.15.81:0/3187147831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd684165070 0x7fd684165500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68919f700 1 -- 172.21.15.81:0/3187147831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd684165070 msgr2=0x7fd684165500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:44.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68919f700 1 --2- 172.21.15.81:0/3187147831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd684165070 0x7fd684165500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68919f700 1 -- 172.21.15.81:0/3187147831 >> v2:172.21.15.117:3300/0 conn(0x7fd684071e70 msgr2=0x7fd6840b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:44.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68919f700 1 --2- 172.21.15.81:0/3187147831 >> v2:172.21.15.117:3300/0 conn(0x7fd684071e70 0x7fd6840b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68919f700 1 -- 172.21.15.81:0/3187147831 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6840ae990 con 0x7fd6840b3c20 2024-05-08T23:32:44.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68919f700 1 --2- 172.21.15.81:0/3187147831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6840b3c20 0x7fd6841624b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd68406b800 tx=0x7fd680009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e77b2ae4f0abbe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:44.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd67bfff700 1 -- 172.21.15.81:0/3187147831 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd680011830 con 0x7fd6840b3c20 2024-05-08T23:32:44.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd67bfff700 1 -- 172.21.15.81:0/3187147831 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd680011990 con 0x7fd6840b3c20 2024-05-08T23:32:44.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd67bfff700 1 -- 172.21.15.81:0/3187147831 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd680011d20 con 0x7fd6840b3c20 2024-05-08T23:32:44.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 -- 172.21.15.81:0/3187147831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6840b3c20 msgr2=0x7fd6841624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 --2- 172.21.15.81:0/3187147831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6840b3c20 0x7fd6841624b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd68406b800 tx=0x7fd680009bf0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 -- 172.21.15.81:0/3187147831 shutdown_connections 2024-05-08T23:32:44.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 --2- 172.21.15.81:0/3187147831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd684165070 0x7fd684165500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 --2- 172.21.15.81:0/3187147831 >> v2:172.21.15.117:3300/0 conn(0x7fd684071e70 0x7fd6840b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 --2- 172.21.15.81:0/3187147831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6840b3c20 0x7fd6841624b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 -- 172.21.15.81:0/3187147831 >> 172.21.15.81:0/3187147831 conn(0x7fd68406c330 msgr2=0x7fd68406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:44.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 -- 172.21.15.81:0/3187147831 shutdown_connections 2024-05-08T23:32:44.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.011+0000 7fd68a1a1700 1 -- 172.21.15.81:0/3187147831 wait complete. 2024-05-08T23:32:44.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.015+0000 7fd68a1a1700 1 Processor -- start 2024-05-08T23:32:44.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.015+0000 7fd68a1a1700 1 -- start start 2024-05-08T23:32:44.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.015+0000 7fd68a1a1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd684071e70 0x7fd684151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.015+0000 7fd68a1a1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6840b3c20 0x7fd68414e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.015+0000 7fd68a1a1700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd684165070 0x7fd684158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68a1a1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840cea00 con 0x7fd684071e70 2024-05-08T23:32:44.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68919f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd684071e70 0x7fd684151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68a1a1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd684159a40 con 0x7fd6840b3c20 2024-05-08T23:32:44.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68a1a1700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6841591a0 con 0x7fd684165070 2024-05-08T23:32:44.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68919f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd684071e70 0x7fd684151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34338/0 (socket says 172.21.15.81:34338) 2024-05-08T23:32:44.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68919f700 1 -- 172.21.15.81:0/841548333 learned_addr learned my addr 172.21.15.81:0/841548333 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:44.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68899e700 1 --2- 172.21.15.81:0/841548333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6840b3c20 0x7fd68414e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd6899a0700 1 --2- 172.21.15.81:0/841548333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd684165070 0x7fd684158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68919f700 1 -- 172.21.15.81:0/841548333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd684165070 msgr2=0x7fd684158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68919f700 1 --2- 172.21.15.81:0/841548333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd684165070 0x7fd684158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68919f700 1 -- 172.21.15.81:0/841548333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6840b3c20 msgr2=0x7fd68414e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68919f700 1 --2- 172.21.15.81:0/841548333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6840b3c20 0x7fd68414e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68919f700 1 -- 172.21.15.81:0/841548333 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd680009570 con 0x7fd684071e70 2024-05-08T23:32:44.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68919f700 1 --2- 172.21.15.81:0/841548333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd684071e70 0x7fd684151760 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd680000c00 tx=0x7fd680004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:44.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd679ffb700 1 -- 172.21.15.81:0/841548333 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd680004510 con 0x7fd684071e70 2024-05-08T23:32:44.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd679ffb700 1 -- 172.21.15.81:0/841548333 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68001e070 con 0x7fd684071e70 2024-05-08T23:32:44.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd679ffb700 1 -- 172.21.15.81:0/841548333 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6800038e0 con 0x7fd684071e70 2024-05-08T23:32:44.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68a1a1700 1 -- 172.21.15.81:0/841548333 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd684159480 con 0x7fd684071e70 2024-05-08T23:32:44.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68a1a1700 1 -- 172.21.15.81:0/841548333 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd68415a2d0 con 0x7fd684071e70 2024-05-08T23:32:44.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd679ffb700 1 -- 172.21.15.81:0/841548333 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd680003a40 con 0x7fd684071e70 2024-05-08T23:32:44.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.019+0000 7fd68a1a1700 1 -- 172.21.15.81:0/841548333 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6840684b0 con 0x7fd684071e70 2024-05-08T23:32:44.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.023+0000 7fd679ffb700 1 --2- 172.21.15.81:0/841548333 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd668066730 0x7fd668068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.023+0000 7fd679ffb700 1 -- 172.21.15.81:0/841548333 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd680084bf0 con 0x7fd684071e70 2024-05-08T23:32:44.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.023+0000 7fd68899e700 1 --2- 172.21.15.81:0/841548333 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd668066730 0x7fd668068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.023+0000 7fd68899e700 1 --2- 172.21.15.81:0/841548333 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd668066730 0x7fd668068bf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd6841637e0 tx=0x7fd674009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:44.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.027+0000 7fd679ffb700 1 -- 172.21.15.81:0/841548333 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd680052090 con 0x7fd684071e70 2024-05-08T23:32:44.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.231+0000 7fd68a1a1700 1 -- 172.21.15.81:0/841548333 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd684071c50 con 0x7fd684071e70 2024-05-08T23:32:44.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.235+0000 7fd679ffb700 1 -- 172.21.15.81:0/841548333 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4743 (secure 0 0 0) 0x7fd680015090 con 0x7fd684071e70 2024-05-08T23:32:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:44.236 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/2105335102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":2105335102},{"type":"v1","addr":"172.21.15.117:6834","nonce":2105335102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2900127785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2900127785},{"type":"v1","addr":"172.21.15.81:6835","nonce":2900127785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1899281779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1899281779},{"type":"v1","addr":"172.21.15.81:6837","nonce":1899281779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:32:40.852853+0000","modified":"2024-05-08T23:32:43.867414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":15311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_15311":{"gid":15311,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/3401411259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3401411259},{"type":"v1","addr":"172.21.15.117:6835","nonce":3401411259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:32:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.235+0000 7fd68a1a1700 1 -- 172.21.15.81:0/841548333 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd668066730 msgr2=0x7fd668068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.235+0000 7fd68a1a1700 1 --2- 172.21.15.81:0/841548333 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd668066730 0x7fd668068bf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd6841637e0 tx=0x7fd674009240 comp rx=0 tx=0).stop 2024-05-08T23:32:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.235+0000 7fd68a1a1700 1 -- 172.21.15.81:0/841548333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd684071e70 msgr2=0x7fd684151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.235+0000 7fd68a1a1700 1 --2- 172.21.15.81:0/841548333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd684071e70 0x7fd684151760 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd680000c00 tx=0x7fd680004770 comp rx=0 tx=0).stop 2024-05-08T23:32:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.235+0000 7fd68a1a1700 1 -- 172.21.15.81:0/841548333 shutdown_connections 2024-05-08T23:32:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.235+0000 7fd68a1a1700 1 --2- 172.21.15.81:0/841548333 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd668066730 0x7fd668068bf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.235+0000 7fd68a1a1700 1 --2- 172.21.15.81:0/841548333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd684165070 0x7fd684158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.235+0000 7fd68a1a1700 1 --2- 172.21.15.81:0/841548333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6840b3c20 0x7fd68414e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.235+0000 7fd68a1a1700 1 --2- 172.21.15.81:0/841548333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd684071e70 0x7fd684151760 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.235+0000 7fd68a1a1700 1 -- 172.21.15.81:0/841548333 >> 172.21.15.81:0/841548333 conn(0x7fd68406c330 msgr2=0x7fd684162ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:44.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.235+0000 7fd68a1a1700 1 -- 172.21.15.81:0/841548333 shutdown_connections 2024-05-08T23:32:44.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.235+0000 7fd68a1a1700 1 -- 172.21.15.81:0/841548333 wait complete. 2024-05-08T23:32:44.239 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 23 2024-05-08T23:32:44.249 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:44.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f6252c700 1 Processor -- start 2024-05-08T23:32:44.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f6252c700 1 -- start start 2024-05-08T23:32:44.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f6252c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3f5c0ce510 0x7f3f5c15a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f6252c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f5c15aaa0 0x7f3f5c13d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f6252c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5c0ce080 con 0x7f3f5c071ab0 2024-05-08T23:32:44.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f6252c700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5c0cd7c0 con 0x7f3f5c0ce510 2024-05-08T23:32:44.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f6252c700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5c0cda30 con 0x7f3f5c15aaa0 2024-05-08T23:32:44.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f60d29700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3f5c0ce510 0x7f3f5c15a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f60d29700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3f5c0ce510 0x7f3f5c15a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34342/0 (socket says 172.21.15.81:34342) 2024-05-08T23:32:44.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f60d29700 1 -- 172.21.15.81:0/243830971 learned_addr learned my addr 172.21.15.81:0/243830971 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:44.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f60d29700 1 -- 172.21.15.81:0/243830971 >> v2:172.21.15.117:3300/0 conn(0x7f3f5c15aaa0 msgr2=0x7f3f5c13d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:44.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f61d2b700 1 --2- 172.21.15.81:0/243830971 >> v2:172.21.15.117:3300/0 conn(0x7f3f5c15aaa0 0x7f3f5c13d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f60d29700 1 --2- 172.21.15.81:0/243830971 >> v2:172.21.15.117:3300/0 conn(0x7f3f5c15aaa0 0x7f3f5c13d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f60d29700 1 -- 172.21.15.81:0/243830971 >> v1:172.21.15.117:6789/0 conn(0x7f3f5c071ab0 legacy=0x7f3f5c071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:44.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f60d29700 1 -- 172.21.15.81:0/243830971 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c0cdd10 con 0x7f3f5c0ce510 2024-05-08T23:32:44.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.359+0000 7f3f61d2b700 1 --2- 172.21.15.81:0/243830971 >> v2:172.21.15.117:3300/0 conn(0x7f3f5c15aaa0 0x7f3f5c13d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:44.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.363+0000 7f3f60d29700 1 --2- 172.21.15.81:0/243830971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f5c0ce510 0x7f3f5c15a560 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c074590 tx=0x7f3f4c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49594869e417efc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:44.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.363+0000 7f3f53fff700 1 -- 172.21.15.81:0/243830971 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c004230 con 0x7f3f5c0ce510 2024-05-08T23:32:44.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.363+0000 7f3f53fff700 1 -- 172.21.15.81:0/243830971 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c004390 con 0x7f3f5c0ce510 2024-05-08T23:32:44.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.363+0000 7f3f53fff700 1 -- 172.21.15.81:0/243830971 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0036c0 con 0x7f3f5c0ce510 2024-05-08T23:32:44.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.363+0000 7f3f6252c700 1 -- 172.21.15.81:0/243830971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f5c0ce510 msgr2=0x7f3f5c15a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.363+0000 7f3f6252c700 1 --2- 172.21.15.81:0/243830971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f5c0ce510 0x7f3f5c15a560 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c074590 tx=0x7f3f4c009bf0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.363+0000 7f3f6252c700 1 -- 172.21.15.81:0/243830971 shutdown_connections 2024-05-08T23:32:44.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.363+0000 7f3f6252c700 1 --2- 172.21.15.81:0/243830971 >> v2:172.21.15.117:3300/0 conn(0x7f3f5c15aaa0 0x7f3f5c13d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.363+0000 7f3f6252c700 1 --2- 172.21.15.81:0/243830971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f5c0ce510 0x7f3f5c15a560 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.363+0000 7f3f6252c700 1 -- 172.21.15.81:0/243830971 >> 172.21.15.81:0/243830971 conn(0x7f3f5c06c330 msgr2=0x7f3f5c06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:44.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.363+0000 7f3f6252c700 1 -- 172.21.15.81:0/243830971 shutdown_connections 2024-05-08T23:32:44.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.363+0000 7f3f6252c700 1 -- 172.21.15.81:0/243830971 wait complete. 2024-05-08T23:32:44.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.363+0000 7f3f6252c700 1 Processor -- start 2024-05-08T23:32:44.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.363+0000 7f3f6252c700 1 -- start start 2024-05-08T23:32:44.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f6252c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f5c071ab0 0x7f3f5c1530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f6252c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f5c0ce510 0x7f3f5c139450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f6252c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5c15aaa0 0x7f3f5c135360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f6252c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c13acd0 con 0x7f3f5c0ce510 2024-05-08T23:32:44.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f6252c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c13b1d0 con 0x7f3f5c15aaa0 2024-05-08T23:32:44.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f6252c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c153960 con 0x7f3f5c071ab0 2024-05-08T23:32:44.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f60d29700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f5c0ce510 0x7f3f5c139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f60d29700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f5c0ce510 0x7f3f5c139450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34344/0 (socket says 172.21.15.81:34344) 2024-05-08T23:32:44.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f60d29700 1 -- 172.21.15.81:0/3967539571 learned_addr learned my addr 172.21.15.81:0/3967539571 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:44.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f61d2b700 1 --2- 172.21.15.81:0/3967539571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5c15aaa0 0x7f3f5c135360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f6152a700 1 --2- 172.21.15.81:0/3967539571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f5c071ab0 0x7f3f5c1530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f60d29700 1 -- 172.21.15.81:0/3967539571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f5c071ab0 msgr2=0x7f3f5c1530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f60d29700 1 --2- 172.21.15.81:0/3967539571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f5c071ab0 0x7f3f5c1530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f60d29700 1 -- 172.21.15.81:0/3967539571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5c15aaa0 msgr2=0x7f3f5c135360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f60d29700 1 --2- 172.21.15.81:0/3967539571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5c15aaa0 0x7f3f5c135360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f60d29700 1 -- 172.21.15.81:0/3967539571 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c009570 con 0x7f3f5c0ce510 2024-05-08T23:32:44.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f60d29700 1 --2- 172.21.15.81:0/3967539571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f5c0ce510 0x7f3f5c139450 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c004000 tx=0x7f3f4c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:44.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f51ffb700 1 -- 172.21.15.81:0/3967539571 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c00b860 con 0x7f3f5c0ce510 2024-05-08T23:32:44.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f51ffb700 1 -- 172.21.15.81:0/3967539571 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c00b9c0 con 0x7f3f5c0ce510 2024-05-08T23:32:44.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f6252c700 1 -- 172.21.15.81:0/3967539571 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5c153c40 con 0x7f3f5c0ce510 2024-05-08T23:32:44.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f51ffb700 1 -- 172.21.15.81:0/3967539571 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c00bd50 con 0x7f3f5c0ce510 2024-05-08T23:32:44.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f6252c700 1 -- 172.21.15.81:0/3967539571 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5c154200 con 0x7f3f5c0ce510 2024-05-08T23:32:44.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.367+0000 7f3f6252c700 1 -- 172.21.15.81:0/3967539571 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5c0684b0 con 0x7f3f5c0ce510 2024-05-08T23:32:44.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.371+0000 7f3f51ffb700 1 -- 172.21.15.81:0/3967539571 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3f4c021070 con 0x7f3f5c0ce510 2024-05-08T23:32:44.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.371+0000 7f3f51ffb700 1 --2- 172.21.15.81:0/3967539571 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f40066730 0x7f3f40068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.371+0000 7f3f51ffb700 1 -- 172.21.15.81:0/3967539571 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f4c085a50 con 0x7f3f5c0ce510 2024-05-08T23:32:44.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.371+0000 7f3f6152a700 1 --2- 172.21.15.81:0/3967539571 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f40066730 0x7f3f40068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.371+0000 7f3f6152a700 1 --2- 172.21.15.81:0/3967539571 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f40066730 0x7f3f40068bf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0b3d40 tx=0x7f3f580090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:44.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.379+0000 7f3f51ffb700 1 -- 172.21.15.81:0/3967539571 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3f4c052e50 con 0x7f3f5c0ce510 2024-05-08T23:32:44.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.551+0000 7f3f6252c700 1 -- 172.21.15.81:0/3967539571 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f5c0684b0 con 0x7f3f5c0ce510 2024-05-08T23:32:44.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.551+0000 7f3f51ffb700 1 -- 172.21.15.81:0/3967539571 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f3f4c056470 con 0x7f3f5c0ce510 2024-05-08T23:32:44.555 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:44.555 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":52,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:32:41.855853+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:39.161796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:39.853963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:44.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.555+0000 7f3f6252c700 1 -- 172.21.15.81:0/3967539571 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f40066730 msgr2=0x7f3f40068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.555+0000 7f3f6252c700 1 --2- 172.21.15.81:0/3967539571 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f40066730 0x7f3f40068bf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0b3d40 tx=0x7f3f580090e0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.555+0000 7f3f6252c700 1 -- 172.21.15.81:0/3967539571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f5c0ce510 msgr2=0x7f3f5c139450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.555+0000 7f3f6252c700 1 --2- 172.21.15.81:0/3967539571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f5c0ce510 0x7f3f5c139450 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c004000 tx=0x7f3f4c004770 comp rx=0 tx=0).stop 2024-05-08T23:32:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.555+0000 7f3f6252c700 1 -- 172.21.15.81:0/3967539571 shutdown_connections 2024-05-08T23:32:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.555+0000 7f3f6252c700 1 --2- 172.21.15.81:0/3967539571 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f40066730 0x7f3f40068bf0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.555+0000 7f3f6252c700 1 --2- 172.21.15.81:0/3967539571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5c15aaa0 0x7f3f5c135360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.555+0000 7f3f6252c700 1 --2- 172.21.15.81:0/3967539571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f5c0ce510 0x7f3f5c139450 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.555+0000 7f3f6252c700 1 --2- 172.21.15.81:0/3967539571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f5c071ab0 0x7f3f5c1530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.555+0000 7f3f6252c700 1 -- 172.21.15.81:0/3967539571 >> 172.21.15.81:0/3967539571 conn(0x7f3f5c06c330 msgr2=0x7f3f5c0662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.555+0000 7f3f6252c700 1 -- 172.21.15.81:0/3967539571 shutdown_connections 2024-05-08T23:32:44.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.555+0000 7f3f6252c700 1 -- 172.21.15.81:0/3967539571 wait complete. 2024-05-08T23:32:44.571 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T23:32:44.571 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:32:44.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.679+0000 7f8776684700 1 Processor -- start 2024-05-08T23:32:44.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.679+0000 7f8776684700 1 -- start start 2024-05-08T23:32:44.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8776684700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8770071ab0 0x7f87700ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8776684700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8770071e90 0x7f877015a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8776684700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87700cea50 con 0x7f877015aaa0 2024-05-08T23:32:44.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8776684700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87700ce080 con 0x7f8770071ab0 2024-05-08T23:32:44.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8776684700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87700cd7c0 con 0x7f8770071e90 2024-05-08T23:32:44.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8775682700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8770071ab0 0x7f87700ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8775682700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8770071ab0 0x7f87700ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34358/0 (socket says 172.21.15.81:34358) 2024-05-08T23:32:44.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8775682700 1 -- 172.21.15.81:0/806935758 learned_addr learned my addr 172.21.15.81:0/806935758 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:44.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8774e81700 1 --2- 172.21.15.81:0/806935758 >> v2:172.21.15.117:3300/0 conn(0x7f8770071e90 0x7f877015a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8775682700 1 -- 172.21.15.81:0/806935758 >> v2:172.21.15.117:3300/0 conn(0x7f8770071e90 msgr2=0x7f877015a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:44.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8775682700 1 --2- 172.21.15.81:0/806935758 >> v2:172.21.15.117:3300/0 conn(0x7f8770071e90 0x7f877015a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8775682700 1 -- 172.21.15.81:0/806935758 >> v1:172.21.15.117:6789/0 conn(0x7f877015aaa0 legacy=0x7f877013d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8775682700 1 -- 172.21.15.81:0/806935758 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87700cda30 con 0x7f8770071ab0 2024-05-08T23:32:44.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8775682700 1 --2- 172.21.15.81:0/806935758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770071ab0 0x7f87700ce510 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f877006bb20 tx=0x7f8764009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62462a7d5007ec2e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:44.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8763fff700 1 -- 172.21.15.81:0/806935758 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764011830 con 0x7f8770071ab0 2024-05-08T23:32:44.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8763fff700 1 -- 172.21.15.81:0/806935758 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8764011990 con 0x7f8770071ab0 2024-05-08T23:32:44.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8763fff700 1 -- 172.21.15.81:0/806935758 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764011d20 con 0x7f8770071ab0 2024-05-08T23:32:44.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8776684700 1 -- 172.21.15.81:0/806935758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770071ab0 msgr2=0x7f87700ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8776684700 1 --2- 172.21.15.81:0/806935758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770071ab0 0x7f87700ce510 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f877006bb20 tx=0x7f8764009bf0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8776684700 1 -- 172.21.15.81:0/806935758 shutdown_connections 2024-05-08T23:32:44.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8776684700 1 --2- 172.21.15.81:0/806935758 >> v2:172.21.15.117:3300/0 conn(0x7f8770071e90 0x7f877015a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8776684700 1 --2- 172.21.15.81:0/806935758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770071ab0 0x7f87700ce510 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8776684700 1 -- 172.21.15.81:0/806935758 >> 172.21.15.81:0/806935758 conn(0x7f877006c330 msgr2=0x7f877006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:44.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8776684700 1 -- 172.21.15.81:0/806935758 shutdown_connections 2024-05-08T23:32:44.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.683+0000 7f8776684700 1 -- 172.21.15.81:0/806935758 wait complete. 2024-05-08T23:32:44.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8776684700 1 Processor -- start 2024-05-08T23:32:44.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8776684700 1 -- start start 2024-05-08T23:32:44.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8776684700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8770071ab0 0x7f8770142750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8776684700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770071e90 0x7f877013a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8776684700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f877015aaa0 0x7f87701ef760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8776684700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87700b36a0 con 0x7f8770071e90 2024-05-08T23:32:44.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8776684700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8770142c90 con 0x7f877015aaa0 2024-05-08T23:32:44.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8776684700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87700b2a90 con 0x7f8770071ab0 2024-05-08T23:32:44.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8774e81700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770071e90 0x7f877013a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8774e81700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770071e90 0x7f877013a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34374/0 (socket says 172.21.15.81:34374) 2024-05-08T23:32:44.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8774e81700 1 -- 172.21.15.81:0/1939333893 learned_addr learned my addr 172.21.15.81:0/1939333893 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:44.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8775682700 1 --2- 172.21.15.81:0/1939333893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8770071ab0 0x7f8770142750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8775e83700 1 --2- 172.21.15.81:0/1939333893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f877015aaa0 0x7f87701ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8774e81700 1 -- 172.21.15.81:0/1939333893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8770071ab0 msgr2=0x7f8770142750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8774e81700 1 --2- 172.21.15.81:0/1939333893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8770071ab0 0x7f8770142750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8774e81700 1 -- 172.21.15.81:0/1939333893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f877015aaa0 msgr2=0x7f87701ef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8774e81700 1 --2- 172.21.15.81:0/1939333893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f877015aaa0 0x7f87701ef760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8774e81700 1 -- 172.21.15.81:0/1939333893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8764009570 con 0x7f8770071e90 2024-05-08T23:32:44.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8775e83700 1 --2- 172.21.15.81:0/1939333893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f877015aaa0 0x7f87701ef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:44.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8774e81700 1 --2- 172.21.15.81:0/1939333893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770071e90 0x7f877013a8c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f876c010e20 tx=0x7f876c00e640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:44.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8761ffb700 1 -- 172.21.15.81:0/1939333893 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c009740 con 0x7f8770071e90 2024-05-08T23:32:44.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8761ffb700 1 -- 172.21.15.81:0/1939333893 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c00ee60 con 0x7f8770071e90 2024-05-08T23:32:44.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8761ffb700 1 -- 172.21.15.81:0/1939333893 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c011670 con 0x7f8770071e90 2024-05-08T23:32:44.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8776684700 1 -- 172.21.15.81:0/1939333893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87700b2d70 con 0x7f8770071e90 2024-05-08T23:32:44.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.687+0000 7f8776684700 1 -- 172.21.15.81:0/1939333893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87700b3330 con 0x7f8770071e90 2024-05-08T23:32:44.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.691+0000 7f8761ffb700 1 -- 172.21.15.81:0/1939333893 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f876c0117d0 con 0x7f8770071e90 2024-05-08T23:32:44.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.691+0000 7f8776684700 1 -- 172.21.15.81:0/1939333893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87700b34f0 con 0x7f8770071e90 2024-05-08T23:32:44.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.691+0000 7f8761ffb700 1 --2- 172.21.15.81:0/1939333893 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8754066730 0x7f8754068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:44.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.691+0000 7f8761ffb700 1 -- 172.21.15.81:0/1939333893 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f876c086370 con 0x7f8770071e90 2024-05-08T23:32:44.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.691+0000 7f8775682700 1 --2- 172.21.15.81:0/1939333893 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8754066730 0x7f8754068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:44.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.691+0000 7f8775682700 1 --2- 172.21.15.81:0/1939333893 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8754066730 0x7f8754068bf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8764000c00 tx=0x7f8764008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:44.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.703+0000 7f8761ffb700 1 -- 172.21.15.81:0/1939333893 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f876c0536d0 con 0x7f8770071e90 2024-05-08T23:32:44.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.871+0000 7f8776684700 1 -- 172.21.15.81:0/1939333893 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f877015af30 con 0x7f8754066730 2024-05-08T23:32:44.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.883+0000 7f8761ffb700 1 -- 172.21.15.81:0/1939333893 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276327 (secure 0 0 0) 0x7f8764011b00 con 0x7f8754066730 2024-05-08T23:32:44.888 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:44.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.887+0000 7f8776684700 1 -- 172.21.15.81:0/1939333893 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8754066730 msgr2=0x7f8754068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.887+0000 7f8776684700 1 --2- 172.21.15.81:0/1939333893 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8754066730 0x7f8754068bf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8764000c00 tx=0x7f8764008030 comp rx=0 tx=0).stop 2024-05-08T23:32:44.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.887+0000 7f8776684700 1 -- 172.21.15.81:0/1939333893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770071e90 msgr2=0x7f877013a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:44.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.887+0000 7f8776684700 1 --2- 172.21.15.81:0/1939333893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770071e90 0x7f877013a8c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f876c010e20 tx=0x7f876c00e640 comp rx=0 tx=0).stop 2024-05-08T23:32:44.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.887+0000 7f8776684700 1 -- 172.21.15.81:0/1939333893 shutdown_connections 2024-05-08T23:32:44.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.887+0000 7f8776684700 1 --2- 172.21.15.81:0/1939333893 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8754066730 0x7f8754068bf0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.887+0000 7f8776684700 1 --2- 172.21.15.81:0/1939333893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f877015aaa0 0x7f87701ef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.887+0000 7f8776684700 1 --2- 172.21.15.81:0/1939333893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8770071e90 0x7f877013a8c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.887+0000 7f8776684700 1 --2- 172.21.15.81:0/1939333893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8770071ab0 0x7f8770142750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:44.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.887+0000 7f8776684700 1 -- 172.21.15.81:0/1939333893 >> 172.21.15.81:0/1939333893 conn(0x7f877006c330 msgr2=0x7f877013b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:44.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.887+0000 7f8776684700 1 -- 172.21.15.81:0/1939333893 shutdown_connections 2024-05-08T23:32:44.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:44.887+0000 7f8776684700 1 -- 172.21.15.81:0/1939333893 wait complete. 2024-05-08T23:32:44.892 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:32:44.904 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-05-08T23:32:43.367360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":44,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225900,"kb_used_data":5616,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754748820,"statfs":{"total":773094113280,"available":772862791680,"internally_reserved":0,"allocated":5750784,"data_stored":3039310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":225431096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000993"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.482923+0000","last_change":"2024-05-08T23:32:39.482923+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.482923+0000","last_undegraded":"2024-05-08T23:32:39.482923+0000","last_fullsized":"2024-05-08T23:32:39.482923+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:13.825466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.479952+0000","last_change":"2024-05-08T23:32:39.479952+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.479952+0000","last_undegraded":"2024-05-08T23:32:39.479952+0000","last_fullsized":"2024-05-08T23:32:39.479952+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:43.629558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.476540+0000","last_change":"2024-05-08T23:32:39.476540+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.476540+0000","last_undegraded":"2024-05-08T23:32:39.476540+0000","last_fullsized":"2024-05-08T23:32:39.476540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:39.770766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481883+0000","last_change":"2024-05-08T23:32:39.481883+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481883+0000","last_undegraded":"2024-05-08T23:32:39.481883+0000","last_fullsized":"2024-05-08T23:32:39.481883+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:12.523765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477601+0000","last_change":"2024-05-08T23:32:39.477601+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477601+0000","last_undegraded":"2024-05-08T23:32:39.477601+0000","last_fullsized":"2024-05-08T23:32:39.477601+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:41.416678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481005+0000","last_change":"2024-05-08T23:32:39.481005+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481005+0000","last_undegraded":"2024-05-08T23:32:39.481005+0000","last_fullsized":"2024-05-08T23:32:39.481005+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:14.674732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477408+0000","last_change":"2024-05-08T23:32:39.477408+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477408+0000","last_undegraded":"2024-05-08T23:32:39.477408+0000","last_fullsized":"2024-05-08T23:32:39.477408+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:31.833826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481770+0000","last_change":"2024-05-08T23:32:39.481770+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481770+0000","last_undegraded":"2024-05-08T23:32:39.481770+0000","last_fullsized":"2024-05-08T23:32:39.481770+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:24.848996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483474+0000","last_change":"2024-05-08T23:32:39.483474+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483474+0000","last_undegraded":"2024-05-08T23:32:39.483474+0000","last_fullsized":"2024-05-08T23:32:39.483474+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:31.286020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481540+0000","last_change":"2024-05-08T23:32:39.481540+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481540+0000","last_undegraded":"2024-05-08T23:32:39.481540+0000","last_fullsized":"2024-05-08T23:32:39.481540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:31.133500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483298+0000","last_change":"2024-05-08T23:32:39.483298+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483298+0000","last_undegraded":"2024-05-08T23:32:39.483298+0000","last_fullsized":"2024-05-08T23:32:39.483298+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:43:57.594449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481124+0000","last_change":"2024-05-08T23:32:39.481124+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481124+0000","last_undegraded":"2024-05-08T23:32:39.481124+0000","last_fullsized":"2024-05-08T23:32:39.481124+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:33:25.871051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483428+0000","last_change":"2024-05-08T23:32:39.483428+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483428+0000","last_undegraded":"2024-05-08T23:32:39.483428+0000","last_fullsized":"2024-05-08T23:32:39.483428+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:06.250791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477222+0000","last_change":"2024-05-08T23:32:39.477222+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477222+0000","last_undegraded":"2024-05-08T23:32:39.477222+0000","last_fullsized":"2024-05-08T23:32:39.477222+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:54.923508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.484968+0000","last_change":"2024-05-08T23:32:39.484968+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.484968+0000","last_undegraded":"2024-05-08T23:32:39.484968+0000","last_fullsized":"2024-05-08T23:32:39.484968+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:18:31.349703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.482871+0000","last_change":"2024-05-08T23:32:39.482871+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.482871+0000","last_undegraded":"2024-05-08T23:32:39.482871+0000","last_fullsized":"2024-05-08T23:32:39.482871+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:20:31.576039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481313+0000","last_change":"2024-05-08T23:32:39.481313+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481313+0000","last_undegraded":"2024-05-08T23:32:39.481313+0000","last_fullsized":"2024-05-08T23:32:39.481313+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:21.704760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483932+0000","last_change":"2024-05-08T23:32:39.483932+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483932+0000","last_undegraded":"2024-05-08T23:32:39.483932+0000","last_fullsized":"2024-05-08T23:32:39.483932+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:46.898315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.484322+0000","last_change":"2024-05-08T23:32:39.484322+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.484322+0000","last_undegraded":"2024-05-08T23:32:39.484322+0000","last_fullsized":"2024-05-08T23:32:39.484322+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477580+0000","last_change":"2024-05-08T23:32:39.477580+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477580+0000","last_undegraded":"2024-05-08T23:32:39.477580+0000","last_fullsized":"2024-05-08T23:32:39.477580+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:19.747747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481541+0000","last_change":"2024-05-08T23:32:39.481541+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481541+0000","last_undegraded":"2024-05-08T23:32:39.481541+0000","last_fullsized":"2024-05-08T23:32:39.481541+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:36.991964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":186,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T23:32:36.447910+0000","last_change":"2024-05-08T23:30:27.226484+0000","last_active":"2024-05-08T23:32:36.447910+0000","last_peered":"2024-05-08T23:32:36.447910+0000","last_clean":"2024-05-08T23:32:36.447910+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:32:36.447910+0000","last_undegraded":"2024-05-08T23:32:36.447910+0000","last_fullsized":"2024-05-08T23:32:36.447910+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_clean_scrub_stamp":"2024-05-08T23:30:26.204118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:14.267287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":42,"acting":42,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.89400000000000002,"15min":0.90500000000000003},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1719999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.96899999999999997,"15min":0.97899999999999998},"min":{"1min":0.58299999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1910000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.89700000000000002}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.59099999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6579999999999999,"5min":2.004,"15min":2.004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.92600000000000005,"15min":0.92400000000000004},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.327,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.68200000000000005}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88300000000000001,"15min":0.88500000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.95999999999999996,"15min":0.97099999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.218,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.78900000000000003}]},{"osd":3,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.89200000000000002,"15min":0.88400000000000001},"min":{"1min":0.65100000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96899999999999997,"15min":0.97799999999999998},"min":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2290000000000001,"5min":1.272,"15min":1.272},"last":0.85399999999999998}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.71499999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.1439999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0249999999999999,"15min":1.0269999999999999},"min":{"1min":0.67800000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.95299999999999996}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0469999999999999,"15min":1.056},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4450000000000001,"5min":2.109,"15min":2.109},"last":1.006},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1240000000000001,"15min":1.1439999999999999},"min":{"1min":0.75800000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.98099999999999998}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1970000000000001,"15min":1.2070000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5660000000000001,"5min":2.544,"15min":2.544},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.103,"15min":1.109},"min":{"1min":0.81200000000000006,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.0669999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2969999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.97099999999999997,"15min":0.98299999999999998},"min":{"1min":0.63700000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1859999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.65700000000000003}]},{"osd":1,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.90900000000000003,"15min":0.90400000000000003},"min":{"1min":0.67300000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88400000000000001,"15min":0.878},"min":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.0309999999999999}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.91800000000000004,"15min":0.91700000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.385,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.93700000000000006,"15min":0.92500000000000004},"min":{"1min":0.70299999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.454,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.93100000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.97199999999999998,"15min":0.98399999999999999},"min":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.252,"5min":1.395,"15min":1.395},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.004,"5min":0.96099999999999997,"15min":0.95399999999999996},"min":{"1min":0.63700000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.0049999999999999}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.995,"15min":1.0129999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.1339999999999999,"15min":1.1599999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.484,"5min":2.464,"15min":2.464},"last":0.78500000000000003}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0800000000000001,"15min":1.0820000000000001},"min":{"1min":0.82099999999999995,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.871}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1930000000000001,"15min":1.216},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.163,"5min":1.1759999999999999,"15min":1.1779999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.2909999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.84099999999999997,"15min":0.85099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2030000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77900000000000003,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.4910000000000001}]},{"osd":1,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.70999999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1459999999999999,"5min":1.268,"15min":1.268},"last":1.47},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.88800000000000001,"15min":0.90000000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.125,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.0920000000000001}]},{"osd":2,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.88800000000000001,"15min":0.89800000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89900000000000002,"15min":0.90700000000000003},"min":{"1min":0.65400000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0249999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.5669999999999999}]},{"osd":3,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.85299999999999998,"15min":0.84899999999999998},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1759999999999999,"5min":1.23,"15min":1.23},"last":1.3120000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2390000000000001,"5min":1.431,"15min":1.431},"last":1.357}]},{"osd":5,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3420000000000001,"5min":1.413,"15min":1.413},"last":1.7669999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95899999999999996,"15min":0.95099999999999996},"min":{"1min":0.73999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.278,"5min":1.504,"15min":1.504},"last":1.409}]},{"osd":6,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0269999999999999,"15min":1.0309999999999999},"min":{"1min":0.73899999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.52},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1419999999999999,"15min":1.1559999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3859999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.867}]},{"osd":7,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0800000000000001,"15min":1.0840000000000001},"min":{"1min":0.69899999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4379999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.653},{"interface":"front","average":{"1min":1.113,"5min":1.143,"15min":1.1479999999999999},"min":{"1min":0.94799999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4099999999999999,"5min":1.583,"15min":1.583},"last":1.7290000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.49099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.46300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":1.032,"15min":1.032},"last":0.67700000000000005}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70599999999999996,"15min":0.71099999999999997},"min":{"1min":0.50700000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.89300000000000002,"5min":1,"15min":1},"last":0.98899999999999999}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0469999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80200000000000005,"15min":0.79900000000000004},"min":{"1min":0.56699999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0229999999999999,"5min":1.242,"15min":1.242},"last":0.71499999999999997}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.65600000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.129,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.96699999999999997,"15min":0.97399999999999998},"min":{"1min":0.78000000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.244,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.749}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.014,"15min":1.022},"min":{"1min":0.80300000000000005,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.161,"5min":1.623,"15min":1.623},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.995,"15min":1.004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1359999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.89000000000000001}]},{"osd":6,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.997,"15min":1.0009999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.232,"5min":1.482,"15min":1.482},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.093,"15min":1.111},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3480000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.0209999999999999}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0800000000000001,"15min":1.0860000000000001},"min":{"1min":0.78400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.456,"5min":1.514,"15min":1.514},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0580000000000001,"15min":1.0600000000000001},"min":{"1min":0.75,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.2869999999999999,"5min":1.552,"15min":1.552},"last":0.96099999999999997}]}]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80400000000000005,"5min":1.099,"15min":1.099},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0660000000000001,"5min":1.129,"15min":1.129},"last":0.51700000000000002}]},{"osd":1,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70999999999999996,"15min":0.70899999999999996},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0509999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.84799999999999998}]},{"osd":4,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.96799999999999997,"15min":0.97799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.27,"5min":1.367,"15min":1.367},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.95599999999999996,"15min":0.96399999999999997},"min":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98999999999999999,"15min":0.98599999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.3},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.91600000000000004}]},{"osd":6,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0620000000000001,"15min":1.0740000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4239999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.22},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.075,"15min":1.075},"min":{"1min":0.85699999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.381}]},{"osd":7,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1220000000000001,"15min":1.1240000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1000000000000001,"15min":1.1020000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.492,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.0109999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.51300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.44,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.442}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.628},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.68899999999999995,"15min":0.67000000000000004},"min":{"1min":0.53900000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.39000000000000001}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73499999999999999,"15min":0.72599999999999998},"min":{"1min":0.57999999999999996,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.42799999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.038,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.95099999999999996,"15min":0.95199999999999996},"min":{"1min":0.67600000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.256,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":0.88},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.95499999999999996,"15min":0.94299999999999995},"min":{"1min":0.77300000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.2709999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.70499999999999996}]},{"osd":5,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0109999999999999,"15min":1.0089999999999999},"min":{"1min":0.72299999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.403,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.75},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.077,"15min":1.079},"min":{"1min":0.85499999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3879999999999999,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.81499999999999995}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0369999999999999,"15min":1.034},"min":{"1min":0.80000000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.4330000000000001,"5min":1.633,"15min":1.633},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.095,"5min":1.135,"15min":1.1419999999999999},"min":{"1min":0.80500000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.4750000000000001,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.91800000000000004}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.0209999999999999,"15min":1.0009999999999999},"min":{"1min":0.83099999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.371,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.081,"15min":1.0820000000000001},"min":{"1min":0.84999999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.306,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.80100000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542174,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1155072,"data_stored":818327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.88300000000000001,"15min":0.89700000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.117}]},{"osd":1,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.86399999999999999,"15min":0.86699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1379999999999999,"5min":1.359,"15min":1.359},"last":0.502},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91200000000000003,"15min":0.91600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.23,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.94899999999999995}]},{"osd":2,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88,"15min":0.873},"min":{"1min":0.57499999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88,"15min":0.88500000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.133,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.68700000000000006}]},{"osd":3,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.91300000000000003,"15min":0.91100000000000003},"min":{"1min":0.60699999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.99299999999999999,"15min":0.99099999999999999},"min":{"1min":0.80100000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2470000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":1.0169999999999999}]},{"osd":4,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.90400000000000003,"15min":0.91200000000000003},"min":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.73999999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.1899999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.028,"15min":1.028},"min":{"1min":0.64100000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.405,"5min":1.714,"15min":1.714},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.1950000000000001,"5min":1.325,"15min":1.325},"last":0.90700000000000003}]},{"osd":6,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.048,"15min":1.048},"min":{"1min":0.86599999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.4430000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.105,"15min":1.107},"min":{"1min":0.83999999999999997,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.359,"5min":1.496,"15min":1.496},"last":0.82399999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28164,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343676,"statfs":{"total":96636764160,"available":96607924224,"internally_reserved":0,"allocated":1155072,"data_stored":818327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.47499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":1.351},"last":0.71799999999999997}]},{"osd":2,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.51500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.238,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.546,"5min":1.869,"15min":1.869},"last":0.80100000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.84199999999999997,"15min":0.85599999999999998},"min":{"1min":0.55200000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.94199999999999995,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.89300000000000002}]},{"osd":4,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97199999999999998,"15min":0.97299999999999998},"min":{"1min":0.71999999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.357,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.90800000000000003,"15min":0.89600000000000002},"min":{"1min":0.69899999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.746}]},{"osd":5,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.052,"15min":1.0600000000000001},"min":{"1min":0.76800000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3819999999999999,"5min":1.474,"15min":1.474},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.92500000000000004,"15min":0.91400000000000003},"min":{"1min":0.72699999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.98099999999999998}]},{"osd":6,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.056,"15min":1.0720000000000001},"min":{"1min":0.76900000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.171,"5min":1.736,"15min":1.736},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1180000000000001,"15min":1.121},"min":{"1min":0.89000000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4830000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.0009999999999999}]},{"osd":7,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.113,"15min":1.117},"min":{"1min":0.66600000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.397,"5min":1.72,"15min":1.72},"last":1.115},{"interface":"front","average":{"1min":1.097,"5min":1.109,"15min":1.111},"min":{"1min":0.86399999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.1439999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:32:44.909 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:32:45.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.019+0000 7f556759e700 1 Processor -- start 2024-05-08T23:32:45.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.019+0000 7f556759e700 1 -- start start 2024-05-08T23:32:45.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556815ac70 0x7f556813d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556759e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55680ce080 con 0x7f5568071ab0 2024-05-08T23:32:45.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556759e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55680cd7c0 con 0x7f55680ce510 2024-05-08T23:32:45.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55680cda30 con 0x7f556815ac70 2024-05-08T23:32:45.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556659c700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5568071ab0 0x7f5568071e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35326/0 (socket says 172.21.15.81:35326) 2024-05-08T23:32:45.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556659c700 1 -- 172.21.15.81:0/1151667796 learned_addr learned my addr 172.21.15.81:0/1151667796 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:45.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f5566d9d700 1 --2- 172.21.15.81:0/1151667796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556815ac70 0x7f556813d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556559a700 1 -- 172.21.15.81:0/1151667796 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695083847 0 0) 0x7f55680ce080 con 0x7f5568071ab0 2024-05-08T23:32:45.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556559a700 1 -- 172.21.15.81:0/1151667796 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55540035d0 con 0x7f5568071ab0 2024-05-08T23:32:45.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556559a700 1 -- 172.21.15.81:0/1151667796 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515645492 0 0) 0x7f55680cd7c0 con 0x7f55680ce510 2024-05-08T23:32:45.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556559a700 1 -- 172.21.15.81:0/1151667796 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55680ce080 con 0x7f55680ce510 2024-05-08T23:32:45.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f5566d9d700 1 -- 172.21.15.81:0/1151667796 >> v1:172.21.15.117:6789/0 conn(0x7f55680ce510 legacy=0x7f556815a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f5566d9d700 1 -- 172.21.15.81:0/1151667796 >> v1:172.21.15.81:6789/0 conn(0x7f5568071ab0 legacy=0x7f5568071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f5566d9d700 1 -- 172.21.15.81:0/1151667796 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55680cdd10 con 0x7f556815ac70 2024-05-08T23:32:45.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f5566d9d700 1 --2- 172.21.15.81:0/1151667796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556815ac70 0x7f556813d500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5560009880 tx=0x7f5560009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3716407bcf6ed3a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:45.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556559a700 1 -- 172.21.15.81:0/1151667796 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560011830 con 0x7f556815ac70 2024-05-08T23:32:45.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556559a700 1 -- 172.21.15.81:0/1151667796 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5560011990 con 0x7f556815ac70 2024-05-08T23:32:45.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556559a700 1 -- 172.21.15.81:0/1151667796 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560011d00 con 0x7f556815ac70 2024-05-08T23:32:45.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556759e700 1 -- 172.21.15.81:0/1151667796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556815ac70 msgr2=0x7f556813d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556759e700 1 --2- 172.21.15.81:0/1151667796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556815ac70 0x7f556813d500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5560009880 tx=0x7f5560009b90 comp rx=0 tx=0).stop 2024-05-08T23:32:45.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556759e700 1 -- 172.21.15.81:0/1151667796 shutdown_connections 2024-05-08T23:32:45.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556759e700 1 --2- 172.21.15.81:0/1151667796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556815ac70 0x7f556813d500 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556759e700 1 -- 172.21.15.81:0/1151667796 >> 172.21.15.81:0/1151667796 conn(0x7f556806c330 msgr2=0x7f556806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:45.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556759e700 1 -- 172.21.15.81:0/1151667796 shutdown_connections 2024-05-08T23:32:45.028 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.023+0000 7f556759e700 1 -- 172.21.15.81:0/1151667796 wait complete. 2024-05-08T23:32:45.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.027+0000 7f556759e700 1 Processor -- start 2024-05-08T23:32:45.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.027+0000 7f556759e700 1 -- start start 2024-05-08T23:32:45.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.027+0000 7f556759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5568071ab0 0x7f55680b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.027+0000 7f556759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55680ce510 0x7f55681edc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.027+0000 7f556759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f556815ac70 0x7f556813a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.027+0000 7f556759e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55681f2930 con 0x7f556815ac70 2024-05-08T23:32:45.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.027+0000 7f556759e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55681f1db0 con 0x7f55680ce510 2024-05-08T23:32:45.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.027+0000 7f556759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55681f2090 con 0x7f5568071ab0 2024-05-08T23:32:45.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f5566d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f556815ac70 0x7f556813a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f5566d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f556815ac70 0x7f556813a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34386/0 (socket says 172.21.15.81:34386) 2024-05-08T23:32:45.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f5566d9d700 1 -- 172.21.15.81:0/3750720331 learned_addr learned my addr 172.21.15.81:0/3750720331 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:45.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f5565d9b700 1 --2- 172.21.15.81:0/3750720331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55680ce510 0x7f55681edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f556659c700 1 --2- 172.21.15.81:0/3750720331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5568071ab0 0x7f55680b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f5566d9d700 1 -- 172.21.15.81:0/3750720331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5568071ab0 msgr2=0x7f55680b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f5566d9d700 1 --2- 172.21.15.81:0/3750720331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5568071ab0 0x7f55680b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f5566d9d700 1 -- 172.21.15.81:0/3750720331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55680ce510 msgr2=0x7f55681edc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f5566d9d700 1 --2- 172.21.15.81:0/3750720331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55680ce510 0x7f55681edc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f5566d9d700 1 -- 172.21.15.81:0/3750720331 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55600094b0 con 0x7f556815ac70 2024-05-08T23:32:45.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f5565d9b700 1 --2- 172.21.15.81:0/3750720331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55680ce510 0x7f55681edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:45.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f5566d9d700 1 --2- 172.21.15.81:0/3750720331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f556815ac70 0x7f556813a740 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5560013040 tx=0x7f5560004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:45.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f555effd700 1 -- 172.21.15.81:0/3750720331 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560004550 con 0x7f556815ac70 2024-05-08T23:32:45.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f555effd700 1 -- 172.21.15.81:0/3750720331 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55600036c0 con 0x7f556815ac70 2024-05-08T23:32:45.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f556759e700 1 -- 172.21.15.81:0/3750720331 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55681f2370 con 0x7f556815ac70 2024-05-08T23:32:45.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f555effd700 1 -- 172.21.15.81:0/3750720331 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560003a70 con 0x7f556815ac70 2024-05-08T23:32:45.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f556759e700 1 -- 172.21.15.81:0/3750720331 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55681f1a10 con 0x7f556815ac70 2024-05-08T23:32:45.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f556759e700 1 -- 172.21.15.81:0/3750720331 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55681406f0 con 0x7f556815ac70 2024-05-08T23:32:45.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.031+0000 7f555effd700 1 -- 172.21.15.81:0/3750720331 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5560003bd0 con 0x7f556815ac70 2024-05-08T23:32:45.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.035+0000 7f555effd700 1 --2- 172.21.15.81:0/3750720331 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5548066760 0x7f5548068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.035+0000 7f555effd700 1 -- 172.21.15.81:0/3750720331 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5560085680 con 0x7f556815ac70 2024-05-08T23:32:45.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.035+0000 7f556659c700 1 --2- 172.21.15.81:0/3750720331 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5548066760 0x7f5548068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.035+0000 7f556659c700 1 --2- 172.21.15.81:0/3750720331 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5548066760 0x7f5548068c20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5558000f60 tx=0x7f5558009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:45.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.039+0000 7f555effd700 1 -- 172.21.15.81:0/3750720331 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f55681406f0 con 0x7f556815ac70 2024-05-08T23:32:45.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.207+0000 7f556759e700 1 -- 172.21.15.81:0/3750720331 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f556815b2c0 con 0x7f5548066760 2024-05-08T23:32:45.214 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.211+0000 7f555effd700 1 -- 172.21.15.81:0/3750720331 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276327 (secure 0 0 0) 0x7f55580014a0 con 0x7f5548066760 2024-05-08T23:32:45.219 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:45.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.219+0000 7f556759e700 1 -- 172.21.15.81:0/3750720331 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5548066760 msgr2=0x7f5548068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.219+0000 7f556759e700 1 --2- 172.21.15.81:0/3750720331 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5548066760 0x7f5548068c20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5558000f60 tx=0x7f5558009040 comp rx=0 tx=0).stop 2024-05-08T23:32:45.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.219+0000 7f556759e700 1 -- 172.21.15.81:0/3750720331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f556815ac70 msgr2=0x7f556813a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.219+0000 7f556759e700 1 --2- 172.21.15.81:0/3750720331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f556815ac70 0x7f556813a740 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5560013040 tx=0x7f5560004770 comp rx=0 tx=0).stop 2024-05-08T23:32:45.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.219+0000 7f556759e700 1 -- 172.21.15.81:0/3750720331 shutdown_connections 2024-05-08T23:32:45.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.219+0000 7f556759e700 1 --2- 172.21.15.81:0/3750720331 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5548066760 0x7f5548068c20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.219+0000 7f556759e700 1 --2- 172.21.15.81:0/3750720331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f556815ac70 0x7f556813a740 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.219+0000 7f556759e700 1 --2- 172.21.15.81:0/3750720331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55680ce510 0x7f55681edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.219+0000 7f556759e700 1 --2- 172.21.15.81:0/3750720331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5568071ab0 0x7f55680b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.219+0000 7f556759e700 1 -- 172.21.15.81:0/3750720331 >> 172.21.15.81:0/3750720331 conn(0x7f556806c330 msgr2=0x7f5568159ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:45.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.219+0000 7f556759e700 1 -- 172.21.15.81:0/3750720331 shutdown_connections 2024-05-08T23:32:45.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.219+0000 7f556759e700 1 -- 172.21.15.81:0/3750720331 wait complete. 2024-05-08T23:32:45.222 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:32:45.233 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-05-08T23:32:43.367360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":44,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225900,"kb_used_data":5616,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754748820,"statfs":{"total":773094113280,"available":772862791680,"internally_reserved":0,"allocated":5750784,"data_stored":3039310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":225431096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000993"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.482923+0000","last_change":"2024-05-08T23:32:39.482923+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.482923+0000","last_undegraded":"2024-05-08T23:32:39.482923+0000","last_fullsized":"2024-05-08T23:32:39.482923+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:13.825466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.479952+0000","last_change":"2024-05-08T23:32:39.479952+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.479952+0000","last_undegraded":"2024-05-08T23:32:39.479952+0000","last_fullsized":"2024-05-08T23:32:39.479952+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:43.629558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.476540+0000","last_change":"2024-05-08T23:32:39.476540+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.476540+0000","last_undegraded":"2024-05-08T23:32:39.476540+0000","last_fullsized":"2024-05-08T23:32:39.476540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:39.770766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481883+0000","last_change":"2024-05-08T23:32:39.481883+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481883+0000","last_undegraded":"2024-05-08T23:32:39.481883+0000","last_fullsized":"2024-05-08T23:32:39.481883+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:12.523765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477601+0000","last_change":"2024-05-08T23:32:39.477601+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477601+0000","last_undegraded":"2024-05-08T23:32:39.477601+0000","last_fullsized":"2024-05-08T23:32:39.477601+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:41.416678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481005+0000","last_change":"2024-05-08T23:32:39.481005+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481005+0000","last_undegraded":"2024-05-08T23:32:39.481005+0000","last_fullsized":"2024-05-08T23:32:39.481005+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:14.674732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477408+0000","last_change":"2024-05-08T23:32:39.477408+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477408+0000","last_undegraded":"2024-05-08T23:32:39.477408+0000","last_fullsized":"2024-05-08T23:32:39.477408+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:31.833826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481770+0000","last_change":"2024-05-08T23:32:39.481770+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481770+0000","last_undegraded":"2024-05-08T23:32:39.481770+0000","last_fullsized":"2024-05-08T23:32:39.481770+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:24.848996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483474+0000","last_change":"2024-05-08T23:32:39.483474+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483474+0000","last_undegraded":"2024-05-08T23:32:39.483474+0000","last_fullsized":"2024-05-08T23:32:39.483474+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:31.286020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481540+0000","last_change":"2024-05-08T23:32:39.481540+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481540+0000","last_undegraded":"2024-05-08T23:32:39.481540+0000","last_fullsized":"2024-05-08T23:32:39.481540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:31.133500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483298+0000","last_change":"2024-05-08T23:32:39.483298+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483298+0000","last_undegraded":"2024-05-08T23:32:39.483298+0000","last_fullsized":"2024-05-08T23:32:39.483298+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:43:57.594449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481124+0000","last_change":"2024-05-08T23:32:39.481124+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481124+0000","last_undegraded":"2024-05-08T23:32:39.481124+0000","last_fullsized":"2024-05-08T23:32:39.481124+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:33:25.871051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483428+0000","last_change":"2024-05-08T23:32:39.483428+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483428+0000","last_undegraded":"2024-05-08T23:32:39.483428+0000","last_fullsized":"2024-05-08T23:32:39.483428+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:06.250791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477222+0000","last_change":"2024-05-08T23:32:39.477222+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477222+0000","last_undegraded":"2024-05-08T23:32:39.477222+0000","last_fullsized":"2024-05-08T23:32:39.477222+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:54.923508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.484968+0000","last_change":"2024-05-08T23:32:39.484968+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.484968+0000","last_undegraded":"2024-05-08T23:32:39.484968+0000","last_fullsized":"2024-05-08T23:32:39.484968+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:18:31.349703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.482871+0000","last_change":"2024-05-08T23:32:39.482871+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.482871+0000","last_undegraded":"2024-05-08T23:32:39.482871+0000","last_fullsized":"2024-05-08T23:32:39.482871+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:20:31.576039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481313+0000","last_change":"2024-05-08T23:32:39.481313+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481313+0000","last_undegraded":"2024-05-08T23:32:39.481313+0000","last_fullsized":"2024-05-08T23:32:39.481313+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:21.704760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483932+0000","last_change":"2024-05-08T23:32:39.483932+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483932+0000","last_undegraded":"2024-05-08T23:32:39.483932+0000","last_fullsized":"2024-05-08T23:32:39.483932+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:46.898315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.484322+0000","last_change":"2024-05-08T23:32:39.484322+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.484322+0000","last_undegraded":"2024-05-08T23:32:39.484322+0000","last_fullsized":"2024-05-08T23:32:39.484322+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477580+0000","last_change":"2024-05-08T23:32:39.477580+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477580+0000","last_undegraded":"2024-05-08T23:32:39.477580+0000","last_fullsized":"2024-05-08T23:32:39.477580+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:19.747747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481541+0000","last_change":"2024-05-08T23:32:39.481541+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481541+0000","last_undegraded":"2024-05-08T23:32:39.481541+0000","last_fullsized":"2024-05-08T23:32:39.481541+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:36.991964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:39.461956+0000","last_change":"2024-05-08T23:32:39.461956+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.461956+0000","last_undegraded":"2024-05-08T23:32:39.461956+0000","last_fullsized":"2024-05-08T23:32:39.461956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":186,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T23:32:36.447910+0000","last_change":"2024-05-08T23:30:27.226484+0000","last_active":"2024-05-08T23:32:36.447910+0000","last_peered":"2024-05-08T23:32:36.447910+0000","last_clean":"2024-05-08T23:32:36.447910+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:32:36.447910+0000","last_undegraded":"2024-05-08T23:32:36.447910+0000","last_fullsized":"2024-05-08T23:32:36.447910+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_clean_scrub_stamp":"2024-05-08T23:30:26.204118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:14.267287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":42,"acting":42,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.89400000000000002,"15min":0.90500000000000003},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1719999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.96899999999999997,"15min":0.97899999999999998},"min":{"1min":0.58299999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1910000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.89700000000000002}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.59099999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6579999999999999,"5min":2.004,"15min":2.004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.92600000000000005,"15min":0.92400000000000004},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.327,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.68200000000000005}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88300000000000001,"15min":0.88500000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.95999999999999996,"15min":0.97099999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.218,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.78900000000000003}]},{"osd":3,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.89200000000000002,"15min":0.88400000000000001},"min":{"1min":0.65100000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96899999999999997,"15min":0.97799999999999998},"min":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2290000000000001,"5min":1.272,"15min":1.272},"last":0.85399999999999998}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.71499999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.1439999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0249999999999999,"15min":1.0269999999999999},"min":{"1min":0.67800000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.95299999999999996}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0469999999999999,"15min":1.056},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4450000000000001,"5min":2.109,"15min":2.109},"last":1.006},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1240000000000001,"15min":1.1439999999999999},"min":{"1min":0.75800000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.98099999999999998}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1970000000000001,"15min":1.2070000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5660000000000001,"5min":2.544,"15min":2.544},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.103,"15min":1.109},"min":{"1min":0.81200000000000006,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.0669999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2969999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.97099999999999997,"15min":0.98299999999999998},"min":{"1min":0.63700000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1859999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.65700000000000003}]},{"osd":1,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.90900000000000003,"15min":0.90400000000000003},"min":{"1min":0.67300000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88400000000000001,"15min":0.878},"min":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.0309999999999999}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.91800000000000004,"15min":0.91700000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.385,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.93700000000000006,"15min":0.92500000000000004},"min":{"1min":0.70299999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.454,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.93100000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.97199999999999998,"15min":0.98399999999999999},"min":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.252,"5min":1.395,"15min":1.395},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.004,"5min":0.96099999999999997,"15min":0.95399999999999996},"min":{"1min":0.63700000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.0049999999999999}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.995,"15min":1.0129999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.1339999999999999,"15min":1.1599999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.484,"5min":2.464,"15min":2.464},"last":0.78500000000000003}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0800000000000001,"15min":1.0820000000000001},"min":{"1min":0.82099999999999995,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.871}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1930000000000001,"15min":1.216},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.163,"5min":1.1759999999999999,"15min":1.1779999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.2909999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.84099999999999997,"15min":0.85099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2030000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77900000000000003,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.4910000000000001}]},{"osd":1,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.70999999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1459999999999999,"5min":1.268,"15min":1.268},"last":1.47},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.88800000000000001,"15min":0.90000000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.125,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.0920000000000001}]},{"osd":2,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.88800000000000001,"15min":0.89800000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89900000000000002,"15min":0.90700000000000003},"min":{"1min":0.65400000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0249999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.5669999999999999}]},{"osd":3,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.85299999999999998,"15min":0.84899999999999998},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1759999999999999,"5min":1.23,"15min":1.23},"last":1.3120000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2390000000000001,"5min":1.431,"15min":1.431},"last":1.357}]},{"osd":5,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3420000000000001,"5min":1.413,"15min":1.413},"last":1.7669999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95899999999999996,"15min":0.95099999999999996},"min":{"1min":0.73999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.278,"5min":1.504,"15min":1.504},"last":1.409}]},{"osd":6,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0269999999999999,"15min":1.0309999999999999},"min":{"1min":0.73899999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.52},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1419999999999999,"15min":1.1559999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3859999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.867}]},{"osd":7,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0800000000000001,"15min":1.0840000000000001},"min":{"1min":0.69899999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4379999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.653},{"interface":"front","average":{"1min":1.113,"5min":1.143,"15min":1.1479999999999999},"min":{"1min":0.94799999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4099999999999999,"5min":1.583,"15min":1.583},"last":1.7290000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.49099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.46300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":1.032,"15min":1.032},"last":0.67700000000000005}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70599999999999996,"15min":0.71099999999999997},"min":{"1min":0.50700000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.89300000000000002,"5min":1,"15min":1},"last":0.98899999999999999}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0469999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80200000000000005,"15min":0.79900000000000004},"min":{"1min":0.56699999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0229999999999999,"5min":1.242,"15min":1.242},"last":0.71499999999999997}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.65600000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.129,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.96699999999999997,"15min":0.97399999999999998},"min":{"1min":0.78000000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.244,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.749}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.014,"15min":1.022},"min":{"1min":0.80300000000000005,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.161,"5min":1.623,"15min":1.623},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.995,"15min":1.004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1359999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.89000000000000001}]},{"osd":6,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.997,"15min":1.0009999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.232,"5min":1.482,"15min":1.482},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.093,"15min":1.111},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3480000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.0209999999999999}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0800000000000001,"15min":1.0860000000000001},"min":{"1min":0.78400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.456,"5min":1.514,"15min":1.514},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0580000000000001,"15min":1.0600000000000001},"min":{"1min":0.75,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.2869999999999999,"5min":1.552,"15min":1.552},"last":0.96099999999999997}]}]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80400000000000005,"5min":1.099,"15min":1.099},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0660000000000001,"5min":1.129,"15min":1.129},"last":0.51700000000000002}]},{"osd":1,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70999999999999996,"15min":0.70899999999999996},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0509999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.84799999999999998}]},{"osd":4,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.96799999999999997,"15min":0.97799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.27,"5min":1.367,"15min":1.367},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.95599999999999996,"15min":0.96399999999999997},"min":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98999999999999999,"15min":0.98599999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.3},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.91600000000000004}]},{"osd":6,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0620000000000001,"15min":1.0740000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4239999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.22},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.075,"15min":1.075},"min":{"1min":0.85699999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.381}]},{"osd":7,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1220000000000001,"15min":1.1240000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1000000000000001,"15min":1.1020000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.492,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.0109999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.51300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.44,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.442}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.628},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.68899999999999995,"15min":0.67000000000000004},"min":{"1min":0.53900000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.39000000000000001}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73499999999999999,"15min":0.72599999999999998},"min":{"1min":0.57999999999999996,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.42799999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.038,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.95099999999999996,"15min":0.95199999999999996},"min":{"1min":0.67600000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.256,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":0.88},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.95499999999999996,"15min":0.94299999999999995},"min":{"1min":0.77300000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.2709999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.70499999999999996}]},{"osd":5,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0109999999999999,"15min":1.0089999999999999},"min":{"1min":0.72299999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.403,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.75},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.077,"15min":1.079},"min":{"1min":0.85499999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3879999999999999,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.81499999999999995}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0369999999999999,"15min":1.034},"min":{"1min":0.80000000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.4330000000000001,"5min":1.633,"15min":1.633},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.095,"5min":1.135,"15min":1.1419999999999999},"min":{"1min":0.80500000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.4750000000000001,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.91800000000000004}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.0209999999999999,"15min":1.0009999999999999},"min":{"1min":0.83099999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.371,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.081,"15min":1.0820000000000001},"min":{"1min":0.84999999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.306,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.80100000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542174,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1155072,"data_stored":818327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.88300000000000001,"15min":0.89700000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.117}]},{"osd":1,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.86399999999999999,"15min":0.86699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1379999999999999,"5min":1.359,"15min":1.359},"last":0.502},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91200000000000003,"15min":0.91600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.23,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.94899999999999995}]},{"osd":2,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88,"15min":0.873},"min":{"1min":0.57499999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88,"15min":0.88500000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.133,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.68700000000000006}]},{"osd":3,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.91300000000000003,"15min":0.91100000000000003},"min":{"1min":0.60699999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.99299999999999999,"15min":0.99099999999999999},"min":{"1min":0.80100000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2470000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":1.0169999999999999}]},{"osd":4,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.90400000000000003,"15min":0.91200000000000003},"min":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.73999999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.1899999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.028,"15min":1.028},"min":{"1min":0.64100000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.405,"5min":1.714,"15min":1.714},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.1950000000000001,"5min":1.325,"15min":1.325},"last":0.90700000000000003}]},{"osd":6,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.048,"15min":1.048},"min":{"1min":0.86599999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.4430000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.105,"15min":1.107},"min":{"1min":0.83999999999999997,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.359,"5min":1.496,"15min":1.496},"last":0.82399999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28164,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343676,"statfs":{"total":96636764160,"available":96607924224,"internally_reserved":0,"allocated":1155072,"data_stored":818327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.47499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":1.351},"last":0.71799999999999997}]},{"osd":2,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.51500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.238,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.546,"5min":1.869,"15min":1.869},"last":0.80100000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.84199999999999997,"15min":0.85599999999999998},"min":{"1min":0.55200000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.94199999999999995,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.89300000000000002}]},{"osd":4,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97199999999999998,"15min":0.97299999999999998},"min":{"1min":0.71999999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.357,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.90800000000000003,"15min":0.89600000000000002},"min":{"1min":0.69899999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.746}]},{"osd":5,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.052,"15min":1.0600000000000001},"min":{"1min":0.76800000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3819999999999999,"5min":1.474,"15min":1.474},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.92500000000000004,"15min":0.91400000000000003},"min":{"1min":0.72699999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.98099999999999998}]},{"osd":6,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.056,"15min":1.0720000000000001},"min":{"1min":0.76900000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.171,"5min":1.736,"15min":1.736},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1180000000000001,"15min":1.121},"min":{"1min":0.89000000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4830000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.0009999999999999}]},{"osd":7,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.113,"15min":1.117},"min":{"1min":0.66600000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.397,"5min":1.72,"15min":1.72},"last":1.115},{"interface":"front","average":{"1min":1.097,"5min":1.109,"15min":1.111},"min":{"1min":0.86399999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.1439999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:32:45.237 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:32:45.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8e0a94700 1 Processor -- start 2024-05-08T23:32:45.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8e0a94700 1 -- start start 2024-05-08T23:32:45.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8e0a94700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc0ce510 0x7fe8dc15a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8e0a94700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe8dc15aaa0 0x7fe8dc13d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8e0a94700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8dc0ce080 con 0x7fe8dc071ab0 2024-05-08T23:32:45.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8e0a94700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc0cd7c0 con 0x7fe8dc15aaa0 2024-05-08T23:32:45.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8e0a94700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0cda30 con 0x7fe8dc0ce510 2024-05-08T23:32:45.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8daffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc0ce510 0x7fe8dc15a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8dbfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe8dc15aaa0 0x7fe8dc13d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8daffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc0ce510 0x7fe8dc15a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:50038/0 (socket says 172.21.15.81:50038) 2024-05-08T23:32:45.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8daffd700 1 -- 172.21.15.81:0/994574684 learned_addr learned my addr 172.21.15.81:0/994574684 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:45.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8db7fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe8dc071ab0 0x7fe8dc071e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:46982/0 (socket says 172.21.15.81:46982) 2024-05-08T23:32:45.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8daffd700 1 -- 172.21.15.81:0/994574684 >> v2:172.21.15.117:3300/0 conn(0x7fe8dc15aaa0 msgr2=0x7fe8dc13d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:45.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8daffd700 1 --2- 172.21.15.81:0/994574684 >> v2:172.21.15.117:3300/0 conn(0x7fe8dc15aaa0 0x7fe8dc13d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8daffd700 1 -- 172.21.15.81:0/994574684 >> v1:172.21.15.117:6789/0 conn(0x7fe8dc071ab0 legacy=0x7fe8dc071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.351+0000 7fe8daffd700 1 -- 172.21.15.81:0/994574684 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc0cdd10 con 0x7fe8dc0ce510 2024-05-08T23:32:45.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.355+0000 7fe8daffd700 1 --2- 172.21.15.81:0/994574684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc0ce510 0x7fe8dc15a560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009910 tx=0x7fe8d0009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aefb4251a793a0bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:45.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.355+0000 7fe8da7fc700 1 -- 172.21.15.81:0/994574684 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0011830 con 0x7fe8dc0ce510 2024-05-08T23:32:45.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.355+0000 7fe8da7fc700 1 -- 172.21.15.81:0/994574684 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d0011990 con 0x7fe8dc0ce510 2024-05-08T23:32:45.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.355+0000 7fe8da7fc700 1 -- 172.21.15.81:0/994574684 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0011d40 con 0x7fe8dc0ce510 2024-05-08T23:32:45.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.355+0000 7fe8e0a94700 1 -- 172.21.15.81:0/994574684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc0ce510 msgr2=0x7fe8dc15a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.355+0000 7fe8e0a94700 1 --2- 172.21.15.81:0/994574684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc0ce510 0x7fe8dc15a560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009910 tx=0x7fe8d0009c20 comp rx=0 tx=0).stop 2024-05-08T23:32:45.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.355+0000 7fe8e0a94700 1 -- 172.21.15.81:0/994574684 shutdown_connections 2024-05-08T23:32:45.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.355+0000 7fe8e0a94700 1 --2- 172.21.15.81:0/994574684 >> v2:172.21.15.117:3300/0 conn(0x7fe8dc15aaa0 0x7fe8dc13d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.355+0000 7fe8e0a94700 1 --2- 172.21.15.81:0/994574684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc0ce510 0x7fe8dc15a560 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.355+0000 7fe8e0a94700 1 -- 172.21.15.81:0/994574684 >> 172.21.15.81:0/994574684 conn(0x7fe8dc06c330 msgr2=0x7fe8dc06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:45.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.355+0000 7fe8e0a94700 1 -- 172.21.15.81:0/994574684 shutdown_connections 2024-05-08T23:32:45.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.355+0000 7fe8e0a94700 1 -- 172.21.15.81:0/994574684 wait complete. 2024-05-08T23:32:45.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.355+0000 7fe8e0a94700 1 Processor -- start 2024-05-08T23:32:45.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.355+0000 7fe8e0a94700 1 -- start start 2024-05-08T23:32:45.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8e0a94700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc071ab0 0x7fe8dc0b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8e0a94700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8dc0ce510 0x7fe8dc156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8e0a94700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8dc15aaa0 0x7fe8dc1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8e0a94700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc13daf0 con 0x7fe8dc15aaa0 2024-05-08T23:32:45.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8e0a94700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0b4400 con 0x7fe8dc0ce510 2024-05-08T23:32:45.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8e0a94700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0b4650 con 0x7fe8dc071ab0 2024-05-08T23:32:45.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8dbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8dc15aaa0 0x7fe8dc1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8dbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8dc15aaa0 0x7fe8dc1571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34400/0 (socket says 172.21.15.81:34400) 2024-05-08T23:32:45.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8dbfff700 1 -- 172.21.15.81:0/561794837 learned_addr learned my addr 172.21.15.81:0/561794837 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:45.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8db7fe700 1 --2- 172.21.15.81:0/561794837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc071ab0 0x7fe8dc0b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8daffd700 1 --2- 172.21.15.81:0/561794837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8dc0ce510 0x7fe8dc156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8dbfff700 1 -- 172.21.15.81:0/561794837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc071ab0 msgr2=0x7fe8dc0b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8dbfff700 1 --2- 172.21.15.81:0/561794837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc071ab0 0x7fe8dc0b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8dbfff700 1 -- 172.21.15.81:0/561794837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8dc0ce510 msgr2=0x7fe8dc156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8dbfff700 1 --2- 172.21.15.81:0/561794837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8dc0ce510 0x7fe8dc156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8dbfff700 1 -- 172.21.15.81:0/561794837 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d0009570 con 0x7fe8dc15aaa0 2024-05-08T23:32:45.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8dbfff700 1 --2- 172.21.15.81:0/561794837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8dc15aaa0 0x7fe8dc1571a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009d30 tx=0x7fe8d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:45.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8c3fff700 1 -- 172.21.15.81:0/561794837 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400cc90 con 0x7fe8dc15aaa0 2024-05-08T23:32:45.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8c3fff700 1 -- 172.21.15.81:0/561794837 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d400f040 con 0x7fe8dc15aaa0 2024-05-08T23:32:45.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8e0a94700 1 -- 172.21.15.81:0/561794837 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8dc0b4930 con 0x7fe8dc15aaa0 2024-05-08T23:32:45.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8c3fff700 1 -- 172.21.15.81:0/561794837 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4009e80 con 0x7fe8dc15aaa0 2024-05-08T23:32:45.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8e0a94700 1 -- 172.21.15.81:0/561794837 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc13d780 con 0x7fe8dc15aaa0 2024-05-08T23:32:45.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.359+0000 7fe8e0a94700 1 -- 172.21.15.81:0/561794837 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8dc066990 con 0x7fe8dc15aaa0 2024-05-08T23:32:45.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.363+0000 7fe8c3fff700 1 -- 172.21.15.81:0/561794837 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe8d4014480 con 0x7fe8dc15aaa0 2024-05-08T23:32:45.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.363+0000 7fe8c3fff700 1 --2- 172.21.15.81:0/561794837 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe8c8066730 0x7fe8c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.363+0000 7fe8c3fff700 1 -- 172.21.15.81:0/561794837 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8d4084a50 con 0x7fe8dc15aaa0 2024-05-08T23:32:45.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.363+0000 7fe8db7fe700 1 --2- 172.21.15.81:0/561794837 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe8c8066730 0x7fe8c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.363+0000 7fe8db7fe700 1 --2- 172.21.15.81:0/561794837 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe8c8066730 0x7fe8c8068bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc000f60 tx=0x7fe8cc007ef0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:45.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.371+0000 7fe8c3fff700 1 -- 172.21.15.81:0/561794837 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe8dc066990 con 0x7fe8dc15aaa0 2024-05-08T23:32:45.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.539+0000 7fe8e0a94700 1 -- 172.21.15.81:0/561794837 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe8dc066990 con 0x7fe8c8066730 2024-05-08T23:32:45.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.555+0000 7fe8c3fff700 1 -- 172.21.15.81:0/561794837 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281151 (secure 0 0 0) 0x7fe8dc066990 con 0x7fe8c8066730 2024-05-08T23:32:45.562 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:45.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.559+0000 7fe8e0a94700 1 -- 172.21.15.81:0/561794837 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe8c8066730 msgr2=0x7fe8c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.559+0000 7fe8e0a94700 1 --2- 172.21.15.81:0/561794837 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe8c8066730 0x7fe8c8068bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc000f60 tx=0x7fe8cc007ef0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.559+0000 7fe8e0a94700 1 -- 172.21.15.81:0/561794837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8dc15aaa0 msgr2=0x7fe8dc1571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.559+0000 7fe8e0a94700 1 --2- 172.21.15.81:0/561794837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8dc15aaa0 0x7fe8dc1571a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009d30 tx=0x7fe8d400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.559+0000 7fe8e0a94700 1 -- 172.21.15.81:0/561794837 shutdown_connections 2024-05-08T23:32:45.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.559+0000 7fe8e0a94700 1 --2- 172.21.15.81:0/561794837 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe8c8066730 0x7fe8c8068bf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.559+0000 7fe8e0a94700 1 --2- 172.21.15.81:0/561794837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8dc15aaa0 0x7fe8dc1571a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.559+0000 7fe8e0a94700 1 --2- 172.21.15.81:0/561794837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8dc0ce510 0x7fe8dc156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.559+0000 7fe8e0a94700 1 --2- 172.21.15.81:0/561794837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc071ab0 0x7fe8dc0b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.559+0000 7fe8e0a94700 1 -- 172.21.15.81:0/561794837 >> 172.21.15.81:0/561794837 conn(0x7fe8dc06c330 msgr2=0x7fe8dc06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:45.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.559+0000 7fe8e0a94700 1 -- 172.21.15.81:0/561794837 shutdown_connections 2024-05-08T23:32:45.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.559+0000 7fe8e0a94700 1 -- 172.21.15.81:0/561794837 wait complete. 2024-05-08T23:32:45.565 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:32:45.578 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-05-08T23:32:45.367756+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590514,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":167,"num_write_kb":2425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230084,"kb_used_data":5728,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":754744636,"statfs":{"total":773094113280,"available":772858507264,"internally_reserved":0,"allocated":5865472,"data_stored":3089824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":229625400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":146,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.894936"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486717+0000","last_change":"2024-05-08T23:32:40.887862+0000","last_active":"2024-05-08T23:32:42.486717+0000","last_peered":"2024-05-08T23:32:42.486717+0000","last_clean":"2024-05-08T23:32:42.486717+0000","last_became_active":"2024-05-08T23:32:40.887591+0000","last_became_peered":"2024-05-08T23:32:40.887591+0000","last_unstale":"2024-05-08T23:32:42.486717+0000","last_undegraded":"2024-05-08T23:32:42.486717+0000","last_fullsized":"2024-05-08T23:32:42.486717+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:14.349815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869810+0000","last_change":"2024-05-08T23:32:40.881814+0000","last_active":"2024-05-08T23:32:41.869810+0000","last_peered":"2024-05-08T23:32:41.869810+0000","last_clean":"2024-05-08T23:32:41.869810+0000","last_became_active":"2024-05-08T23:32:40.879664+0000","last_became_peered":"2024-05-08T23:32:40.879664+0000","last_unstale":"2024-05-08T23:32:41.869810+0000","last_undegraded":"2024-05-08T23:32:41.869810+0000","last_fullsized":"2024-05-08T23:32:41.869810+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:51.917024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486379+0000","last_change":"2024-05-08T23:32:40.886145+0000","last_active":"2024-05-08T23:32:42.486379+0000","last_peered":"2024-05-08T23:32:42.486379+0000","last_clean":"2024-05-08T23:32:42.486379+0000","last_became_active":"2024-05-08T23:32:40.885884+0000","last_became_peered":"2024-05-08T23:32:40.885884+0000","last_unstale":"2024-05-08T23:32:42.486379+0000","last_undegraded":"2024-05-08T23:32:42.486379+0000","last_fullsized":"2024-05-08T23:32:42.486379+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:51.073137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869331+0000","last_change":"2024-05-08T23:32:40.872563+0000","last_active":"2024-05-08T23:32:41.869331+0000","last_peered":"2024-05-08T23:32:41.869331+0000","last_clean":"2024-05-08T23:32:41.869331+0000","last_became_active":"2024-05-08T23:32:40.872068+0000","last_became_peered":"2024-05-08T23:32:40.872068+0000","last_unstale":"2024-05-08T23:32:41.869331+0000","last_undegraded":"2024-05-08T23:32:41.869331+0000","last_fullsized":"2024-05-08T23:32:41.869331+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:33.567285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869928+0000","last_change":"2024-05-08T23:32:40.873779+0000","last_active":"2024-05-08T23:32:41.869928+0000","last_peered":"2024-05-08T23:32:41.869928+0000","last_clean":"2024-05-08T23:32:41.869928+0000","last_became_active":"2024-05-08T23:32:40.873035+0000","last_became_peered":"2024-05-08T23:32:40.873035+0000","last_unstale":"2024-05-08T23:32:41.869928+0000","last_undegraded":"2024-05-08T23:32:41.869928+0000","last_fullsized":"2024-05-08T23:32:41.869928+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:38.166519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485500+0000","last_change":"2024-05-08T23:32:40.885270+0000","last_active":"2024-05-08T23:32:42.485500+0000","last_peered":"2024-05-08T23:32:42.485500+0000","last_clean":"2024-05-08T23:32:42.485500+0000","last_became_active":"2024-05-08T23:32:40.884990+0000","last_became_peered":"2024-05-08T23:32:40.884990+0000","last_unstale":"2024-05-08T23:32:42.485500+0000","last_undegraded":"2024-05-08T23:32:42.485500+0000","last_fullsized":"2024-05-08T23:32:42.485500+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:34.323573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870291+0000","last_change":"2024-05-08T23:32:40.883179+0000","last_active":"2024-05-08T23:32:41.870291+0000","last_peered":"2024-05-08T23:32:41.870291+0000","last_clean":"2024-05-08T23:32:41.870291+0000","last_became_active":"2024-05-08T23:32:40.882263+0000","last_became_peered":"2024-05-08T23:32:40.882263+0000","last_unstale":"2024-05-08T23:32:41.870291+0000","last_undegraded":"2024-05-08T23:32:41.870291+0000","last_fullsized":"2024-05-08T23:32:41.870291+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:52.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486835+0000","last_change":"2024-05-08T23:32:40.887882+0000","last_active":"2024-05-08T23:32:42.486835+0000","last_peered":"2024-05-08T23:32:42.486835+0000","last_clean":"2024-05-08T23:32:42.486835+0000","last_became_active":"2024-05-08T23:32:40.887591+0000","last_became_peered":"2024-05-08T23:32:40.887591+0000","last_unstale":"2024-05-08T23:32:42.486835+0000","last_undegraded":"2024-05-08T23:32:42.486835+0000","last_fullsized":"2024-05-08T23:32:42.486835+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:03.982050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869905+0000","last_change":"2024-05-08T23:32:40.872252+0000","last_active":"2024-05-08T23:32:41.869905+0000","last_peered":"2024-05-08T23:32:41.869905+0000","last_clean":"2024-05-08T23:32:41.869905+0000","last_became_active":"2024-05-08T23:32:40.872011+0000","last_became_peered":"2024-05-08T23:32:40.872011+0000","last_unstale":"2024-05-08T23:32:41.869905+0000","last_undegraded":"2024-05-08T23:32:41.869905+0000","last_fullsized":"2024-05-08T23:32:41.869905+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:59.249962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869149+0000","last_change":"2024-05-08T23:32:40.871672+0000","last_active":"2024-05-08T23:32:41.869149+0000","last_peered":"2024-05-08T23:32:41.869149+0000","last_clean":"2024-05-08T23:32:41.869149+0000","last_became_active":"2024-05-08T23:32:40.871428+0000","last_became_peered":"2024-05-08T23:32:40.871428+0000","last_unstale":"2024-05-08T23:32:41.869149+0000","last_undegraded":"2024-05-08T23:32:41.869149+0000","last_fullsized":"2024-05-08T23:32:41.869149+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:51.073137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870558+0000","last_change":"2024-05-08T23:32:40.882242+0000","last_active":"2024-05-08T23:32:41.870558+0000","last_peered":"2024-05-08T23:32:41.870558+0000","last_clean":"2024-05-08T23:32:41.870558+0000","last_became_active":"2024-05-08T23:32:40.881992+0000","last_became_peered":"2024-05-08T23:32:40.881992+0000","last_unstale":"2024-05-08T23:32:41.870558+0000","last_undegraded":"2024-05-08T23:32:41.870558+0000","last_fullsized":"2024-05-08T23:32:41.870558+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:33.892287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869488+0000","last_change":"2024-05-08T23:32:40.882835+0000","last_active":"2024-05-08T23:32:41.869488+0000","last_peered":"2024-05-08T23:32:41.869488+0000","last_clean":"2024-05-08T23:32:41.869488+0000","last_became_active":"2024-05-08T23:32:40.882284+0000","last_became_peered":"2024-05-08T23:32:40.882284+0000","last_unstale":"2024-05-08T23:32:41.869488+0000","last_undegraded":"2024-05-08T23:32:41.869488+0000","last_fullsized":"2024-05-08T23:32:41.869488+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:22.400514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485731+0000","last_change":"2024-05-08T23:32:40.884162+0000","last_active":"2024-05-08T23:32:42.485731+0000","last_peered":"2024-05-08T23:32:42.485731+0000","last_clean":"2024-05-08T23:32:42.485731+0000","last_became_active":"2024-05-08T23:32:40.883891+0000","last_became_peered":"2024-05-08T23:32:40.883891+0000","last_unstale":"2024-05-08T23:32:42.485731+0000","last_undegraded":"2024-05-08T23:32:42.485731+0000","last_fullsized":"2024-05-08T23:32:42.485731+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:22.400514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869593+0000","last_change":"2024-05-08T23:32:40.872374+0000","last_active":"2024-05-08T23:32:41.869593+0000","last_peered":"2024-05-08T23:32:41.869593+0000","last_clean":"2024-05-08T23:32:41.869593+0000","last_became_active":"2024-05-08T23:32:40.871713+0000","last_became_peered":"2024-05-08T23:32:40.871713+0000","last_unstale":"2024-05-08T23:32:41.869593+0000","last_undegraded":"2024-05-08T23:32:41.869593+0000","last_fullsized":"2024-05-08T23:32:41.869593+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:03.982050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485613+0000","last_change":"2024-05-08T23:32:40.888222+0000","last_active":"2024-05-08T23:32:42.485613+0000","last_peered":"2024-05-08T23:32:42.485613+0000","last_clean":"2024-05-08T23:32:42.485613+0000","last_became_active":"2024-05-08T23:32:40.887649+0000","last_became_peered":"2024-05-08T23:32:40.887649+0000","last_unstale":"2024-05-08T23:32:42.485613+0000","last_undegraded":"2024-05-08T23:32:42.485613+0000","last_fullsized":"2024-05-08T23:32:42.485613+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:56.942715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.482923+0000","last_change":"2024-05-08T23:32:39.482923+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.482923+0000","last_undegraded":"2024-05-08T23:32:39.482923+0000","last_fullsized":"2024-05-08T23:32:39.482923+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:13.825466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.479952+0000","last_change":"2024-05-08T23:32:39.479952+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.479952+0000","last_undegraded":"2024-05-08T23:32:39.479952+0000","last_fullsized":"2024-05-08T23:32:39.479952+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:43.629558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.476540+0000","last_change":"2024-05-08T23:32:39.476540+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.476540+0000","last_undegraded":"2024-05-08T23:32:39.476540+0000","last_fullsized":"2024-05-08T23:32:39.476540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:39.770766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481883+0000","last_change":"2024-05-08T23:32:39.481883+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481883+0000","last_undegraded":"2024-05-08T23:32:39.481883+0000","last_fullsized":"2024-05-08T23:32:39.481883+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:12.523765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869722+0000","last_change":"2024-05-08T23:32:40.503223+0000","last_active":"2024-05-08T23:32:41.869722+0000","last_peered":"2024-05-08T23:32:41.869722+0000","last_clean":"2024-05-08T23:32:41.869722+0000","last_became_active":"2024-05-08T23:32:40.502907+0000","last_became_peered":"2024-05-08T23:32:40.502907+0000","last_unstale":"2024-05-08T23:32:41.869722+0000","last_undegraded":"2024-05-08T23:32:41.869722+0000","last_fullsized":"2024-05-08T23:32:41.869722+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:43.304902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486163+0000","last_change":"2024-05-08T23:32:40.884604+0000","last_active":"2024-05-08T23:32:42.486163+0000","last_peered":"2024-05-08T23:32:42.486163+0000","last_clean":"2024-05-08T23:32:42.486163+0000","last_became_active":"2024-05-08T23:32:40.884337+0000","last_became_peered":"2024-05-08T23:32:40.884337+0000","last_unstale":"2024-05-08T23:32:42.486163+0000","last_undegraded":"2024-05-08T23:32:42.486163+0000","last_fullsized":"2024-05-08T23:32:42.486163+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:44.757107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477601+0000","last_change":"2024-05-08T23:32:39.477601+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477601+0000","last_undegraded":"2024-05-08T23:32:39.477601+0000","last_fullsized":"2024-05-08T23:32:39.477601+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:41.416678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481005+0000","last_change":"2024-05-08T23:32:39.481005+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481005+0000","last_undegraded":"2024-05-08T23:32:39.481005+0000","last_fullsized":"2024-05-08T23:32:39.481005+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:14.674732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.487020+0000","last_change":"2024-05-08T23:32:40.885545+0000","last_active":"2024-05-08T23:32:42.487020+0000","last_peered":"2024-05-08T23:32:42.487020+0000","last_clean":"2024-05-08T23:32:42.487020+0000","last_became_active":"2024-05-08T23:32:40.885195+0000","last_became_peered":"2024-05-08T23:32:40.885195+0000","last_unstale":"2024-05-08T23:32:42.487020+0000","last_undegraded":"2024-05-08T23:32:42.487020+0000","last_fullsized":"2024-05-08T23:32:42.487020+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:33.567285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869218+0000","last_change":"2024-05-08T23:32:40.500395+0000","last_active":"2024-05-08T23:32:41.869218+0000","last_peered":"2024-05-08T23:32:41.869218+0000","last_clean":"2024-05-08T23:32:41.869218+0000","last_became_active":"2024-05-08T23:32:40.500130+0000","last_became_peered":"2024-05-08T23:32:40.500130+0000","last_unstale":"2024-05-08T23:32:41.869218+0000","last_undegraded":"2024-05-08T23:32:41.869218+0000","last_fullsized":"2024-05-08T23:32:41.869218+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:08.134598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.487136+0000","last_change":"2024-05-08T23:32:40.502985+0000","last_active":"2024-05-08T23:32:42.487136+0000","last_peered":"2024-05-08T23:32:42.487136+0000","last_clean":"2024-05-08T23:32:42.487136+0000","last_became_active":"2024-05-08T23:32:40.502677+0000","last_became_peered":"2024-05-08T23:32:40.502677+0000","last_unstale":"2024-05-08T23:32:42.487136+0000","last_undegraded":"2024-05-08T23:32:42.487136+0000","last_fullsized":"2024-05-08T23:32:42.487136+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:26.160105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477408+0000","last_change":"2024-05-08T23:32:39.477408+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477408+0000","last_undegraded":"2024-05-08T23:32:39.477408+0000","last_fullsized":"2024-05-08T23:32:39.477408+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:31.833826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870114+0000","last_change":"2024-05-08T23:32:40.877145+0000","last_active":"2024-05-08T23:32:41.870114+0000","last_peered":"2024-05-08T23:32:41.870114+0000","last_clean":"2024-05-08T23:32:41.870114+0000","last_became_active":"2024-05-08T23:32:40.876105+0000","last_became_peered":"2024-05-08T23:32:40.876105+0000","last_unstale":"2024-05-08T23:32:41.870114+0000","last_undegraded":"2024-05-08T23:32:41.870114+0000","last_fullsized":"2024-05-08T23:32:41.870114+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:44.757107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486576+0000","last_change":"2024-05-08T23:32:40.503254+0000","last_active":"2024-05-08T23:32:42.486576+0000","last_peered":"2024-05-08T23:32:42.486576+0000","last_clean":"2024-05-08T23:32:42.486576+0000","last_became_active":"2024-05-08T23:32:40.503019+0000","last_became_peered":"2024-05-08T23:32:40.503019+0000","last_unstale":"2024-05-08T23:32:42.486576+0000","last_undegraded":"2024-05-08T23:32:42.486576+0000","last_fullsized":"2024-05-08T23:32:42.486576+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:32.059412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481770+0000","last_change":"2024-05-08T23:32:39.481770+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481770+0000","last_undegraded":"2024-05-08T23:32:39.481770+0000","last_fullsized":"2024-05-08T23:32:39.481770+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:24.848996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869846+0000","last_change":"2024-05-08T23:32:40.501242+0000","last_active":"2024-05-08T23:32:41.869846+0000","last_peered":"2024-05-08T23:32:41.869846+0000","last_clean":"2024-05-08T23:32:41.869846+0000","last_became_active":"2024-05-08T23:32:40.500998+0000","last_became_peered":"2024-05-08T23:32:40.500998+0000","last_unstale":"2024-05-08T23:32:41.869846+0000","last_undegraded":"2024-05-08T23:32:41.869846+0000","last_fullsized":"2024-05-08T23:32:41.869846+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:09.047211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870109+0000","last_change":"2024-05-08T23:32:40.873424+0000","last_active":"2024-05-08T23:32:41.870109+0000","last_peered":"2024-05-08T23:32:41.870109+0000","last_clean":"2024-05-08T23:32:41.870109+0000","last_became_active":"2024-05-08T23:32:40.873161+0000","last_became_peered":"2024-05-08T23:32:40.873161+0000","last_unstale":"2024-05-08T23:32:41.870109+0000","last_undegraded":"2024-05-08T23:32:41.870109+0000","last_fullsized":"2024-05-08T23:32:41.870109+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:14.349815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483474+0000","last_change":"2024-05-08T23:32:39.483474+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483474+0000","last_undegraded":"2024-05-08T23:32:39.483474+0000","last_fullsized":"2024-05-08T23:32:39.483474+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:31.286020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485852+0000","last_change":"2024-05-08T23:32:40.499587+0000","last_active":"2024-05-08T23:32:42.485852+0000","last_peered":"2024-05-08T23:32:42.485852+0000","last_clean":"2024-05-08T23:32:42.485852+0000","last_became_active":"2024-05-08T23:32:40.499243+0000","last_became_peered":"2024-05-08T23:32:40.499243+0000","last_unstale":"2024-05-08T23:32:42.485852+0000","last_undegraded":"2024-05-08T23:32:42.485852+0000","last_fullsized":"2024-05-08T23:32:42.485852+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:53.323381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870508+0000","last_change":"2024-05-08T23:32:40.872864+0000","last_active":"2024-05-08T23:32:41.870508+0000","last_peered":"2024-05-08T23:32:41.870508+0000","last_clean":"2024-05-08T23:32:41.870508+0000","last_became_active":"2024-05-08T23:32:40.872596+0000","last_became_peered":"2024-05-08T23:32:40.872596+0000","last_unstale":"2024-05-08T23:32:41.870508+0000","last_undegraded":"2024-05-08T23:32:41.870508+0000","last_fullsized":"2024-05-08T23:32:41.870508+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:18.771018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481540+0000","last_change":"2024-05-08T23:32:39.481540+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481540+0000","last_undegraded":"2024-05-08T23:32:39.481540+0000","last_fullsized":"2024-05-08T23:32:39.481540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:31.133500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870719+0000","last_change":"2024-05-08T23:32:40.499582+0000","last_active":"2024-05-08T23:32:41.870719+0000","last_peered":"2024-05-08T23:32:41.870719+0000","last_clean":"2024-05-08T23:32:41.870719+0000","last_became_active":"2024-05-08T23:32:40.497931+0000","last_became_peered":"2024-05-08T23:32:40.497931+0000","last_unstale":"2024-05-08T23:32:41.870719+0000","last_undegraded":"2024-05-08T23:32:41.870719+0000","last_fullsized":"2024-05-08T23:32:41.870719+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:40.305843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483298+0000","last_change":"2024-05-08T23:32:39.483298+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483298+0000","last_undegraded":"2024-05-08T23:32:39.483298+0000","last_fullsized":"2024-05-08T23:32:39.483298+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:43:57.594449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481124+0000","last_change":"2024-05-08T23:32:39.481124+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481124+0000","last_undegraded":"2024-05-08T23:32:39.481124+0000","last_fullsized":"2024-05-08T23:32:39.481124+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:33:25.871051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483428+0000","last_change":"2024-05-08T23:32:39.483428+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483428+0000","last_undegraded":"2024-05-08T23:32:39.483428+0000","last_fullsized":"2024-05-08T23:32:39.483428+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:06.250791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477222+0000","last_change":"2024-05-08T23:32:39.477222+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477222+0000","last_undegraded":"2024-05-08T23:32:39.477222+0000","last_fullsized":"2024-05-08T23:32:39.477222+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:54.923508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.484968+0000","last_change":"2024-05-08T23:32:39.484968+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.484968+0000","last_undegraded":"2024-05-08T23:32:39.484968+0000","last_fullsized":"2024-05-08T23:32:39.484968+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:18:31.349703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870336+0000","last_change":"2024-05-08T23:32:40.874535+0000","last_active":"2024-05-08T23:32:41.870336+0000","last_peered":"2024-05-08T23:32:41.870336+0000","last_clean":"2024-05-08T23:32:41.870336+0000","last_became_active":"2024-05-08T23:32:40.874316+0000","last_became_peered":"2024-05-08T23:32:40.874316+0000","last_unstale":"2024-05-08T23:32:41.870336+0000","last_undegraded":"2024-05-08T23:32:41.870336+0000","last_fullsized":"2024-05-08T23:32:41.870336+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:56.942715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869562+0000","last_change":"2024-05-08T23:32:40.500526+0000","last_active":"2024-05-08T23:32:41.869562+0000","last_peered":"2024-05-08T23:32:41.869562+0000","last_clean":"2024-05-08T23:32:41.869562+0000","last_became_active":"2024-05-08T23:32:40.500219+0000","last_became_peered":"2024-05-08T23:32:40.500219+0000","last_unstale":"2024-05-08T23:32:41.869562+0000","last_undegraded":"2024-05-08T23:32:41.869562+0000","last_fullsized":"2024-05-08T23:32:41.869562+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:56.869056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.482871+0000","last_change":"2024-05-08T23:32:39.482871+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.482871+0000","last_undegraded":"2024-05-08T23:32:39.482871+0000","last_fullsized":"2024-05-08T23:32:39.482871+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:20:31.576039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481313+0000","last_change":"2024-05-08T23:32:39.481313+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481313+0000","last_undegraded":"2024-05-08T23:32:39.481313+0000","last_fullsized":"2024-05-08T23:32:39.481313+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:21.704760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486777+0000","last_change":"2024-05-08T23:32:40.499075+0000","last_active":"2024-05-08T23:32:42.486777+0000","last_peered":"2024-05-08T23:32:42.486777+0000","last_clean":"2024-05-08T23:32:42.486777+0000","last_became_active":"2024-05-08T23:32:40.498774+0000","last_became_peered":"2024-05-08T23:32:40.498774+0000","last_unstale":"2024-05-08T23:32:42.486777+0000","last_undegraded":"2024-05-08T23:32:42.486777+0000","last_fullsized":"2024-05-08T23:32:42.486777+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:09.302638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869732+0000","last_change":"2024-05-08T23:32:40.499734+0000","last_active":"2024-05-08T23:32:41.869732+0000","last_peered":"2024-05-08T23:32:41.869732+0000","last_clean":"2024-05-08T23:32:41.869732+0000","last_became_active":"2024-05-08T23:32:40.497679+0000","last_became_peered":"2024-05-08T23:32:40.497679+0000","last_unstale":"2024-05-08T23:32:41.869732+0000","last_undegraded":"2024-05-08T23:32:41.869732+0000","last_fullsized":"2024-05-08T23:32:41.869732+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:58.686919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869659+0000","last_change":"2024-05-08T23:32:40.885944+0000","last_active":"2024-05-08T23:32:41.869659+0000","last_peered":"2024-05-08T23:32:41.869659+0000","last_clean":"2024-05-08T23:32:41.869659+0000","last_became_active":"2024-05-08T23:32:40.885687+0000","last_became_peered":"2024-05-08T23:32:40.885687+0000","last_unstale":"2024-05-08T23:32:41.869659+0000","last_undegraded":"2024-05-08T23:32:41.869659+0000","last_fullsized":"2024-05-08T23:32:41.869659+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:33.069307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483932+0000","last_change":"2024-05-08T23:32:39.483932+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483932+0000","last_undegraded":"2024-05-08T23:32:39.483932+0000","last_fullsized":"2024-05-08T23:32:39.483932+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:46.898315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.484322+0000","last_change":"2024-05-08T23:32:39.484322+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.484322+0000","last_undegraded":"2024-05-08T23:32:39.484322+0000","last_fullsized":"2024-05-08T23:32:39.484322+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869988+0000","last_change":"2024-05-08T23:32:40.873427+0000","last_active":"2024-05-08T23:32:41.869988+0000","last_peered":"2024-05-08T23:32:41.869988+0000","last_clean":"2024-05-08T23:32:41.869988+0000","last_became_active":"2024-05-08T23:32:40.873194+0000","last_became_peered":"2024-05-08T23:32:40.873194+0000","last_unstale":"2024-05-08T23:32:41.869988+0000","last_undegraded":"2024-05-08T23:32:41.869988+0000","last_fullsized":"2024-05-08T23:32:41.869988+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:34.323573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477580+0000","last_change":"2024-05-08T23:32:39.477580+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477580+0000","last_undegraded":"2024-05-08T23:32:39.477580+0000","last_fullsized":"2024-05-08T23:32:39.477580+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:19.747747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481541+0000","last_change":"2024-05-08T23:32:39.481541+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481541+0000","last_undegraded":"2024-05-08T23:32:39.481541+0000","last_fullsized":"2024-05-08T23:32:39.481541+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:36.991964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870540+0000","last_change":"2024-05-08T23:32:40.503644+0000","last_active":"2024-05-08T23:32:41.870540+0000","last_peered":"2024-05-08T23:32:41.870540+0000","last_clean":"2024-05-08T23:32:41.870540+0000","last_became_active":"2024-05-08T23:32:40.503283+0000","last_became_peered":"2024-05-08T23:32:40.503283+0000","last_unstale":"2024-05-08T23:32:41.870540+0000","last_undegraded":"2024-05-08T23:32:41.870540+0000","last_fullsized":"2024-05-08T23:32:41.870540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:59.435733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":190,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.876580+0000","last_change":"2024-05-08T23:30:27.226484+0000","last_active":"2024-05-08T23:32:40.876580+0000","last_peered":"2024-05-08T23:32:40.876580+0000","last_clean":"2024-05-08T23:32:40.876580+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:32:40.876580+0000","last_undegraded":"2024-05-08T23:32:40.876580+0000","last_fullsized":"2024-05-08T23:32:40.876580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_clean_scrub_stamp":"2024-05-08T23:30:26.204118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:14.267287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":46,"acting":46,"num_store_stats":2},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":146,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.89400000000000002,"15min":0.90500000000000003},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1719999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.96899999999999997,"15min":0.97899999999999998},"min":{"1min":0.58299999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1910000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.89700000000000002}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.59099999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6579999999999999,"5min":2.004,"15min":2.004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.92600000000000005,"15min":0.92400000000000004},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.327,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.68200000000000005}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88300000000000001,"15min":0.88500000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.95999999999999996,"15min":0.97099999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.218,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.78900000000000003}]},{"osd":3,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.89200000000000002,"15min":0.88400000000000001},"min":{"1min":0.65100000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96899999999999997,"15min":0.97799999999999998},"min":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2290000000000001,"5min":1.272,"15min":1.272},"last":0.85399999999999998}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.71499999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.1439999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0249999999999999,"15min":1.0269999999999999},"min":{"1min":0.67800000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.95299999999999996}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0469999999999999,"15min":1.056},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4450000000000001,"5min":2.109,"15min":2.109},"last":1.006},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1240000000000001,"15min":1.1439999999999999},"min":{"1min":0.75800000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.98099999999999998}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1970000000000001,"15min":1.2070000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5660000000000001,"5min":2.544,"15min":2.544},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.103,"15min":1.109},"min":{"1min":0.81200000000000006,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.0669999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2969999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.97099999999999997,"15min":0.98299999999999998},"min":{"1min":0.63700000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1859999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.65700000000000003}]},{"osd":1,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.90900000000000003,"15min":0.90400000000000003},"min":{"1min":0.67300000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88400000000000001,"15min":0.878},"min":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.0309999999999999}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.91800000000000004,"15min":0.91700000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.385,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.93700000000000006,"15min":0.92500000000000004},"min":{"1min":0.70299999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.454,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.93100000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.97199999999999998,"15min":0.98399999999999999},"min":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.252,"5min":1.395,"15min":1.395},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.004,"5min":0.96099999999999997,"15min":0.95399999999999996},"min":{"1min":0.63700000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.0049999999999999}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.995,"15min":1.0129999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.1339999999999999,"15min":1.1599999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.484,"5min":2.464,"15min":2.464},"last":0.78500000000000003}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0800000000000001,"15min":1.0820000000000001},"min":{"1min":0.82099999999999995,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.871}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1930000000000001,"15min":1.216},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.163,"5min":1.1759999999999999,"15min":1.1779999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.2909999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.84099999999999997,"15min":0.85099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2030000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77900000000000003,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.4910000000000001}]},{"osd":1,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.70999999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1459999999999999,"5min":1.268,"15min":1.268},"last":1.47},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.88800000000000001,"15min":0.90000000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.125,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.0920000000000001}]},{"osd":2,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.88800000000000001,"15min":0.89800000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89900000000000002,"15min":0.90700000000000003},"min":{"1min":0.65400000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0249999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.5669999999999999}]},{"osd":3,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.85299999999999998,"15min":0.84899999999999998},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1759999999999999,"5min":1.23,"15min":1.23},"last":1.3120000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2390000000000001,"5min":1.431,"15min":1.431},"last":1.357}]},{"osd":5,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3420000000000001,"5min":1.413,"15min":1.413},"last":1.7669999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95899999999999996,"15min":0.95099999999999996},"min":{"1min":0.73999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.278,"5min":1.504,"15min":1.504},"last":1.409}]},{"osd":6,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0269999999999999,"15min":1.0309999999999999},"min":{"1min":0.73899999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.52},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1419999999999999,"15min":1.1559999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3859999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.867}]},{"osd":7,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0800000000000001,"15min":1.0840000000000001},"min":{"1min":0.69899999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4379999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.653},{"interface":"front","average":{"1min":1.113,"5min":1.143,"15min":1.1479999999999999},"min":{"1min":0.94799999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4099999999999999,"5min":1.583,"15min":1.583},"last":1.7290000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.49099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.46300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":1.032,"15min":1.032},"last":0.67700000000000005}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70599999999999996,"15min":0.71099999999999997},"min":{"1min":0.50700000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.89300000000000002,"5min":1,"15min":1},"last":0.98899999999999999}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0469999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80200000000000005,"15min":0.79900000000000004},"min":{"1min":0.56699999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0229999999999999,"5min":1.242,"15min":1.242},"last":0.71499999999999997}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.65600000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.129,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.96699999999999997,"15min":0.97399999999999998},"min":{"1min":0.78000000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.244,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.749}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.014,"15min":1.022},"min":{"1min":0.80300000000000005,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.161,"5min":1.623,"15min":1.623},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.995,"15min":1.004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1359999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.89000000000000001}]},{"osd":6,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.997,"15min":1.0009999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.232,"5min":1.482,"15min":1.482},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.093,"15min":1.111},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3480000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.0209999999999999}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0800000000000001,"15min":1.0860000000000001},"min":{"1min":0.78400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.456,"5min":1.514,"15min":1.514},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0580000000000001,"15min":1.0600000000000001},"min":{"1min":0.75,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.2869999999999999,"5min":1.552,"15min":1.552},"last":0.96099999999999997}]}]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80400000000000005,"5min":1.099,"15min":1.099},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0660000000000001,"5min":1.129,"15min":1.129},"last":0.51700000000000002}]},{"osd":1,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70999999999999996,"15min":0.70899999999999996},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0509999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.84799999999999998}]},{"osd":4,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.96799999999999997,"15min":0.97799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.27,"5min":1.367,"15min":1.367},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.95599999999999996,"15min":0.96399999999999997},"min":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98999999999999999,"15min":0.98599999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.3},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.91600000000000004}]},{"osd":6,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0620000000000001,"15min":1.0740000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4239999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.22},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.075,"15min":1.075},"min":{"1min":0.85699999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.381}]},{"osd":7,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1220000000000001,"15min":1.1240000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1000000000000001,"15min":1.1020000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.492,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.0109999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.51300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.44,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.442}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.628},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.68899999999999995,"15min":0.67000000000000004},"min":{"1min":0.53900000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.39000000000000001}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73499999999999999,"15min":0.72599999999999998},"min":{"1min":0.57999999999999996,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.42799999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.038,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.95099999999999996,"15min":0.95199999999999996},"min":{"1min":0.67600000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.256,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":0.88},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.95499999999999996,"15min":0.94299999999999995},"min":{"1min":0.77300000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.2709999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.70499999999999996}]},{"osd":5,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0109999999999999,"15min":1.0089999999999999},"min":{"1min":0.72299999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.403,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.75},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.077,"15min":1.079},"min":{"1min":0.85499999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3879999999999999,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.81499999999999995}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0369999999999999,"15min":1.034},"min":{"1min":0.80000000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.4330000000000001,"5min":1.633,"15min":1.633},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.095,"5min":1.135,"15min":1.1419999999999999},"min":{"1min":0.80500000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.4750000000000001,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.91800000000000004}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.0209999999999999,"15min":1.0009999999999999},"min":{"1min":0.83099999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.371,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.081,"15min":1.0820000000000001},"min":{"1min":0.84999999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.306,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.80100000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542175,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":843618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.88300000000000001,"15min":0.89700000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.84699999999999998}]},{"osd":1,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.86399999999999999,"15min":0.86699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1379999999999999,"5min":1.359,"15min":1.359},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91200000000000003,"15min":0.91600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.23,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.747}]},{"osd":2,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88,"15min":0.873},"min":{"1min":0.57499999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88,"15min":0.88500000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.133,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.97699999999999998}]},{"osd":3,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.91300000000000003,"15min":0.91100000000000003},"min":{"1min":0.60699999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.99299999999999999,"15min":0.99099999999999999},"min":{"1min":0.80100000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2470000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.94999999999999996}]},{"osd":4,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.90400000000000003,"15min":0.91200000000000003},"min":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.73999999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.1899999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.1080000000000001}]},{"osd":5,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.028,"15min":1.028},"min":{"1min":0.64100000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.405,"5min":1.714,"15min":1.714},"last":1.069},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.1950000000000001,"5min":1.325,"15min":1.325},"last":1.165}]},{"osd":6,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.048,"15min":1.048},"min":{"1min":0.86599999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.4430000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.105,"15min":1.107},"min":{"1min":0.83999999999999997,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.359,"5min":1.496,"15min":1.496},"last":1.151}]}]},{"osd":0,"up_from":14,"seq":60129542175,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":843550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.47499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":1.351},"last":0.71799999999999997}]},{"osd":2,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.51500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.238,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.546,"5min":1.869,"15min":1.869},"last":0.80100000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.84199999999999997,"15min":0.85599999999999998},"min":{"1min":0.55200000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.94199999999999995,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.89300000000000002}]},{"osd":4,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97199999999999998,"15min":0.97299999999999998},"min":{"1min":0.71999999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.357,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.90800000000000003,"15min":0.89600000000000002},"min":{"1min":0.69899999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.746}]},{"osd":5,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.052,"15min":1.0600000000000001},"min":{"1min":0.76800000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3819999999999999,"5min":1.474,"15min":1.474},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.92500000000000004,"15min":0.91400000000000003},"min":{"1min":0.72699999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.98099999999999998}]},{"osd":6,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.056,"15min":1.0720000000000001},"min":{"1min":0.76900000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.171,"5min":1.736,"15min":1.736},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1180000000000001,"15min":1.121},"min":{"1min":0.89000000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4830000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.0009999999999999}]},{"osd":7,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.113,"15min":1.117},"min":{"1min":0.66600000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.397,"5min":1.72,"15min":1.72},"last":1.115},{"interface":"front","average":{"1min":1.097,"5min":1.109,"15min":1.111},"min":{"1min":0.86399999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.1439999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:32:45.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2024-05-08T23:32:45.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2024-05-08T23:32:45.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2024-05-08T23:32:45.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2024-05-08T23:32:45.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2024-05-08T23:32:45.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2024-05-08T23:32:45.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2024-05-08T23:32:45.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2024-05-08T23:32:45.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2024-05-08T23:32:45.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2024-05-08T23:32:45.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2024-05-08T23:32:45.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2024-05-08T23:32:45.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2024-05-08T23:32:45.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2024-05-08T23:32:45.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2024-05-08T23:32:45.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2024-05-08T23:32:45.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2024-05-08T23:32:45.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2024-05-08T23:32:45.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-05-08T23:32:45.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.482923+0000', 'last_change': '2024-05-08T23:32:39.482923+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.482923+0000', 'last_undegraded': '2024-05-08T23:32:39.482923+0000', 'last_fullsized': '2024-05-08T23:32:39.482923+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T10:23:13.825466+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T23:32:45.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2024-05-08T23:32:45.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-05-08T23:32:45.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.479952+0000', 'last_change': '2024-05-08T23:32:39.479952+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.479952+0000', 'last_undegraded': '2024-05-08T23:32:39.479952+0000', 'last_fullsized': '2024-05-08T23:32:39.479952+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T06:11:43.629558+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T23:32:45.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2024-05-08T23:32:45.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-05-08T23:32:45.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.476540+0000', 'last_change': '2024-05-08T23:32:39.476540+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.476540+0000', 'last_undegraded': '2024-05-08T23:32:39.476540+0000', 'last_fullsized': '2024-05-08T23:32:39.476540+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T08:36:39.770766+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T23:32:45.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-05-08T23:32:45.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-05-08T23:32:45.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.481883+0000', 'last_change': '2024-05-08T23:32:39.481883+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.481883+0000', 'last_undegraded': '2024-05-08T23:32:39.481883+0000', 'last_fullsized': '2024-05-08T23:32:39.481883+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T02:09:12.523765+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T23:32:45.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2024-05-08T23:32:45.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-05-08T23:32:45.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.477601+0000', 'last_change': '2024-05-08T23:32:39.477601+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.477601+0000', 'last_undegraded': '2024-05-08T23:32:39.477601+0000', 'last_fullsized': '2024-05-08T23:32:39.477601+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T01:21:41.416678+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T23:32:45.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-05-08T23:32:45.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-05-08T23:32:45.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.481005+0000', 'last_change': '2024-05-08T23:32:39.481005+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.481005+0000', 'last_undegraded': '2024-05-08T23:32:39.481005+0000', 'last_fullsized': '2024-05-08T23:32:39.481005+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T03:39:14.674732+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T23:32:45.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2024-05-08T23:32:45.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2024-05-08T23:32:45.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-05-08T23:32:45.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.477408+0000', 'last_change': '2024-05-08T23:32:39.477408+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.477408+0000', 'last_undegraded': '2024-05-08T23:32:39.477408+0000', 'last_fullsized': '2024-05-08T23:32:39.477408+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-09T23:34:31.833826+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T23:32:45.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2024-05-08T23:32:45.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-05-08T23:32:45.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.481770+0000', 'last_change': '2024-05-08T23:32:39.481770+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.481770+0000', 'last_undegraded': '2024-05-08T23:32:39.481770+0000', 'last_fullsized': '2024-05-08T23:32:39.481770+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:46:24.848996+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:32:45.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2024-05-08T23:32:45.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-05-08T23:32:45.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.483474+0000', 'last_change': '2024-05-08T23:32:39.483474+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.483474+0000', 'last_undegraded': '2024-05-08T23:32:39.483474+0000', 'last_fullsized': '2024-05-08T23:32:39.483474+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T04:38:31.286020+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T23:32:45.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-05-08T23:32:45.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-05-08T23:32:45.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.481540+0000', 'last_change': '2024-05-08T23:32:39.481540+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.481540+0000', 'last_undegraded': '2024-05-08T23:32:39.481540+0000', 'last_fullsized': '2024-05-08T23:32:39.481540+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T10:00:31.133500+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:32:45.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-05-08T23:32:45.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2024-05-08T23:32:45.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-05-08T23:32:45.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.483298+0000', 'last_change': '2024-05-08T23:32:39.483298+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.483298+0000', 'last_undegraded': '2024-05-08T23:32:39.483298+0000', 'last_fullsized': '2024-05-08T23:32:39.483298+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T04:43:57.594449+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T23:32:45.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2024-05-08T23:32:45.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-05-08T23:32:45.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.481124+0000', 'last_change': '2024-05-08T23:32:39.481124+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.481124+0000', 'last_undegraded': '2024-05-08T23:32:39.481124+0000', 'last_fullsized': '2024-05-08T23:32:39.481124+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T03:33:25.871051+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T23:32:45.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2024-05-08T23:32:45.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-05-08T23:32:45.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.483428+0000', 'last_change': '2024-05-08T23:32:39.483428+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.483428+0000', 'last_undegraded': '2024-05-08T23:32:39.483428+0000', 'last_fullsized': '2024-05-08T23:32:39.483428+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T08:36:06.250791+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T23:32:45.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2024-05-08T23:32:45.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-05-08T23:32:45.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.477222+0000', 'last_change': '2024-05-08T23:32:39.477222+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.477222+0000', 'last_undegraded': '2024-05-08T23:32:39.477222+0000', 'last_fullsized': '2024-05-08T23:32:39.477222+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T07:23:54.923508+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T23:32:45.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2024-05-08T23:32:45.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-05-08T23:32:45.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.484968+0000', 'last_change': '2024-05-08T23:32:39.484968+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.484968+0000', 'last_undegraded': '2024-05-08T23:32:39.484968+0000', 'last_fullsized': '2024-05-08T23:32:39.484968+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T02:18:31.349703+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:32:45.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2024-05-08T23:32:45.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-05-08T23:32:45.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.482871+0000', 'last_change': '2024-05-08T23:32:39.482871+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.482871+0000', 'last_undegraded': '2024-05-08T23:32:39.482871+0000', 'last_fullsized': '2024-05-08T23:32:39.482871+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T01:20:31.576039+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T23:32:45.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2024-05-08T23:32:45.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-05-08T23:32:45.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.481313+0000', 'last_change': '2024-05-08T23:32:39.481313+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.481313+0000', 'last_undegraded': '2024-05-08T23:32:39.481313+0000', 'last_fullsized': '2024-05-08T23:32:39.481313+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T09:27:21.704760+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T23:32:45.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-05-08T23:32:45.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-05-08T23:32:45.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-05-08T23:32:45.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.483932+0000', 'last_change': '2024-05-08T23:32:39.483932+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.483932+0000', 'last_undegraded': '2024-05-08T23:32:39.483932+0000', 'last_fullsized': '2024-05-08T23:32:39.483932+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T09:02:46.898315+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:32:45.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2024-05-08T23:32:45.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-05-08T23:32:45.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.484322+0000', 'last_change': '2024-05-08T23:32:39.484322+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.484322+0000', 'last_undegraded': '2024-05-08T23:32:39.484322+0000', 'last_fullsized': '2024-05-08T23:32:39.484322+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T00:37:00.581348+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T23:32:45.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-05-08T23:32:45.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.477580+0000', 'last_change': '2024-05-08T23:32:39.477580+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.477580+0000', 'last_undegraded': '2024-05-08T23:32:39.477580+0000', 'last_fullsized': '2024-05-08T23:32:39.477580+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T01:29:19.747747+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T23:32:45.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-05-08T23:32:45.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-05-08T23:32:45.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-05-08T23:32:39.481541+0000', 'last_change': '2024-05-08T23:32:39.481541+0000', 'last_active': '2024-05-08T23:32:39.461956+0000', 'last_peered': '2024-05-08T23:32:39.461956+0000', 'last_clean': '2024-05-08T23:32:39.461956+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:39.481541+0000', 'last_undegraded': '2024-05-08T23:32:39.481541+0000', 'last_fullsized': '2024-05-08T23:32:39.481541+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:39.461956+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-05-10T08:36:36.991964+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T23:32:45.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2024-05-08T23:32:45.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-08T23:32:40.471851+0000', 'last_change': '2024-05-08T23:32:40.471851+0000', 'last_active': '2024-05-08T23:32:40.471851+0000', 'last_peered': '2024-05-08T23:32:40.471851+0000', 'last_clean': '2024-05-08T23:32:40.471851+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T23:32:40.471851+0000', 'last_undegraded': '2024-05-08T23:32:40.471851+0000', 'last_fullsized': '2024-05-08T23:32:40.471851+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'last_clean_scrub_stamp': '2024-05-08T23:32:40.471851+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-08T23:32:45.596 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T23:32:45.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.703+0000 7f0a2ffff700 1 Processor -- start 2024-05-08T23:32:45.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.703+0000 7f0a2ffff700 1 -- start start 2024-05-08T23:32:45.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2ffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0a300aed40 0x7f0a3015e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3006b650 0x7f0a300b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2ffff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3015e670 con 0x7f0a300b3750 2024-05-08T23:32:45.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2ffff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a300b35e0 con 0x7f0a300aed40 2024-05-08T23:32:45.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a300b4120 con 0x7f0a3006b650 2024-05-08T23:32:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2f7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3006b650 0x7f0a300b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2e7fc700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f0a300b3750 0x7f0a30160cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:46984/0 (socket says 172.21.15.81:46984) 2024-05-08T23:32:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2f7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3006b650 0x7f0a300b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:50062/0 (socket says 172.21.15.81:50062) 2024-05-08T23:32:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2e7fc700 1 -- 172.21.15.81:0/1463356743 learned_addr learned my addr 172.21.15.81:0/1463356743 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2effd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0a300aed40 0x7f0a3015e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2f7fe700 1 -- 172.21.15.81:0/1463356743 >> v2:172.21.15.117:3300/0 conn(0x7f0a300aed40 msgr2=0x7f0a3015e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2f7fe700 1 --2- 172.21.15.81:0/1463356743 >> v2:172.21.15.117:3300/0 conn(0x7f0a300aed40 0x7f0a3015e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2f7fe700 1 -- 172.21.15.81:0/1463356743 >> v1:172.21.15.117:6789/0 conn(0x7f0a300b3750 legacy=0x7f0a30160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2f7fe700 1 -- 172.21.15.81:0/1463356743 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a300b2960 con 0x7f0a3006b650 2024-05-08T23:32:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2f7fe700 1 --2- 172.21.15.81:0/1463356743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3006b650 0x7f0a300b3be0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009d90 tx=0x7f0a2800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce33db35a42d21f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2dffb700 1 -- 172.21.15.81:0/1463356743 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800e070 con 0x7f0a3006b650 2024-05-08T23:32:45.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2dffb700 1 -- 172.21.15.81:0/1463356743 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2800bb20 con 0x7f0a3006b650 2024-05-08T23:32:45.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2dffb700 1 -- 172.21.15.81:0/1463356743 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28012430 con 0x7f0a3006b650 2024-05-08T23:32:45.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2ffff700 1 -- 172.21.15.81:0/1463356743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3006b650 msgr2=0x7f0a300b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.707+0000 7f0a2ffff700 1 --2- 172.21.15.81:0/1463356743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3006b650 0x7f0a300b3be0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009d90 tx=0x7f0a2800b330 comp rx=0 tx=0).stop 2024-05-08T23:32:45.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.711+0000 7f0a2ffff700 1 -- 172.21.15.81:0/1463356743 shutdown_connections 2024-05-08T23:32:45.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.711+0000 7f0a2ffff700 1 --2- 172.21.15.81:0/1463356743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3006b650 0x7f0a300b3be0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.711+0000 7f0a2ffff700 1 --2- 172.21.15.81:0/1463356743 >> v2:172.21.15.117:3300/0 conn(0x7f0a300aed40 0x7f0a3015e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.711+0000 7f0a2ffff700 1 -- 172.21.15.81:0/1463356743 >> 172.21.15.81:0/1463356743 conn(0x7f0a300b16c0 msgr2=0x7f0a300b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:45.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.711+0000 7f0a2ffff700 1 -- 172.21.15.81:0/1463356743 shutdown_connections 2024-05-08T23:32:45.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.711+0000 7f0a2ffff700 1 -- 172.21.15.81:0/1463356743 wait complete. 2024-05-08T23:32:45.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.711+0000 7f0a2ffff700 1 Processor -- start 2024-05-08T23:32:45.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.711+0000 7f0a2ffff700 1 -- start start 2024-05-08T23:32:45.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3006b650 0x7f0a3013d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a300aed40 0x7f0a30156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a300b3750 0x7f0a3014cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2ffff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a30140460 con 0x7f0a3006b650 2024-05-08T23:32:45.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2ffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3015ecf0 con 0x7f0a300aed40 2024-05-08T23:32:45.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3006b650 0x7f0a3013d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3015efd0 con 0x7f0a300b3750 2024-05-08T23:32:45.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3006b650 0x7f0a3013d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34406/0 (socket says 172.21.15.81:34406) 2024-05-08T23:32:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2effd700 1 -- 172.21.15.81:0/1508209828 learned_addr learned my addr 172.21.15.81:0/1508209828 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2e7fc700 1 --2- 172.21.15.81:0/1508209828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a300aed40 0x7f0a30156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2f7fe700 1 --2- 172.21.15.81:0/1508209828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a300b3750 0x7f0a3014cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2effd700 1 -- 172.21.15.81:0/1508209828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a300b3750 msgr2=0x7f0a3014cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2effd700 1 --2- 172.21.15.81:0/1508209828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a300b3750 0x7f0a3014cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2effd700 1 -- 172.21.15.81:0/1508209828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a300aed40 msgr2=0x7f0a30156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2effd700 1 --2- 172.21.15.81:0/1508209828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a300aed40 0x7f0a30156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2effd700 1 -- 172.21.15.81:0/1508209828 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a28009a70 con 0x7f0a3006b650 2024-05-08T23:32:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2effd700 1 --2- 172.21.15.81:0/1508209828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3006b650 0x7f0a3013d5d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000e840 tx=0x7f0a2000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a27fff700 1 -- 172.21.15.81:0/1508209828 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a200095a0 con 0x7f0a3006b650 2024-05-08T23:32:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a27fff700 1 -- 172.21.15.81:0/1508209828 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a20013680 con 0x7f0a3006b650 2024-05-08T23:32:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2ffff700 1 -- 172.21.15.81:0/1508209828 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3015f2b0 con 0x7f0a3006b650 2024-05-08T23:32:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a27fff700 1 -- 172.21.15.81:0/1508209828 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a200139f0 con 0x7f0a3006b650 2024-05-08T23:32:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.715+0000 7f0a2ffff700 1 -- 172.21.15.81:0/1508209828 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3015f870 con 0x7f0a3006b650 2024-05-08T23:32:45.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.719+0000 7f0a2ffff700 1 -- 172.21.15.81:0/1508209828 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a300684b0 con 0x7f0a3006b650 2024-05-08T23:32:45.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.719+0000 7f0a27fff700 1 -- 172.21.15.81:0/1508209828 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0a20013b50 con 0x7f0a3006b650 2024-05-08T23:32:45.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.719+0000 7f0a27fff700 1 --2- 172.21.15.81:0/1508209828 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0a10066730 0x7f0a10068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:45.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.719+0000 7f0a2e7fc700 1 --2- 172.21.15.81:0/1508209828 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0a10066730 0x7f0a10068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:45.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.719+0000 7f0a27fff700 1 -- 172.21.15.81:0/1508209828 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a20084040 con 0x7f0a3006b650 2024-05-08T23:32:45.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.719+0000 7f0a2e7fc700 1 --2- 172.21.15.81:0/1508209828 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0a10066730 0x7f0a10068bf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0a300b3b80 tx=0x7f0a18007f30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:45.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.727+0000 7f0a27fff700 1 -- 172.21.15.81:0/1508209828 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0a20051310 con 0x7f0a3006b650 2024-05-08T23:32:45.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.963+0000 7f0a2ffff700 1 -- 172.21.15.81:0/1508209828 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0a300acd20 con 0x7f0a3006b650 2024-05-08T23:32:45.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.963+0000 7f0a27fff700 1 -- 172.21.15.81:0/1508209828 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1334 (secure 0 0 0) 0x7f0a20054930 con 0x7f0a3006b650 2024-05-08T23:32:45.967 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:45.967 INFO:teuthology.orchestra.run.smithi081.stdout:{"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":150,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":52,"num_osds":8,"num_up_osds":8,"osd_up_since":1715211025,"num_in_osds":8,"osd_in_since":1715211016,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":41},{"state_name":"active+clean","count":35},{"state_name":"creating+peering","count":21}],"num_pgs":97,"num_pools":3,"num_objects":7,"data_bytes":590514,"bytes_used":235606016,"bytes_avail":772858507264,"bytes_total":773094113280,"unknown_pgs_ratio":0.42268040776252747,"inactive_pgs_ratio":0.2164948433637619,"write_bytes_sec":1563,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":23,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"b","status":"up:active","gid":15311}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-05-08T23:32:33.364767+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:32:45.969 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.967+0000 7f0a2ffff700 1 -- 172.21.15.81:0/1508209828 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0a10066730 msgr2=0x7f0a10068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.969 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.967+0000 7f0a2ffff700 1 --2- 172.21.15.81:0/1508209828 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0a10066730 0x7f0a10068bf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0a300b3b80 tx=0x7f0a18007f30 comp rx=0 tx=0).stop 2024-05-08T23:32:45.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.967+0000 7f0a2ffff700 1 -- 172.21.15.81:0/1508209828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3006b650 msgr2=0x7f0a3013d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:45.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.967+0000 7f0a2ffff700 1 --2- 172.21.15.81:0/1508209828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3006b650 0x7f0a3013d5d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000e840 tx=0x7f0a2000eb50 comp rx=0 tx=0).stop 2024-05-08T23:32:45.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.967+0000 7f0a2ffff700 1 -- 172.21.15.81:0/1508209828 shutdown_connections 2024-05-08T23:32:45.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.967+0000 7f0a2ffff700 1 --2- 172.21.15.81:0/1508209828 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f0a10066730 0x7f0a10068bf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.967+0000 7f0a2ffff700 1 --2- 172.21.15.81:0/1508209828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a300b3750 0x7f0a3014cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.967+0000 7f0a2ffff700 1 --2- 172.21.15.81:0/1508209828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a300aed40 0x7f0a30156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.967+0000 7f0a2ffff700 1 --2- 172.21.15.81:0/1508209828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3006b650 0x7f0a3013d5d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:45.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.967+0000 7f0a2ffff700 1 -- 172.21.15.81:0/1508209828 >> 172.21.15.81:0/1508209828 conn(0x7f0a300b16c0 msgr2=0x7f0a300af120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:45.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.967+0000 7f0a2ffff700 1 -- 172.21.15.81:0/1508209828 shutdown_connections 2024-05-08T23:32:45.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:45.967+0000 7f0a2ffff700 1 -- 172.21.15.81:0/1508209828 wait complete. 2024-05-08T23:32:45.982 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-08T23:32:45.982 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:32:46.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.087+0000 7f2ab882a700 1 Processor -- start 2024-05-08T23:32:46.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab882a700 1 -- start start 2024-05-08T23:32:46.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab882a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aac0e11d0 0x7f2aac0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:46.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab882a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2aac038820 0x7f2aac039f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:46.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab882a700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aac0078e0 con 0x7f2aac051010 2024-05-08T23:32:46.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab882a700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aac006f10 con 0x7f2aac038820 2024-05-08T23:32:46.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab882a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0e15b0 con 0x7f2aac0e11d0 2024-05-08T23:32:46.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab3fff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2aac051010 0x7f2aac04bc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49544/0 (socket says 172.21.15.81:49544) 2024-05-08T23:32:46.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab3fff700 1 -- 172.21.15.81:0/1686268959 learned_addr learned my addr 172.21.15.81:0/1686268959 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:46.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab2ffd700 1 --2- 172.21.15.81:0/1686268959 >> v2:172.21.15.117:3300/0 conn(0x7f2aac038820 0x7f2aac039f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:46.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab37fe700 1 --2- 172.21.15.81:0/1686268959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aac0e11d0 0x7f2aac0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab27fc700 1 -- 172.21.15.81:0/1686268959 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375534078 0 0) 0x7f2ab40681c0 con 0x7f2aac051010 2024-05-08T23:32:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab27fc700 1 -- 172.21.15.81:0/1686268959 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a9c0035d0 con 0x7f2aac051010 2024-05-08T23:32:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab2ffd700 1 -- 172.21.15.81:0/1686268959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aac0e11d0 msgr2=0x7f2aac0073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab2ffd700 1 --2- 172.21.15.81:0/1686268959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aac0e11d0 0x7f2aac0073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab2ffd700 1 -- 172.21.15.81:0/1686268959 >> v1:172.21.15.81:6789/0 conn(0x7f2aac051010 legacy=0x7f2aac04bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab2ffd700 1 -- 172.21.15.81:0/1686268959 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac006300 con 0x7f2aac038820 2024-05-08T23:32:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab2ffd700 1 --2- 172.21.15.81:0/1686268959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac038820 0x7f2aac039f70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009850 tx=0x7f2aa4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7967463370c86a29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab27fc700 1 -- 172.21.15.81:0/1686268959 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4011830 con 0x7f2aac038820 2024-05-08T23:32:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab27fc700 1 -- 172.21.15.81:0/1686268959 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa4011990 con 0x7f2aac038820 2024-05-08T23:32:46.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab27fc700 1 -- 172.21.15.81:0/1686268959 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4011d40 con 0x7f2aac038820 2024-05-08T23:32:46.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab882a700 1 -- 172.21.15.81:0/1686268959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac038820 msgr2=0x7f2aac039f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:46.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab882a700 1 --2- 172.21.15.81:0/1686268959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac038820 0x7f2aac039f70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009850 tx=0x7f2aa4009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:46.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab882a700 1 -- 172.21.15.81:0/1686268959 shutdown_connections 2024-05-08T23:32:46.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab882a700 1 --2- 172.21.15.81:0/1686268959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac038820 0x7f2aac039f70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:46.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab882a700 1 --2- 172.21.15.81:0/1686268959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aac0e11d0 0x7f2aac0073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:46.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab882a700 1 -- 172.21.15.81:0/1686268959 >> 172.21.15.81:0/1686268959 conn(0x7f2aac054b70 msgr2=0x7f2aac054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:46.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab882a700 1 -- 172.21.15.81:0/1686268959 shutdown_connections 2024-05-08T23:32:46.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.091+0000 7f2ab882a700 1 -- 172.21.15.81:0/1686268959 wait complete. 2024-05-08T23:32:46.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab882a700 1 Processor -- start 2024-05-08T23:32:46.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab882a700 1 -- start start 2024-05-08T23:32:46.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab882a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aac038820 0x7f2aac0e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:46.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab882a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac051010 0x7f2aac0d7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:46.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab882a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aac0e11d0 0x7f2aac0d7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:46.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab882a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac033a90 con 0x7f2aac038820 2024-05-08T23:32:46.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab882a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac032e80 con 0x7f2aac051010 2024-05-08T23:32:46.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab882a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac033160 con 0x7f2aac0e11d0 2024-05-08T23:32:46.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aac038820 0x7f2aac0e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:46.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab2ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac051010 0x7f2aac0d7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab2ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac051010 0x7f2aac0d7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:40158/0 (socket says 172.21.15.81:40158) 2024-05-08T23:32:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab2ffd700 1 -- 172.21.15.81:0/1586128086 learned_addr learned my addr 172.21.15.81:0/1586128086 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab3fff700 1 --2- 172.21.15.81:0/1586128086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aac0e11d0 0x7f2aac0d7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab37fe700 1 -- 172.21.15.81:0/1586128086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aac0e11d0 msgr2=0x7f2aac0d7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab37fe700 1 --2- 172.21.15.81:0/1586128086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aac0e11d0 0x7f2aac0d7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab37fe700 1 -- 172.21.15.81:0/1586128086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac051010 msgr2=0x7f2aac0d7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab37fe700 1 --2- 172.21.15.81:0/1586128086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac051010 0x7f2aac0d7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab37fe700 1 -- 172.21.15.81:0/1586128086 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa40094b0 con 0x7f2aac038820 2024-05-08T23:32:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.095+0000 7f2ab2ffd700 1 --2- 172.21.15.81:0/1586128086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac051010 0x7f2aac0d7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.099+0000 7f2ab37fe700 1 --2- 172.21.15.81:0/1586128086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aac038820 0x7f2aac0e3b60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800e900 tx=0x7f2aa800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:46.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.099+0000 7f2a9bfff700 1 -- 172.21.15.81:0/1586128086 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa800c640 con 0x7f2aac038820 2024-05-08T23:32:46.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.099+0000 7f2a9bfff700 1 -- 172.21.15.81:0/1586128086 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa800c7a0 con 0x7f2aac038820 2024-05-08T23:32:46.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.099+0000 7f2ab882a700 1 -- 172.21.15.81:0/1586128086 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aac033440 con 0x7f2aac038820 2024-05-08T23:32:46.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.099+0000 7f2a9bfff700 1 -- 172.21.15.81:0/1586128086 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa800cb10 con 0x7f2aac038820 2024-05-08T23:32:46.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.099+0000 7f2ab882a700 1 -- 172.21.15.81:0/1586128086 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac0e40a0 con 0x7f2aac038820 2024-05-08T23:32:46.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.099+0000 7f2ab882a700 1 -- 172.21.15.81:0/1586128086 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aac0e4290 con 0x7f2aac038820 2024-05-08T23:32:46.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.099+0000 7f2a9bfff700 1 -- 172.21.15.81:0/1586128086 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2aa800cc70 con 0x7f2aac038820 2024-05-08T23:32:46.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.099+0000 7f2a9bfff700 1 --2- 172.21.15.81:0/1586128086 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2aa0066730 0x7f2aa0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:46.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.099+0000 7f2a9bfff700 1 -- 172.21.15.81:0/1586128086 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2aa80840f0 con 0x7f2aac038820 2024-05-08T23:32:46.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.099+0000 7f2ab2ffd700 1 --2- 172.21.15.81:0/1586128086 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2aa0066730 0x7f2aa0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:46.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.103+0000 7f2ab2ffd700 1 --2- 172.21.15.81:0/1586128086 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2aa0066730 0x7f2aa0068bf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2aac02fd30 tx=0x7f2aa401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:46.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.111+0000 7f2a9bfff700 1 -- 172.21.15.81:0/1586128086 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2aa80513d0 con 0x7f2aac038820 2024-05-08T23:32:46.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.275+0000 7f2ab882a700 1 -- 172.21.15.81:0/1586128086 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2aac006ba0 con 0x7f2aa0066730 2024-05-08T23:32:46.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.283+0000 7f2a9bfff700 1 -- 172.21.15.81:0/1586128086 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281151 (secure 0 0 0) 0x7f2aa4011b80 con 0x7f2aa0066730 2024-05-08T23:32:46.289 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:46.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.287+0000 7f2ab882a700 1 -- 172.21.15.81:0/1586128086 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2aa0066730 msgr2=0x7f2aa0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:46.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.287+0000 7f2ab882a700 1 --2- 172.21.15.81:0/1586128086 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2aa0066730 0x7f2aa0068bf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2aac02fd30 tx=0x7f2aa401c040 comp rx=0 tx=0).stop 2024-05-08T23:32:46.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.287+0000 7f2ab882a700 1 -- 172.21.15.81:0/1586128086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aac038820 msgr2=0x7f2aac0e3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:46.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.287+0000 7f2ab882a700 1 --2- 172.21.15.81:0/1586128086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aac038820 0x7f2aac0e3b60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800e900 tx=0x7f2aa800ec10 comp rx=0 tx=0).stop 2024-05-08T23:32:46.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.287+0000 7f2ab882a700 1 -- 172.21.15.81:0/1586128086 shutdown_connections 2024-05-08T23:32:46.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.287+0000 7f2ab882a700 1 --2- 172.21.15.81:0/1586128086 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2aa0066730 0x7f2aa0068bf0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:46.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.287+0000 7f2ab882a700 1 --2- 172.21.15.81:0/1586128086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2aac0e11d0 0x7f2aac0d7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:46.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.287+0000 7f2ab882a700 1 --2- 172.21.15.81:0/1586128086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aac051010 0x7f2aac0d7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:46.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.287+0000 7f2ab882a700 1 --2- 172.21.15.81:0/1586128086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aac038820 0x7f2aac0e3b60 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:46.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.287+0000 7f2ab882a700 1 -- 172.21.15.81:0/1586128086 >> 172.21.15.81:0/1586128086 conn(0x7f2aac054b70 msgr2=0x7f2aac04bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:46.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.287+0000 7f2ab882a700 1 -- 172.21.15.81:0/1586128086 shutdown_connections 2024-05-08T23:32:46.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:46.287+0000 7f2ab882a700 1 -- 172.21.15.81:0/1586128086 wait complete. 2024-05-08T23:32:46.293 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:32:46.302 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-05-08T23:32:45.367756+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590514,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":167,"num_write_kb":2425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230084,"kb_used_data":5728,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":754744636,"statfs":{"total":773094113280,"available":772858507264,"internally_reserved":0,"allocated":5865472,"data_stored":3089824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":229625400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":146,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.894936"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486717+0000","last_change":"2024-05-08T23:32:40.887862+0000","last_active":"2024-05-08T23:32:42.486717+0000","last_peered":"2024-05-08T23:32:42.486717+0000","last_clean":"2024-05-08T23:32:42.486717+0000","last_became_active":"2024-05-08T23:32:40.887591+0000","last_became_peered":"2024-05-08T23:32:40.887591+0000","last_unstale":"2024-05-08T23:32:42.486717+0000","last_undegraded":"2024-05-08T23:32:42.486717+0000","last_fullsized":"2024-05-08T23:32:42.486717+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:14.349815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869810+0000","last_change":"2024-05-08T23:32:40.881814+0000","last_active":"2024-05-08T23:32:41.869810+0000","last_peered":"2024-05-08T23:32:41.869810+0000","last_clean":"2024-05-08T23:32:41.869810+0000","last_became_active":"2024-05-08T23:32:40.879664+0000","last_became_peered":"2024-05-08T23:32:40.879664+0000","last_unstale":"2024-05-08T23:32:41.869810+0000","last_undegraded":"2024-05-08T23:32:41.869810+0000","last_fullsized":"2024-05-08T23:32:41.869810+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:51.917024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486379+0000","last_change":"2024-05-08T23:32:40.886145+0000","last_active":"2024-05-08T23:32:42.486379+0000","last_peered":"2024-05-08T23:32:42.486379+0000","last_clean":"2024-05-08T23:32:42.486379+0000","last_became_active":"2024-05-08T23:32:40.885884+0000","last_became_peered":"2024-05-08T23:32:40.885884+0000","last_unstale":"2024-05-08T23:32:42.486379+0000","last_undegraded":"2024-05-08T23:32:42.486379+0000","last_fullsized":"2024-05-08T23:32:42.486379+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:51.073137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869331+0000","last_change":"2024-05-08T23:32:40.872563+0000","last_active":"2024-05-08T23:32:41.869331+0000","last_peered":"2024-05-08T23:32:41.869331+0000","last_clean":"2024-05-08T23:32:41.869331+0000","last_became_active":"2024-05-08T23:32:40.872068+0000","last_became_peered":"2024-05-08T23:32:40.872068+0000","last_unstale":"2024-05-08T23:32:41.869331+0000","last_undegraded":"2024-05-08T23:32:41.869331+0000","last_fullsized":"2024-05-08T23:32:41.869331+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:33.567285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869928+0000","last_change":"2024-05-08T23:32:40.873779+0000","last_active":"2024-05-08T23:32:41.869928+0000","last_peered":"2024-05-08T23:32:41.869928+0000","last_clean":"2024-05-08T23:32:41.869928+0000","last_became_active":"2024-05-08T23:32:40.873035+0000","last_became_peered":"2024-05-08T23:32:40.873035+0000","last_unstale":"2024-05-08T23:32:41.869928+0000","last_undegraded":"2024-05-08T23:32:41.869928+0000","last_fullsized":"2024-05-08T23:32:41.869928+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:38.166519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485500+0000","last_change":"2024-05-08T23:32:40.885270+0000","last_active":"2024-05-08T23:32:42.485500+0000","last_peered":"2024-05-08T23:32:42.485500+0000","last_clean":"2024-05-08T23:32:42.485500+0000","last_became_active":"2024-05-08T23:32:40.884990+0000","last_became_peered":"2024-05-08T23:32:40.884990+0000","last_unstale":"2024-05-08T23:32:42.485500+0000","last_undegraded":"2024-05-08T23:32:42.485500+0000","last_fullsized":"2024-05-08T23:32:42.485500+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:34.323573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870291+0000","last_change":"2024-05-08T23:32:40.883179+0000","last_active":"2024-05-08T23:32:41.870291+0000","last_peered":"2024-05-08T23:32:41.870291+0000","last_clean":"2024-05-08T23:32:41.870291+0000","last_became_active":"2024-05-08T23:32:40.882263+0000","last_became_peered":"2024-05-08T23:32:40.882263+0000","last_unstale":"2024-05-08T23:32:41.870291+0000","last_undegraded":"2024-05-08T23:32:41.870291+0000","last_fullsized":"2024-05-08T23:32:41.870291+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:52.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486835+0000","last_change":"2024-05-08T23:32:40.887882+0000","last_active":"2024-05-08T23:32:42.486835+0000","last_peered":"2024-05-08T23:32:42.486835+0000","last_clean":"2024-05-08T23:32:42.486835+0000","last_became_active":"2024-05-08T23:32:40.887591+0000","last_became_peered":"2024-05-08T23:32:40.887591+0000","last_unstale":"2024-05-08T23:32:42.486835+0000","last_undegraded":"2024-05-08T23:32:42.486835+0000","last_fullsized":"2024-05-08T23:32:42.486835+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:03.982050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869905+0000","last_change":"2024-05-08T23:32:40.872252+0000","last_active":"2024-05-08T23:32:41.869905+0000","last_peered":"2024-05-08T23:32:41.869905+0000","last_clean":"2024-05-08T23:32:41.869905+0000","last_became_active":"2024-05-08T23:32:40.872011+0000","last_became_peered":"2024-05-08T23:32:40.872011+0000","last_unstale":"2024-05-08T23:32:41.869905+0000","last_undegraded":"2024-05-08T23:32:41.869905+0000","last_fullsized":"2024-05-08T23:32:41.869905+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:59.249962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869149+0000","last_change":"2024-05-08T23:32:40.871672+0000","last_active":"2024-05-08T23:32:41.869149+0000","last_peered":"2024-05-08T23:32:41.869149+0000","last_clean":"2024-05-08T23:32:41.869149+0000","last_became_active":"2024-05-08T23:32:40.871428+0000","last_became_peered":"2024-05-08T23:32:40.871428+0000","last_unstale":"2024-05-08T23:32:41.869149+0000","last_undegraded":"2024-05-08T23:32:41.869149+0000","last_fullsized":"2024-05-08T23:32:41.869149+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:51.073137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870558+0000","last_change":"2024-05-08T23:32:40.882242+0000","last_active":"2024-05-08T23:32:41.870558+0000","last_peered":"2024-05-08T23:32:41.870558+0000","last_clean":"2024-05-08T23:32:41.870558+0000","last_became_active":"2024-05-08T23:32:40.881992+0000","last_became_peered":"2024-05-08T23:32:40.881992+0000","last_unstale":"2024-05-08T23:32:41.870558+0000","last_undegraded":"2024-05-08T23:32:41.870558+0000","last_fullsized":"2024-05-08T23:32:41.870558+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:33.892287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869488+0000","last_change":"2024-05-08T23:32:40.882835+0000","last_active":"2024-05-08T23:32:41.869488+0000","last_peered":"2024-05-08T23:32:41.869488+0000","last_clean":"2024-05-08T23:32:41.869488+0000","last_became_active":"2024-05-08T23:32:40.882284+0000","last_became_peered":"2024-05-08T23:32:40.882284+0000","last_unstale":"2024-05-08T23:32:41.869488+0000","last_undegraded":"2024-05-08T23:32:41.869488+0000","last_fullsized":"2024-05-08T23:32:41.869488+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:22.400514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485731+0000","last_change":"2024-05-08T23:32:40.884162+0000","last_active":"2024-05-08T23:32:42.485731+0000","last_peered":"2024-05-08T23:32:42.485731+0000","last_clean":"2024-05-08T23:32:42.485731+0000","last_became_active":"2024-05-08T23:32:40.883891+0000","last_became_peered":"2024-05-08T23:32:40.883891+0000","last_unstale":"2024-05-08T23:32:42.485731+0000","last_undegraded":"2024-05-08T23:32:42.485731+0000","last_fullsized":"2024-05-08T23:32:42.485731+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:22.400514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869593+0000","last_change":"2024-05-08T23:32:40.872374+0000","last_active":"2024-05-08T23:32:41.869593+0000","last_peered":"2024-05-08T23:32:41.869593+0000","last_clean":"2024-05-08T23:32:41.869593+0000","last_became_active":"2024-05-08T23:32:40.871713+0000","last_became_peered":"2024-05-08T23:32:40.871713+0000","last_unstale":"2024-05-08T23:32:41.869593+0000","last_undegraded":"2024-05-08T23:32:41.869593+0000","last_fullsized":"2024-05-08T23:32:41.869593+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:03.982050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485613+0000","last_change":"2024-05-08T23:32:40.888222+0000","last_active":"2024-05-08T23:32:42.485613+0000","last_peered":"2024-05-08T23:32:42.485613+0000","last_clean":"2024-05-08T23:32:42.485613+0000","last_became_active":"2024-05-08T23:32:40.887649+0000","last_became_peered":"2024-05-08T23:32:40.887649+0000","last_unstale":"2024-05-08T23:32:42.485613+0000","last_undegraded":"2024-05-08T23:32:42.485613+0000","last_fullsized":"2024-05-08T23:32:42.485613+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:56.942715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.482923+0000","last_change":"2024-05-08T23:32:39.482923+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.482923+0000","last_undegraded":"2024-05-08T23:32:39.482923+0000","last_fullsized":"2024-05-08T23:32:39.482923+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:13.825466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.479952+0000","last_change":"2024-05-08T23:32:39.479952+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.479952+0000","last_undegraded":"2024-05-08T23:32:39.479952+0000","last_fullsized":"2024-05-08T23:32:39.479952+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:43.629558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.476540+0000","last_change":"2024-05-08T23:32:39.476540+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.476540+0000","last_undegraded":"2024-05-08T23:32:39.476540+0000","last_fullsized":"2024-05-08T23:32:39.476540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:39.770766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481883+0000","last_change":"2024-05-08T23:32:39.481883+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481883+0000","last_undegraded":"2024-05-08T23:32:39.481883+0000","last_fullsized":"2024-05-08T23:32:39.481883+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:12.523765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869722+0000","last_change":"2024-05-08T23:32:40.503223+0000","last_active":"2024-05-08T23:32:41.869722+0000","last_peered":"2024-05-08T23:32:41.869722+0000","last_clean":"2024-05-08T23:32:41.869722+0000","last_became_active":"2024-05-08T23:32:40.502907+0000","last_became_peered":"2024-05-08T23:32:40.502907+0000","last_unstale":"2024-05-08T23:32:41.869722+0000","last_undegraded":"2024-05-08T23:32:41.869722+0000","last_fullsized":"2024-05-08T23:32:41.869722+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:43.304902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486163+0000","last_change":"2024-05-08T23:32:40.884604+0000","last_active":"2024-05-08T23:32:42.486163+0000","last_peered":"2024-05-08T23:32:42.486163+0000","last_clean":"2024-05-08T23:32:42.486163+0000","last_became_active":"2024-05-08T23:32:40.884337+0000","last_became_peered":"2024-05-08T23:32:40.884337+0000","last_unstale":"2024-05-08T23:32:42.486163+0000","last_undegraded":"2024-05-08T23:32:42.486163+0000","last_fullsized":"2024-05-08T23:32:42.486163+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:44.757107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477601+0000","last_change":"2024-05-08T23:32:39.477601+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477601+0000","last_undegraded":"2024-05-08T23:32:39.477601+0000","last_fullsized":"2024-05-08T23:32:39.477601+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:41.416678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481005+0000","last_change":"2024-05-08T23:32:39.481005+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481005+0000","last_undegraded":"2024-05-08T23:32:39.481005+0000","last_fullsized":"2024-05-08T23:32:39.481005+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:14.674732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.487020+0000","last_change":"2024-05-08T23:32:40.885545+0000","last_active":"2024-05-08T23:32:42.487020+0000","last_peered":"2024-05-08T23:32:42.487020+0000","last_clean":"2024-05-08T23:32:42.487020+0000","last_became_active":"2024-05-08T23:32:40.885195+0000","last_became_peered":"2024-05-08T23:32:40.885195+0000","last_unstale":"2024-05-08T23:32:42.487020+0000","last_undegraded":"2024-05-08T23:32:42.487020+0000","last_fullsized":"2024-05-08T23:32:42.487020+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:33.567285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869218+0000","last_change":"2024-05-08T23:32:40.500395+0000","last_active":"2024-05-08T23:32:41.869218+0000","last_peered":"2024-05-08T23:32:41.869218+0000","last_clean":"2024-05-08T23:32:41.869218+0000","last_became_active":"2024-05-08T23:32:40.500130+0000","last_became_peered":"2024-05-08T23:32:40.500130+0000","last_unstale":"2024-05-08T23:32:41.869218+0000","last_undegraded":"2024-05-08T23:32:41.869218+0000","last_fullsized":"2024-05-08T23:32:41.869218+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:08.134598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.487136+0000","last_change":"2024-05-08T23:32:40.502985+0000","last_active":"2024-05-08T23:32:42.487136+0000","last_peered":"2024-05-08T23:32:42.487136+0000","last_clean":"2024-05-08T23:32:42.487136+0000","last_became_active":"2024-05-08T23:32:40.502677+0000","last_became_peered":"2024-05-08T23:32:40.502677+0000","last_unstale":"2024-05-08T23:32:42.487136+0000","last_undegraded":"2024-05-08T23:32:42.487136+0000","last_fullsized":"2024-05-08T23:32:42.487136+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:26.160105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477408+0000","last_change":"2024-05-08T23:32:39.477408+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477408+0000","last_undegraded":"2024-05-08T23:32:39.477408+0000","last_fullsized":"2024-05-08T23:32:39.477408+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:31.833826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870114+0000","last_change":"2024-05-08T23:32:40.877145+0000","last_active":"2024-05-08T23:32:41.870114+0000","last_peered":"2024-05-08T23:32:41.870114+0000","last_clean":"2024-05-08T23:32:41.870114+0000","last_became_active":"2024-05-08T23:32:40.876105+0000","last_became_peered":"2024-05-08T23:32:40.876105+0000","last_unstale":"2024-05-08T23:32:41.870114+0000","last_undegraded":"2024-05-08T23:32:41.870114+0000","last_fullsized":"2024-05-08T23:32:41.870114+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:44.757107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486576+0000","last_change":"2024-05-08T23:32:40.503254+0000","last_active":"2024-05-08T23:32:42.486576+0000","last_peered":"2024-05-08T23:32:42.486576+0000","last_clean":"2024-05-08T23:32:42.486576+0000","last_became_active":"2024-05-08T23:32:40.503019+0000","last_became_peered":"2024-05-08T23:32:40.503019+0000","last_unstale":"2024-05-08T23:32:42.486576+0000","last_undegraded":"2024-05-08T23:32:42.486576+0000","last_fullsized":"2024-05-08T23:32:42.486576+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:32.059412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481770+0000","last_change":"2024-05-08T23:32:39.481770+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481770+0000","last_undegraded":"2024-05-08T23:32:39.481770+0000","last_fullsized":"2024-05-08T23:32:39.481770+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:24.848996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869846+0000","last_change":"2024-05-08T23:32:40.501242+0000","last_active":"2024-05-08T23:32:41.869846+0000","last_peered":"2024-05-08T23:32:41.869846+0000","last_clean":"2024-05-08T23:32:41.869846+0000","last_became_active":"2024-05-08T23:32:40.500998+0000","last_became_peered":"2024-05-08T23:32:40.500998+0000","last_unstale":"2024-05-08T23:32:41.869846+0000","last_undegraded":"2024-05-08T23:32:41.869846+0000","last_fullsized":"2024-05-08T23:32:41.869846+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:09.047211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870109+0000","last_change":"2024-05-08T23:32:40.873424+0000","last_active":"2024-05-08T23:32:41.870109+0000","last_peered":"2024-05-08T23:32:41.870109+0000","last_clean":"2024-05-08T23:32:41.870109+0000","last_became_active":"2024-05-08T23:32:40.873161+0000","last_became_peered":"2024-05-08T23:32:40.873161+0000","last_unstale":"2024-05-08T23:32:41.870109+0000","last_undegraded":"2024-05-08T23:32:41.870109+0000","last_fullsized":"2024-05-08T23:32:41.870109+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:14.349815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483474+0000","last_change":"2024-05-08T23:32:39.483474+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483474+0000","last_undegraded":"2024-05-08T23:32:39.483474+0000","last_fullsized":"2024-05-08T23:32:39.483474+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:31.286020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485852+0000","last_change":"2024-05-08T23:32:40.499587+0000","last_active":"2024-05-08T23:32:42.485852+0000","last_peered":"2024-05-08T23:32:42.485852+0000","last_clean":"2024-05-08T23:32:42.485852+0000","last_became_active":"2024-05-08T23:32:40.499243+0000","last_became_peered":"2024-05-08T23:32:40.499243+0000","last_unstale":"2024-05-08T23:32:42.485852+0000","last_undegraded":"2024-05-08T23:32:42.485852+0000","last_fullsized":"2024-05-08T23:32:42.485852+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:53.323381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870508+0000","last_change":"2024-05-08T23:32:40.872864+0000","last_active":"2024-05-08T23:32:41.870508+0000","last_peered":"2024-05-08T23:32:41.870508+0000","last_clean":"2024-05-08T23:32:41.870508+0000","last_became_active":"2024-05-08T23:32:40.872596+0000","last_became_peered":"2024-05-08T23:32:40.872596+0000","last_unstale":"2024-05-08T23:32:41.870508+0000","last_undegraded":"2024-05-08T23:32:41.870508+0000","last_fullsized":"2024-05-08T23:32:41.870508+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:18.771018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481540+0000","last_change":"2024-05-08T23:32:39.481540+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481540+0000","last_undegraded":"2024-05-08T23:32:39.481540+0000","last_fullsized":"2024-05-08T23:32:39.481540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:31.133500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870719+0000","last_change":"2024-05-08T23:32:40.499582+0000","last_active":"2024-05-08T23:32:41.870719+0000","last_peered":"2024-05-08T23:32:41.870719+0000","last_clean":"2024-05-08T23:32:41.870719+0000","last_became_active":"2024-05-08T23:32:40.497931+0000","last_became_peered":"2024-05-08T23:32:40.497931+0000","last_unstale":"2024-05-08T23:32:41.870719+0000","last_undegraded":"2024-05-08T23:32:41.870719+0000","last_fullsized":"2024-05-08T23:32:41.870719+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:40.305843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483298+0000","last_change":"2024-05-08T23:32:39.483298+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483298+0000","last_undegraded":"2024-05-08T23:32:39.483298+0000","last_fullsized":"2024-05-08T23:32:39.483298+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:43:57.594449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481124+0000","last_change":"2024-05-08T23:32:39.481124+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481124+0000","last_undegraded":"2024-05-08T23:32:39.481124+0000","last_fullsized":"2024-05-08T23:32:39.481124+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:33:25.871051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483428+0000","last_change":"2024-05-08T23:32:39.483428+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483428+0000","last_undegraded":"2024-05-08T23:32:39.483428+0000","last_fullsized":"2024-05-08T23:32:39.483428+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:06.250791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477222+0000","last_change":"2024-05-08T23:32:39.477222+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477222+0000","last_undegraded":"2024-05-08T23:32:39.477222+0000","last_fullsized":"2024-05-08T23:32:39.477222+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:54.923508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.484968+0000","last_change":"2024-05-08T23:32:39.484968+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.484968+0000","last_undegraded":"2024-05-08T23:32:39.484968+0000","last_fullsized":"2024-05-08T23:32:39.484968+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:18:31.349703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870336+0000","last_change":"2024-05-08T23:32:40.874535+0000","last_active":"2024-05-08T23:32:41.870336+0000","last_peered":"2024-05-08T23:32:41.870336+0000","last_clean":"2024-05-08T23:32:41.870336+0000","last_became_active":"2024-05-08T23:32:40.874316+0000","last_became_peered":"2024-05-08T23:32:40.874316+0000","last_unstale":"2024-05-08T23:32:41.870336+0000","last_undegraded":"2024-05-08T23:32:41.870336+0000","last_fullsized":"2024-05-08T23:32:41.870336+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:56.942715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869562+0000","last_change":"2024-05-08T23:32:40.500526+0000","last_active":"2024-05-08T23:32:41.869562+0000","last_peered":"2024-05-08T23:32:41.869562+0000","last_clean":"2024-05-08T23:32:41.869562+0000","last_became_active":"2024-05-08T23:32:40.500219+0000","last_became_peered":"2024-05-08T23:32:40.500219+0000","last_unstale":"2024-05-08T23:32:41.869562+0000","last_undegraded":"2024-05-08T23:32:41.869562+0000","last_fullsized":"2024-05-08T23:32:41.869562+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:56.869056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.482871+0000","last_change":"2024-05-08T23:32:39.482871+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.482871+0000","last_undegraded":"2024-05-08T23:32:39.482871+0000","last_fullsized":"2024-05-08T23:32:39.482871+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:20:31.576039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481313+0000","last_change":"2024-05-08T23:32:39.481313+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481313+0000","last_undegraded":"2024-05-08T23:32:39.481313+0000","last_fullsized":"2024-05-08T23:32:39.481313+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:21.704760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486777+0000","last_change":"2024-05-08T23:32:40.499075+0000","last_active":"2024-05-08T23:32:42.486777+0000","last_peered":"2024-05-08T23:32:42.486777+0000","last_clean":"2024-05-08T23:32:42.486777+0000","last_became_active":"2024-05-08T23:32:40.498774+0000","last_became_peered":"2024-05-08T23:32:40.498774+0000","last_unstale":"2024-05-08T23:32:42.486777+0000","last_undegraded":"2024-05-08T23:32:42.486777+0000","last_fullsized":"2024-05-08T23:32:42.486777+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:09.302638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869732+0000","last_change":"2024-05-08T23:32:40.499734+0000","last_active":"2024-05-08T23:32:41.869732+0000","last_peered":"2024-05-08T23:32:41.869732+0000","last_clean":"2024-05-08T23:32:41.869732+0000","last_became_active":"2024-05-08T23:32:40.497679+0000","last_became_peered":"2024-05-08T23:32:40.497679+0000","last_unstale":"2024-05-08T23:32:41.869732+0000","last_undegraded":"2024-05-08T23:32:41.869732+0000","last_fullsized":"2024-05-08T23:32:41.869732+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:58.686919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869659+0000","last_change":"2024-05-08T23:32:40.885944+0000","last_active":"2024-05-08T23:32:41.869659+0000","last_peered":"2024-05-08T23:32:41.869659+0000","last_clean":"2024-05-08T23:32:41.869659+0000","last_became_active":"2024-05-08T23:32:40.885687+0000","last_became_peered":"2024-05-08T23:32:40.885687+0000","last_unstale":"2024-05-08T23:32:41.869659+0000","last_undegraded":"2024-05-08T23:32:41.869659+0000","last_fullsized":"2024-05-08T23:32:41.869659+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:33.069307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.483932+0000","last_change":"2024-05-08T23:32:39.483932+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.483932+0000","last_undegraded":"2024-05-08T23:32:39.483932+0000","last_fullsized":"2024-05-08T23:32:39.483932+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:46.898315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.484322+0000","last_change":"2024-05-08T23:32:39.484322+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.484322+0000","last_undegraded":"2024-05-08T23:32:39.484322+0000","last_fullsized":"2024-05-08T23:32:39.484322+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869988+0000","last_change":"2024-05-08T23:32:40.873427+0000","last_active":"2024-05-08T23:32:41.869988+0000","last_peered":"2024-05-08T23:32:41.869988+0000","last_clean":"2024-05-08T23:32:41.869988+0000","last_became_active":"2024-05-08T23:32:40.873194+0000","last_became_peered":"2024-05-08T23:32:40.873194+0000","last_unstale":"2024-05-08T23:32:41.869988+0000","last_undegraded":"2024-05-08T23:32:41.869988+0000","last_fullsized":"2024-05-08T23:32:41.869988+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:34.323573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.477580+0000","last_change":"2024-05-08T23:32:39.477580+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.477580+0000","last_undegraded":"2024-05-08T23:32:39.477580+0000","last_fullsized":"2024-05-08T23:32:39.477580+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:19.747747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-08T23:32:39.481541+0000","last_change":"2024-05-08T23:32:39.481541+0000","last_active":"2024-05-08T23:32:39.461956+0000","last_peered":"2024-05-08T23:32:39.461956+0000","last_clean":"2024-05-08T23:32:39.461956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:39.481541+0000","last_undegraded":"2024-05-08T23:32:39.481541+0000","last_fullsized":"2024-05-08T23:32:39.481541+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:36.991964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T23:32:40.471851+0000","last_change":"2024-05-08T23:32:40.471851+0000","last_active":"2024-05-08T23:32:40.471851+0000","last_peered":"2024-05-08T23:32:40.471851+0000","last_clean":"2024-05-08T23:32:40.471851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T23:32:40.471851+0000","last_undegraded":"2024-05-08T23:32:40.471851+0000","last_fullsized":"2024-05-08T23:32:40.471851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870540+0000","last_change":"2024-05-08T23:32:40.503644+0000","last_active":"2024-05-08T23:32:41.870540+0000","last_peered":"2024-05-08T23:32:41.870540+0000","last_clean":"2024-05-08T23:32:41.870540+0000","last_became_active":"2024-05-08T23:32:40.503283+0000","last_became_peered":"2024-05-08T23:32:40.503283+0000","last_unstale":"2024-05-08T23:32:41.870540+0000","last_undegraded":"2024-05-08T23:32:41.870540+0000","last_fullsized":"2024-05-08T23:32:41.870540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:59.435733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":190,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.876580+0000","last_change":"2024-05-08T23:30:27.226484+0000","last_active":"2024-05-08T23:32:40.876580+0000","last_peered":"2024-05-08T23:32:40.876580+0000","last_clean":"2024-05-08T23:32:40.876580+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:32:40.876580+0000","last_undegraded":"2024-05-08T23:32:40.876580+0000","last_fullsized":"2024-05-08T23:32:40.876580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_clean_scrub_stamp":"2024-05-08T23:30:26.204118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:14.267287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":46,"acting":46,"num_store_stats":2},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":146,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.89400000000000002,"15min":0.90500000000000003},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1719999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.96899999999999997,"15min":0.97899999999999998},"min":{"1min":0.58299999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1910000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.89700000000000002}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.59099999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6579999999999999,"5min":2.004,"15min":2.004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.92600000000000005,"15min":0.92400000000000004},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.327,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.68200000000000005}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88300000000000001,"15min":0.88500000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.95999999999999996,"15min":0.97099999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.218,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.78900000000000003}]},{"osd":3,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.89200000000000002,"15min":0.88400000000000001},"min":{"1min":0.65100000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96899999999999997,"15min":0.97799999999999998},"min":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2290000000000001,"5min":1.272,"15min":1.272},"last":0.85399999999999998}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.71499999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.1439999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0249999999999999,"15min":1.0269999999999999},"min":{"1min":0.67800000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.95299999999999996}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0469999999999999,"15min":1.056},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4450000000000001,"5min":2.109,"15min":2.109},"last":1.006},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1240000000000001,"15min":1.1439999999999999},"min":{"1min":0.75800000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.98099999999999998}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1970000000000001,"15min":1.2070000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5660000000000001,"5min":2.544,"15min":2.544},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.103,"15min":1.109},"min":{"1min":0.81200000000000006,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.0669999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2969999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.97099999999999997,"15min":0.98299999999999998},"min":{"1min":0.63700000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1859999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.65700000000000003}]},{"osd":1,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.90900000000000003,"15min":0.90400000000000003},"min":{"1min":0.67300000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88400000000000001,"15min":0.878},"min":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.0309999999999999}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.91800000000000004,"15min":0.91700000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.385,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.93700000000000006,"15min":0.92500000000000004},"min":{"1min":0.70299999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.454,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.93100000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.97199999999999998,"15min":0.98399999999999999},"min":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.252,"5min":1.395,"15min":1.395},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.004,"5min":0.96099999999999997,"15min":0.95399999999999996},"min":{"1min":0.63700000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.0049999999999999}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.995,"15min":1.0129999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.1339999999999999,"15min":1.1599999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.484,"5min":2.464,"15min":2.464},"last":0.78500000000000003}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0800000000000001,"15min":1.0820000000000001},"min":{"1min":0.82099999999999995,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.871}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1930000000000001,"15min":1.216},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.163,"5min":1.1759999999999999,"15min":1.1779999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.2909999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.84099999999999997,"15min":0.85099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2030000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77900000000000003,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.4910000000000001}]},{"osd":1,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.70999999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1459999999999999,"5min":1.268,"15min":1.268},"last":1.47},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.88800000000000001,"15min":0.90000000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.125,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.0920000000000001}]},{"osd":2,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.88800000000000001,"15min":0.89800000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89900000000000002,"15min":0.90700000000000003},"min":{"1min":0.65400000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0249999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.5669999999999999}]},{"osd":3,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.85299999999999998,"15min":0.84899999999999998},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1759999999999999,"5min":1.23,"15min":1.23},"last":1.3120000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2390000000000001,"5min":1.431,"15min":1.431},"last":1.357}]},{"osd":5,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3420000000000001,"5min":1.413,"15min":1.413},"last":1.7669999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95899999999999996,"15min":0.95099999999999996},"min":{"1min":0.73999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.278,"5min":1.504,"15min":1.504},"last":1.409}]},{"osd":6,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0269999999999999,"15min":1.0309999999999999},"min":{"1min":0.73899999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.52},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1419999999999999,"15min":1.1559999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3859999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.867}]},{"osd":7,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0800000000000001,"15min":1.0840000000000001},"min":{"1min":0.69899999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4379999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.653},{"interface":"front","average":{"1min":1.113,"5min":1.143,"15min":1.1479999999999999},"min":{"1min":0.94799999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4099999999999999,"5min":1.583,"15min":1.583},"last":1.7290000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.49099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.46300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":1.032,"15min":1.032},"last":0.67700000000000005}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70599999999999996,"15min":0.71099999999999997},"min":{"1min":0.50700000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.89300000000000002,"5min":1,"15min":1},"last":0.98899999999999999}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0469999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80200000000000005,"15min":0.79900000000000004},"min":{"1min":0.56699999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0229999999999999,"5min":1.242,"15min":1.242},"last":0.71499999999999997}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.65600000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.129,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.96699999999999997,"15min":0.97399999999999998},"min":{"1min":0.78000000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.244,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.749}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.014,"15min":1.022},"min":{"1min":0.80300000000000005,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.161,"5min":1.623,"15min":1.623},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.995,"15min":1.004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1359999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.89000000000000001}]},{"osd":6,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.997,"15min":1.0009999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.232,"5min":1.482,"15min":1.482},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.093,"15min":1.111},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3480000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.0209999999999999}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0800000000000001,"15min":1.0860000000000001},"min":{"1min":0.78400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.456,"5min":1.514,"15min":1.514},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0580000000000001,"15min":1.0600000000000001},"min":{"1min":0.75,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.2869999999999999,"5min":1.552,"15min":1.552},"last":0.96099999999999997}]}]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80400000000000005,"5min":1.099,"15min":1.099},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0660000000000001,"5min":1.129,"15min":1.129},"last":0.51700000000000002}]},{"osd":1,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70999999999999996,"15min":0.70899999999999996},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0509999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.84799999999999998}]},{"osd":4,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.96799999999999997,"15min":0.97799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.27,"5min":1.367,"15min":1.367},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.95599999999999996,"15min":0.96399999999999997},"min":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98999999999999999,"15min":0.98599999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.3},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.91600000000000004}]},{"osd":6,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0620000000000001,"15min":1.0740000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4239999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.22},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.075,"15min":1.075},"min":{"1min":0.85699999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.381}]},{"osd":7,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1220000000000001,"15min":1.1240000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1000000000000001,"15min":1.1020000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.492,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.0109999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":233776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.51300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.44,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.442}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.628},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.68899999999999995,"15min":0.67000000000000004},"min":{"1min":0.53900000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.39000000000000001}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73499999999999999,"15min":0.72599999999999998},"min":{"1min":0.57999999999999996,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.42799999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.038,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.95099999999999996,"15min":0.95199999999999996},"min":{"1min":0.67600000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.256,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":0.88},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.95499999999999996,"15min":0.94299999999999995},"min":{"1min":0.77300000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.2709999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.70499999999999996}]},{"osd":5,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0109999999999999,"15min":1.0089999999999999},"min":{"1min":0.72299999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.403,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.75},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.077,"15min":1.079},"min":{"1min":0.85499999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3879999999999999,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.81499999999999995}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0369999999999999,"15min":1.034},"min":{"1min":0.80000000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.4330000000000001,"5min":1.633,"15min":1.633},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.095,"5min":1.135,"15min":1.1419999999999999},"min":{"1min":0.80500000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.4750000000000001,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.91800000000000004}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.0209999999999999,"15min":1.0009999999999999},"min":{"1min":0.83099999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.371,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.081,"15min":1.0820000000000001},"min":{"1min":0.84999999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.306,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.80100000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542175,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":843618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.88300000000000001,"15min":0.89700000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.84699999999999998}]},{"osd":1,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.86399999999999999,"15min":0.86699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1379999999999999,"5min":1.359,"15min":1.359},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91200000000000003,"15min":0.91600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.23,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.747}]},{"osd":2,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88,"15min":0.873},"min":{"1min":0.57499999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88,"15min":0.88500000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.133,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.97699999999999998}]},{"osd":3,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.91300000000000003,"15min":0.91100000000000003},"min":{"1min":0.60699999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.99299999999999999,"15min":0.99099999999999999},"min":{"1min":0.80100000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2470000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.94999999999999996}]},{"osd":4,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.90400000000000003,"15min":0.91200000000000003},"min":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.73999999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.1899999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.1080000000000001}]},{"osd":5,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.028,"15min":1.028},"min":{"1min":0.64100000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.405,"5min":1.714,"15min":1.714},"last":1.069},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.1950000000000001,"5min":1.325,"15min":1.325},"last":1.165}]},{"osd":6,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.048,"15min":1.048},"min":{"1min":0.86599999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.4430000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.105,"15min":1.107},"min":{"1min":0.83999999999999997,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.359,"5min":1.496,"15min":1.496},"last":1.151}]}]},{"osd":0,"up_from":14,"seq":60129542175,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":843550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.47499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":1.351},"last":0.71799999999999997}]},{"osd":2,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.51500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.238,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.546,"5min":1.869,"15min":1.869},"last":0.80100000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.84199999999999997,"15min":0.85599999999999998},"min":{"1min":0.55200000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.94199999999999995,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.89300000000000002}]},{"osd":4,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97199999999999998,"15min":0.97299999999999998},"min":{"1min":0.71999999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.357,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.90800000000000003,"15min":0.89600000000000002},"min":{"1min":0.69899999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.746}]},{"osd":5,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.052,"15min":1.0600000000000001},"min":{"1min":0.76800000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3819999999999999,"5min":1.474,"15min":1.474},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.92500000000000004,"15min":0.91400000000000003},"min":{"1min":0.72699999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.98099999999999998}]},{"osd":6,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.056,"15min":1.0720000000000001},"min":{"1min":0.76900000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.171,"5min":1.736,"15min":1.736},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1180000000000001,"15min":1.121},"min":{"1min":0.89000000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4830000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.0009999999999999}]},{"osd":7,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.113,"15min":1.117},"min":{"1min":0.66600000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.397,"5min":1.72,"15min":1.72},"last":1.115},{"interface":"front","average":{"1min":1.097,"5min":1.109,"15min":1.111},"min":{"1min":0.86399999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.1439999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:32:46.789 INFO:tasks.ceph.mds.b.smithi117.stderr:2024-05-08T23:32:46.788+0000 7faf28d89700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:32:49.307 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:32:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.431+0000 7fd980a51700 1 Processor -- start 2024-05-08T23:32:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.431+0000 7fd980a51700 1 -- start start 2024-05-08T23:32:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd980a51700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c0d37c0 0x7fd97c0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd980a51700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97c0d3c20 con 0x7fd97c0d1690 2024-05-08T23:32:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd980a51700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97c0d1460 con 0x7fd97c0b2ea0 2024-05-08T23:32:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd980a51700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0d3e50 con 0x7fd97c0d37c0 2024-05-08T23:32:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd97b7fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd97c0d1690 0x7fd97c0d1a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49548/0 (socket says 172.21.15.81:49548) 2024-05-08T23:32:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd97b7fe700 1 -- 172.21.15.81:0/1688665868 learned_addr learned my addr 172.21.15.81:0/1688665868 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd97affd700 1 --2- 172.21.15.81:0/1688665868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c0d37c0 0x7fd97c0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd97a7fc700 1 -- 172.21.15.81:0/1688665868 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925957579 0 0) 0x7fd97c0d3c20 con 0x7fd97c0d1690 2024-05-08T23:32:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd97a7fc700 1 -- 172.21.15.81:0/1688665868 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9640035d0 con 0x7fd97c0d1690 2024-05-08T23:32:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd97a7fc700 1 -- 172.21.15.81:0/1688665868 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141546687 0 0) 0x7fd97c0d1460 con 0x7fd97c0b2ea0 2024-05-08T23:32:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd97a7fc700 1 -- 172.21.15.81:0/1688665868 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd97c0d3c20 con 0x7fd97c0b2ea0 2024-05-08T23:32:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd97affd700 1 -- 172.21.15.81:0/1688665868 >> v1:172.21.15.117:6789/0 conn(0x7fd97c0b2ea0 legacy=0x7fd97c0b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd97a7fc700 1 -- 172.21.15.81:0/1688665868 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4089280686 0 0) 0x7fd9640035d0 con 0x7fd97c0d1690 2024-05-08T23:32:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd97affd700 1 -- 172.21.15.81:0/1688665868 >> v1:172.21.15.81:6789/0 conn(0x7fd97c0d1690 legacy=0x7fd97c0d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:49.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd97affd700 1 -- 172.21.15.81:0/1688665868 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c0ce870 con 0x7fd97c0d37c0 2024-05-08T23:32:49.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd97affd700 1 --2- 172.21.15.81:0/1688665868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c0d37c0 0x7fd97c0d0f20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009880 tx=0x7fd96c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=905f7b4297b00f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:49.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd97a7fc700 1 -- 172.21.15.81:0/1688665868 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970001670 con 0x7fd97c0d37c0 2024-05-08T23:32:49.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd97a7fc700 1 -- 172.21.15.81:0/1688665868 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96c004030 con 0x7fd97c0d37c0 2024-05-08T23:32:49.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd97a7fc700 1 -- 172.21.15.81:0/1688665868 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0043c0 con 0x7fd97c0d37c0 2024-05-08T23:32:49.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd980a51700 1 -- 172.21.15.81:0/1688665868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c0d37c0 msgr2=0x7fd97c0d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:49.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd980a51700 1 --2- 172.21.15.81:0/1688665868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c0d37c0 0x7fd97c0d0f20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009880 tx=0x7fd96c009b90 comp rx=0 tx=0).stop 2024-05-08T23:32:49.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.435+0000 7fd980a51700 1 -- 172.21.15.81:0/1688665868 shutdown_connections 2024-05-08T23:32:49.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.439+0000 7fd980a51700 1 --2- 172.21.15.81:0/1688665868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c0d37c0 0x7fd97c0d0f20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:49.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.439+0000 7fd980a51700 1 -- 172.21.15.81:0/1688665868 >> 172.21.15.81:0/1688665868 conn(0x7fd97c09c3c0 msgr2=0x7fd97c0d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:49.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.439+0000 7fd980a51700 1 -- 172.21.15.81:0/1688665868 shutdown_connections 2024-05-08T23:32:49.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.439+0000 7fd980a51700 1 -- 172.21.15.81:0/1688665868 wait complete. 2024-05-08T23:32:49.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.439+0000 7fd980a51700 1 Processor -- start 2024-05-08T23:32:49.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.439+0000 7fd980a51700 1 -- start start 2024-05-08T23:32:49.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd980a51700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c0b2ea0 0x7fd97c0b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:49.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd980a51700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd97c0d1690 0x7fd97c06bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:49.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd980a51700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd97c0d37c0 0x7fd97c06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:49.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd980a51700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0b76c0 con 0x7fd97c0d1690 2024-05-08T23:32:49.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd980a51700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0b79a0 con 0x7fd97c0d37c0 2024-05-08T23:32:49.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd980a51700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0b7c80 con 0x7fd97c0b2ea0 2024-05-08T23:32:49.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd97affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd97c0d1690 0x7fd97c06bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:49.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd97affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd97c0d1690 0x7fd97c06bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56452/0 (socket says 172.21.15.81:56452) 2024-05-08T23:32:49.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd97affd700 1 -- 172.21.15.81:0/3632298810 learned_addr learned my addr 172.21.15.81:0/3632298810 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:49.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd97b7fe700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c0b2ea0 0x7fd97c0b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:49.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd97bfff700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd97c0d37c0 0x7fd97c06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:49.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd97affd700 1 -- 172.21.15.81:0/3632298810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c0b2ea0 msgr2=0x7fd97c0b6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:49.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd97affd700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c0b2ea0 0x7fd97c0b6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:49.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd97affd700 1 -- 172.21.15.81:0/3632298810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd97c0d37c0 msgr2=0x7fd97c06c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:49.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd97affd700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd97c0d37c0 0x7fd97c06c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:49.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd97affd700 1 -- 172.21.15.81:0/3632298810 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c0094b0 con 0x7fd97c0d1690 2024-05-08T23:32:49.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd97bfff700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd97c0d37c0 0x7fd97c06c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:49.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd97b7fe700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c0b2ea0 0x7fd97c0b6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:49.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd97affd700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd97c0d1690 0x7fd97c06bf70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd96c014040 tx=0x7fd96c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:49.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd963fff700 1 -- 172.21.15.81:0/3632298810 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c003d90 con 0x7fd97c0d1690 2024-05-08T23:32:49.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd963fff700 1 -- 172.21.15.81:0/3632298810 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96c011630 con 0x7fd97c0d1690 2024-05-08T23:32:49.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd963fff700 1 -- 172.21.15.81:0/3632298810 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0119e0 con 0x7fd97c0d1690 2024-05-08T23:32:49.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd980a51700 1 -- 172.21.15.81:0/3632298810 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c0b7f60 con 0x7fd97c0d1690 2024-05-08T23:32:49.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd980a51700 1 -- 172.21.15.81:0/3632298810 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97c0b73b0 con 0x7fd97c0d1690 2024-05-08T23:32:49.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.443+0000 7fd980a51700 1 -- 172.21.15.81:0/3632298810 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97c0b8480 con 0x7fd97c0d1690 2024-05-08T23:32:49.447 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.447+0000 7fd963fff700 1 -- 172.21.15.81:0/3632298810 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd96c018050 con 0x7fd97c0d1690 2024-05-08T23:32:49.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.447+0000 7fd963fff700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd968066730 0x7fd968068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:49.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.447+0000 7fd963fff700 1 -- 172.21.15.81:0/3632298810 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd96c084bc0 con 0x7fd97c0d1690 2024-05-08T23:32:49.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.447+0000 7fd97b7fe700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd968066730 0x7fd968068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:49.449 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.447+0000 7fd97b7fe700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd968066730 0x7fd968068bf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd9700017a0 tx=0x7fd970001550 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:49.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.451+0000 7fd963fff700 1 -- 172.21.15.81:0/3632298810 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd97c0b8480 con 0x7fd97c0d1690 2024-05-08T23:32:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.623+0000 7fd980a51700 1 -- 172.21.15.81:0/3632298810 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd97c0b7b10 con 0x7fd968066730 2024-05-08T23:32:49.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.631+0000 7fd963fff700 1 -- 172.21.15.81:0/3632298810 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288213 (secure 0 0 0) 0x7fd97c0b7b10 con 0x7fd968066730 2024-05-08T23:32:49.637 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.635+0000 7fd980a51700 1 -- 172.21.15.81:0/3632298810 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd968066730 msgr2=0x7fd968068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.635+0000 7fd980a51700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd968066730 0x7fd968068bf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd9700017a0 tx=0x7fd970001550 comp rx=0 tx=0).stop 2024-05-08T23:32:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.635+0000 7fd980a51700 1 -- 172.21.15.81:0/3632298810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd97c0d1690 msgr2=0x7fd97c06bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.635+0000 7fd980a51700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd97c0d1690 0x7fd97c06bf70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd96c014040 tx=0x7fd96c004770 comp rx=0 tx=0).stop 2024-05-08T23:32:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.635+0000 7fd980a51700 1 -- 172.21.15.81:0/3632298810 shutdown_connections 2024-05-08T23:32:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.635+0000 7fd980a51700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd968066730 0x7fd968068bf0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.635+0000 7fd980a51700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd97c0d37c0 0x7fd97c06c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.635+0000 7fd980a51700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd97c0d1690 0x7fd97c06bf70 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.635+0000 7fd980a51700 1 --2- 172.21.15.81:0/3632298810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c0b2ea0 0x7fd97c0b6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.635+0000 7fd980a51700 1 -- 172.21.15.81:0/3632298810 >> 172.21.15.81:0/3632298810 conn(0x7fd97c09c3c0 msgr2=0x7fd97c0b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.635+0000 7fd980a51700 1 -- 172.21.15.81:0/3632298810 shutdown_connections 2024-05-08T23:32:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.635+0000 7fd980a51700 1 -- 172.21.15.81:0/3632298810 wait complete. 2024-05-08T23:32:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:32:49.650 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-05-08T23:32:49.369130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230356,"kb_used_data":5992,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":754744364,"statfs":{"total":773094113280,"available":772858228736,"internally_reserved":0,"allocated":6135808,"data_stored":3210468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":229625400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.512813"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.484723+0000","last_change":"2024-05-08T23:32:40.882514+0000","last_active":"2024-05-08T23:32:42.484723+0000","last_peered":"2024-05-08T23:32:42.484723+0000","last_clean":"2024-05-08T23:32:42.484723+0000","last_became_active":"2024-05-08T23:32:40.882268+0000","last_became_peered":"2024-05-08T23:32:40.882268+0000","last_unstale":"2024-05-08T23:32:42.484723+0000","last_undegraded":"2024-05-08T23:32:42.484723+0000","last_fullsized":"2024-05-08T23:32:42.484723+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:37.875587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486717+0000","last_change":"2024-05-08T23:32:40.887862+0000","last_active":"2024-05-08T23:32:42.486717+0000","last_peered":"2024-05-08T23:32:42.486717+0000","last_clean":"2024-05-08T23:32:42.486717+0000","last_became_active":"2024-05-08T23:32:40.887591+0000","last_became_peered":"2024-05-08T23:32:40.887591+0000","last_unstale":"2024-05-08T23:32:42.486717+0000","last_undegraded":"2024-05-08T23:32:42.486717+0000","last_fullsized":"2024-05-08T23:32:42.486717+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:14.349815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869810+0000","last_change":"2024-05-08T23:32:40.881814+0000","last_active":"2024-05-08T23:32:41.869810+0000","last_peered":"2024-05-08T23:32:41.869810+0000","last_clean":"2024-05-08T23:32:41.869810+0000","last_became_active":"2024-05-08T23:32:40.879664+0000","last_became_peered":"2024-05-08T23:32:40.879664+0000","last_unstale":"2024-05-08T23:32:41.869810+0000","last_undegraded":"2024-05-08T23:32:41.869810+0000","last_fullsized":"2024-05-08T23:32:41.869810+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:51.917024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486379+0000","last_change":"2024-05-08T23:32:40.886145+0000","last_active":"2024-05-08T23:32:42.486379+0000","last_peered":"2024-05-08T23:32:42.486379+0000","last_clean":"2024-05-08T23:32:42.486379+0000","last_became_active":"2024-05-08T23:32:40.885884+0000","last_became_peered":"2024-05-08T23:32:40.885884+0000","last_unstale":"2024-05-08T23:32:42.486379+0000","last_undegraded":"2024-05-08T23:32:42.486379+0000","last_fullsized":"2024-05-08T23:32:42.486379+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:51.073137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.872424+0000","last_change":"2024-05-08T23:32:40.880106+0000","last_active":"2024-05-08T23:32:41.872424+0000","last_peered":"2024-05-08T23:32:41.872424+0000","last_clean":"2024-05-08T23:32:41.872424+0000","last_became_active":"2024-05-08T23:32:40.879560+0000","last_became_peered":"2024-05-08T23:32:40.879560+0000","last_unstale":"2024-05-08T23:32:41.872424+0000","last_undegraded":"2024-05-08T23:32:41.872424+0000","last_fullsized":"2024-05-08T23:32:41.872424+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:44.639453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485492+0000","last_change":"2024-05-08T23:32:40.878797+0000","last_active":"2024-05-08T23:32:42.485492+0000","last_peered":"2024-05-08T23:32:42.485492+0000","last_clean":"2024-05-08T23:32:42.485492+0000","last_became_active":"2024-05-08T23:32:40.878320+0000","last_became_peered":"2024-05-08T23:32:40.878320+0000","last_unstale":"2024-05-08T23:32:42.485492+0000","last_undegraded":"2024-05-08T23:32:42.485492+0000","last_fullsized":"2024-05-08T23:32:42.485492+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:01.591244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869331+0000","last_change":"2024-05-08T23:32:40.872563+0000","last_active":"2024-05-08T23:32:41.869331+0000","last_peered":"2024-05-08T23:32:41.869331+0000","last_clean":"2024-05-08T23:32:41.869331+0000","last_became_active":"2024-05-08T23:32:40.872068+0000","last_became_peered":"2024-05-08T23:32:40.872068+0000","last_unstale":"2024-05-08T23:32:41.869331+0000","last_undegraded":"2024-05-08T23:32:41.869331+0000","last_fullsized":"2024-05-08T23:32:41.869331+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:33.567285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.483888+0000","last_change":"2024-05-08T23:32:40.888692+0000","last_active":"2024-05-08T23:32:42.483888+0000","last_peered":"2024-05-08T23:32:42.483888+0000","last_clean":"2024-05-08T23:32:42.483888+0000","last_became_active":"2024-05-08T23:32:40.888361+0000","last_became_peered":"2024-05-08T23:32:40.888361+0000","last_unstale":"2024-05-08T23:32:42.483888+0000","last_undegraded":"2024-05-08T23:32:42.483888+0000","last_fullsized":"2024-05-08T23:32:42.483888+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:39:25.077641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.484747+0000","last_change":"2024-05-08T23:32:40.888029+0000","last_active":"2024-05-08T23:32:42.484747+0000","last_peered":"2024-05-08T23:32:42.484747+0000","last_clean":"2024-05-08T23:32:42.484747+0000","last_became_active":"2024-05-08T23:32:40.881948+0000","last_became_peered":"2024-05-08T23:32:40.881948+0000","last_unstale":"2024-05-08T23:32:42.484747+0000","last_undegraded":"2024-05-08T23:32:42.484747+0000","last_fullsized":"2024-05-08T23:32:42.484747+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:57.202892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.484779+0000","last_change":"2024-05-08T23:32:40.874407+0000","last_active":"2024-05-08T23:32:42.484779+0000","last_peered":"2024-05-08T23:32:42.484779+0000","last_clean":"2024-05-08T23:32:42.484779+0000","last_became_active":"2024-05-08T23:32:40.874145+0000","last_became_peered":"2024-05-08T23:32:40.874145+0000","last_unstale":"2024-05-08T23:32:42.484779+0000","last_undegraded":"2024-05-08T23:32:42.484779+0000","last_fullsized":"2024-05-08T23:32:42.484779+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:55.391720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.871501+0000","last_change":"2024-05-08T23:32:40.880800+0000","last_active":"2024-05-08T23:32:41.871501+0000","last_peered":"2024-05-08T23:32:41.871501+0000","last_clean":"2024-05-08T23:32:41.871501+0000","last_became_active":"2024-05-08T23:32:40.880487+0000","last_became_peered":"2024-05-08T23:32:40.880487+0000","last_unstale":"2024-05-08T23:32:41.871501+0000","last_undegraded":"2024-05-08T23:32:41.871501+0000","last_fullsized":"2024-05-08T23:32:41.871501+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:32.295915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.484502+0000","last_change":"2024-05-08T23:32:40.885523+0000","last_active":"2024-05-08T23:32:42.484502+0000","last_peered":"2024-05-08T23:32:42.484502+0000","last_clean":"2024-05-08T23:32:42.484502+0000","last_became_active":"2024-05-08T23:32:40.885219+0000","last_became_peered":"2024-05-08T23:32:40.885219+0000","last_unstale":"2024-05-08T23:32:42.484502+0000","last_undegraded":"2024-05-08T23:32:42.484502+0000","last_fullsized":"2024-05-08T23:32:42.484502+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:49:34.768485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.484782+0000","last_change":"2024-05-08T23:32:40.880375+0000","last_active":"2024-05-08T23:32:42.484782+0000","last_peered":"2024-05-08T23:32:42.484782+0000","last_clean":"2024-05-08T23:32:42.484782+0000","last_became_active":"2024-05-08T23:32:40.879649+0000","last_became_peered":"2024-05-08T23:32:40.879649+0000","last_unstale":"2024-05-08T23:32:42.484782+0000","last_undegraded":"2024-05-08T23:32:42.484782+0000","last_fullsized":"2024-05-08T23:32:42.484782+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:18.161378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869928+0000","last_change":"2024-05-08T23:32:40.873779+0000","last_active":"2024-05-08T23:32:41.869928+0000","last_peered":"2024-05-08T23:32:41.869928+0000","last_clean":"2024-05-08T23:32:41.869928+0000","last_became_active":"2024-05-08T23:32:40.873035+0000","last_became_peered":"2024-05-08T23:32:40.873035+0000","last_unstale":"2024-05-08T23:32:41.869928+0000","last_undegraded":"2024-05-08T23:32:41.869928+0000","last_fullsized":"2024-05-08T23:32:41.869928+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:38.166519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485500+0000","last_change":"2024-05-08T23:32:40.885270+0000","last_active":"2024-05-08T23:32:42.485500+0000","last_peered":"2024-05-08T23:32:42.485500+0000","last_clean":"2024-05-08T23:32:42.485500+0000","last_became_active":"2024-05-08T23:32:40.884990+0000","last_became_peered":"2024-05-08T23:32:40.884990+0000","last_unstale":"2024-05-08T23:32:42.485500+0000","last_undegraded":"2024-05-08T23:32:42.485500+0000","last_fullsized":"2024-05-08T23:32:42.485500+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:34.323573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.484484+0000","last_change":"2024-05-08T23:32:40.879909+0000","last_active":"2024-05-08T23:32:42.484484+0000","last_peered":"2024-05-08T23:32:42.484484+0000","last_clean":"2024-05-08T23:32:42.484484+0000","last_became_active":"2024-05-08T23:32:40.879679+0000","last_became_peered":"2024-05-08T23:32:40.879679+0000","last_unstale":"2024-05-08T23:32:42.484484+0000","last_undegraded":"2024-05-08T23:32:42.484484+0000","last_fullsized":"2024-05-08T23:32:42.484484+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:52.811144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870291+0000","last_change":"2024-05-08T23:32:40.883179+0000","last_active":"2024-05-08T23:32:41.870291+0000","last_peered":"2024-05-08T23:32:41.870291+0000","last_clean":"2024-05-08T23:32:41.870291+0000","last_became_active":"2024-05-08T23:32:40.882263+0000","last_became_peered":"2024-05-08T23:32:40.882263+0000","last_unstale":"2024-05-08T23:32:41.870291+0000","last_undegraded":"2024-05-08T23:32:41.870291+0000","last_fullsized":"2024-05-08T23:32:41.870291+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:52.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486835+0000","last_change":"2024-05-08T23:32:40.887882+0000","last_active":"2024-05-08T23:32:42.486835+0000","last_peered":"2024-05-08T23:32:42.486835+0000","last_clean":"2024-05-08T23:32:42.486835+0000","last_became_active":"2024-05-08T23:32:40.887591+0000","last_became_peered":"2024-05-08T23:32:40.887591+0000","last_unstale":"2024-05-08T23:32:42.486835+0000","last_undegraded":"2024-05-08T23:32:42.486835+0000","last_fullsized":"2024-05-08T23:32:42.486835+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:03.982050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869905+0000","last_change":"2024-05-08T23:32:40.872252+0000","last_active":"2024-05-08T23:32:41.869905+0000","last_peered":"2024-05-08T23:32:41.869905+0000","last_clean":"2024-05-08T23:32:41.869905+0000","last_became_active":"2024-05-08T23:32:40.872011+0000","last_became_peered":"2024-05-08T23:32:40.872011+0000","last_unstale":"2024-05-08T23:32:41.869905+0000","last_undegraded":"2024-05-08T23:32:41.869905+0000","last_fullsized":"2024-05-08T23:32:41.869905+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:59.249962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485770+0000","last_change":"2024-05-08T23:32:40.880445+0000","last_active":"2024-05-08T23:32:42.485770+0000","last_peered":"2024-05-08T23:32:42.485770+0000","last_clean":"2024-05-08T23:32:42.485770+0000","last_became_active":"2024-05-08T23:32:40.880189+0000","last_became_peered":"2024-05-08T23:32:40.880189+0000","last_unstale":"2024-05-08T23:32:42.485770+0000","last_undegraded":"2024-05-08T23:32:42.485770+0000","last_fullsized":"2024-05-08T23:32:42.485770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:13.786756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485047+0000","last_change":"2024-05-08T23:32:40.879167+0000","last_active":"2024-05-08T23:32:42.485047+0000","last_peered":"2024-05-08T23:32:42.485047+0000","last_clean":"2024-05-08T23:32:42.485047+0000","last_became_active":"2024-05-08T23:32:40.878862+0000","last_became_peered":"2024-05-08T23:32:40.878862+0000","last_unstale":"2024-05-08T23:32:42.485047+0000","last_undegraded":"2024-05-08T23:32:42.485047+0000","last_fullsized":"2024-05-08T23:32:42.485047+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:01:36.805569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.483828+0000","last_change":"2024-05-08T23:32:40.881656+0000","last_active":"2024-05-08T23:32:42.483828+0000","last_peered":"2024-05-08T23:32:42.483828+0000","last_clean":"2024-05-08T23:32:42.483828+0000","last_became_active":"2024-05-08T23:32:40.881386+0000","last_became_peered":"2024-05-08T23:32:40.881386+0000","last_unstale":"2024-05-08T23:32:42.483828+0000","last_undegraded":"2024-05-08T23:32:42.483828+0000","last_fullsized":"2024-05-08T23:32:42.483828+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:53.369948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.871265+0000","last_change":"2024-05-08T23:32:40.884523+0000","last_active":"2024-05-08T23:32:41.871265+0000","last_peered":"2024-05-08T23:32:41.871265+0000","last_clean":"2024-05-08T23:32:41.871265+0000","last_became_active":"2024-05-08T23:32:40.884222+0000","last_became_peered":"2024-05-08T23:32:40.884222+0000","last_unstale":"2024-05-08T23:32:41.871265+0000","last_undegraded":"2024-05-08T23:32:41.871265+0000","last_fullsized":"2024-05-08T23:32:41.871265+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:14.835361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869149+0000","last_change":"2024-05-08T23:32:40.871672+0000","last_active":"2024-05-08T23:32:41.869149+0000","last_peered":"2024-05-08T23:32:41.869149+0000","last_clean":"2024-05-08T23:32:41.869149+0000","last_became_active":"2024-05-08T23:32:40.871428+0000","last_became_peered":"2024-05-08T23:32:40.871428+0000","last_unstale":"2024-05-08T23:32:41.869149+0000","last_undegraded":"2024-05-08T23:32:41.869149+0000","last_fullsized":"2024-05-08T23:32:41.869149+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:51.073137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870558+0000","last_change":"2024-05-08T23:32:40.882242+0000","last_active":"2024-05-08T23:32:41.870558+0000","last_peered":"2024-05-08T23:32:41.870558+0000","last_clean":"2024-05-08T23:32:41.870558+0000","last_became_active":"2024-05-08T23:32:40.881992+0000","last_became_peered":"2024-05-08T23:32:40.881992+0000","last_unstale":"2024-05-08T23:32:41.870558+0000","last_undegraded":"2024-05-08T23:32:41.870558+0000","last_fullsized":"2024-05-08T23:32:41.870558+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:33.892287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.484771+0000","last_change":"2024-05-08T23:32:40.884833+0000","last_active":"2024-05-08T23:32:42.484771+0000","last_peered":"2024-05-08T23:32:42.484771+0000","last_clean":"2024-05-08T23:32:42.484771+0000","last_became_active":"2024-05-08T23:32:40.884593+0000","last_became_peered":"2024-05-08T23:32:40.884593+0000","last_unstale":"2024-05-08T23:32:42.484771+0000","last_undegraded":"2024-05-08T23:32:42.484771+0000","last_fullsized":"2024-05-08T23:32:42.484771+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:07.830275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485733+0000","last_change":"2024-05-08T23:32:40.879541+0000","last_active":"2024-05-08T23:32:42.485733+0000","last_peered":"2024-05-08T23:32:42.485733+0000","last_clean":"2024-05-08T23:32:42.485733+0000","last_became_active":"2024-05-08T23:32:40.879148+0000","last_became_peered":"2024-05-08T23:32:40.879148+0000","last_unstale":"2024-05-08T23:32:42.485733+0000","last_undegraded":"2024-05-08T23:32:42.485733+0000","last_fullsized":"2024-05-08T23:32:42.485733+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:21.065207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485985+0000","last_change":"2024-05-08T23:32:40.877438+0000","last_active":"2024-05-08T23:32:42.485985+0000","last_peered":"2024-05-08T23:32:42.485985+0000","last_clean":"2024-05-08T23:32:42.485985+0000","last_became_active":"2024-05-08T23:32:40.877166+0000","last_became_peered":"2024-05-08T23:32:40.877166+0000","last_unstale":"2024-05-08T23:32:42.485985+0000","last_undegraded":"2024-05-08T23:32:42.485985+0000","last_fullsized":"2024-05-08T23:32:42.485985+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:32.843722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869488+0000","last_change":"2024-05-08T23:32:40.882835+0000","last_active":"2024-05-08T23:32:41.869488+0000","last_peered":"2024-05-08T23:32:41.869488+0000","last_clean":"2024-05-08T23:32:41.869488+0000","last_became_active":"2024-05-08T23:32:40.882284+0000","last_became_peered":"2024-05-08T23:32:40.882284+0000","last_unstale":"2024-05-08T23:32:41.869488+0000","last_undegraded":"2024-05-08T23:32:41.869488+0000","last_fullsized":"2024-05-08T23:32:41.869488+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:22.400514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485731+0000","last_change":"2024-05-08T23:32:40.884162+0000","last_active":"2024-05-08T23:32:42.485731+0000","last_peered":"2024-05-08T23:32:42.485731+0000","last_clean":"2024-05-08T23:32:42.485731+0000","last_became_active":"2024-05-08T23:32:40.883891+0000","last_became_peered":"2024-05-08T23:32:40.883891+0000","last_unstale":"2024-05-08T23:32:42.485731+0000","last_undegraded":"2024-05-08T23:32:42.485731+0000","last_fullsized":"2024-05-08T23:32:42.485731+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:22.400514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869593+0000","last_change":"2024-05-08T23:32:40.872374+0000","last_active":"2024-05-08T23:32:41.869593+0000","last_peered":"2024-05-08T23:32:41.869593+0000","last_clean":"2024-05-08T23:32:41.869593+0000","last_became_active":"2024-05-08T23:32:40.871713+0000","last_became_peered":"2024-05-08T23:32:40.871713+0000","last_unstale":"2024-05-08T23:32:41.869593+0000","last_undegraded":"2024-05-08T23:32:41.869593+0000","last_fullsized":"2024-05-08T23:32:41.869593+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:03.982050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.484955+0000","last_change":"2024-05-08T23:32:40.889214+0000","last_active":"2024-05-08T23:32:42.484955+0000","last_peered":"2024-05-08T23:32:42.484955+0000","last_clean":"2024-05-08T23:32:42.484955+0000","last_became_active":"2024-05-08T23:32:40.888959+0000","last_became_peered":"2024-05-08T23:32:40.888959+0000","last_unstale":"2024-05-08T23:32:42.484955+0000","last_undegraded":"2024-05-08T23:32:42.484955+0000","last_fullsized":"2024-05-08T23:32:42.484955+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:59.591552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485613+0000","last_change":"2024-05-08T23:32:40.888222+0000","last_active":"2024-05-08T23:32:42.485613+0000","last_peered":"2024-05-08T23:32:42.485613+0000","last_clean":"2024-05-08T23:32:42.485613+0000","last_became_active":"2024-05-08T23:32:40.887649+0000","last_became_peered":"2024-05-08T23:32:40.887649+0000","last_unstale":"2024-05-08T23:32:42.485613+0000","last_undegraded":"2024-05-08T23:32:42.485613+0000","last_fullsized":"2024-05-08T23:32:42.485613+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:56.942715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"51'4","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.884613+0000","last_change":"2024-05-08T23:32:40.496534+0000","last_active":"2024-05-08T23:32:40.884613+0000","last_peered":"2024-05-08T23:32:40.884613+0000","last_clean":"2024-05-08T23:32:40.884613+0000","last_became_active":"2024-05-08T23:32:40.496284+0000","last_became_peered":"2024-05-08T23:32:40.496284+0000","last_unstale":"2024-05-08T23:32:40.884613+0000","last_undegraded":"2024-05-08T23:32:40.884613+0000","last_fullsized":"2024-05-08T23:32:40.884613+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:13.825466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485432+0000","last_change":"2024-05-08T23:32:40.881466+0000","last_active":"2024-05-08T23:32:42.485432+0000","last_peered":"2024-05-08T23:32:42.485432+0000","last_clean":"2024-05-08T23:32:42.485432+0000","last_became_active":"2024-05-08T23:32:40.881211+0000","last_became_peered":"2024-05-08T23:32:40.881211+0000","last_unstale":"2024-05-08T23:32:42.485432+0000","last_undegraded":"2024-05-08T23:32:42.485432+0000","last_fullsized":"2024-05-08T23:32:42.485432+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:18.161378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.884884+0000","last_change":"2024-05-08T23:32:40.500874+0000","last_active":"2024-05-08T23:32:40.884884+0000","last_peered":"2024-05-08T23:32:40.884884+0000","last_clean":"2024-05-08T23:32:40.884884+0000","last_became_active":"2024-05-08T23:32:40.500556+0000","last_became_peered":"2024-05-08T23:32:40.500556+0000","last_unstale":"2024-05-08T23:32:40.884884+0000","last_undegraded":"2024-05-08T23:32:40.884884+0000","last_fullsized":"2024-05-08T23:32:40.884884+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:43.629558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.484984+0000","last_change":"2024-05-08T23:32:40.880550+0000","last_active":"2024-05-08T23:32:42.484984+0000","last_peered":"2024-05-08T23:32:42.484984+0000","last_clean":"2024-05-08T23:32:42.484984+0000","last_became_active":"2024-05-08T23:32:40.880273+0000","last_became_peered":"2024-05-08T23:32:40.880273+0000","last_unstale":"2024-05-08T23:32:42.484984+0000","last_undegraded":"2024-05-08T23:32:42.484984+0000","last_fullsized":"2024-05-08T23:32:42.484984+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:21.902005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.881483+0000","last_change":"2024-05-08T23:32:40.499343+0000","last_active":"2024-05-08T23:32:40.881483+0000","last_peered":"2024-05-08T23:32:40.881483+0000","last_clean":"2024-05-08T23:32:40.881483+0000","last_became_active":"2024-05-08T23:32:40.499089+0000","last_became_peered":"2024-05-08T23:32:40.499089+0000","last_unstale":"2024-05-08T23:32:40.881483+0000","last_undegraded":"2024-05-08T23:32:40.881483+0000","last_fullsized":"2024-05-08T23:32:40.881483+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:39.770766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485317+0000","last_change":"2024-05-08T23:32:40.879053+0000","last_active":"2024-05-08T23:32:42.485317+0000","last_peered":"2024-05-08T23:32:42.485317+0000","last_clean":"2024-05-08T23:32:42.485317+0000","last_became_active":"2024-05-08T23:32:40.878783+0000","last_became_peered":"2024-05-08T23:32:40.878783+0000","last_unstale":"2024-05-08T23:32:42.485317+0000","last_undegraded":"2024-05-08T23:32:42.485317+0000","last_fullsized":"2024-05-08T23:32:42.485317+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:29.240978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.886593+0000","last_change":"2024-05-08T23:32:40.496977+0000","last_active":"2024-05-08T23:32:40.886593+0000","last_peered":"2024-05-08T23:32:40.886593+0000","last_clean":"2024-05-08T23:32:40.886593+0000","last_became_active":"2024-05-08T23:32:40.496758+0000","last_became_peered":"2024-05-08T23:32:40.496758+0000","last_unstale":"2024-05-08T23:32:40.886593+0000","last_undegraded":"2024-05-08T23:32:40.886593+0000","last_fullsized":"2024-05-08T23:32:40.886593+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:09:12.523765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.872565+0000","last_change":"2024-05-08T23:32:40.878039+0000","last_active":"2024-05-08T23:32:41.872565+0000","last_peered":"2024-05-08T23:32:41.872565+0000","last_clean":"2024-05-08T23:32:41.872565+0000","last_became_active":"2024-05-08T23:32:40.877789+0000","last_became_peered":"2024-05-08T23:32:40.877789+0000","last_unstale":"2024-05-08T23:32:41.872565+0000","last_undegraded":"2024-05-08T23:32:41.872565+0000","last_fullsized":"2024-05-08T23:32:41.872565+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:36:28.424829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869722+0000","last_change":"2024-05-08T23:32:40.503223+0000","last_active":"2024-05-08T23:32:41.869722+0000","last_peered":"2024-05-08T23:32:41.869722+0000","last_clean":"2024-05-08T23:32:41.869722+0000","last_became_active":"2024-05-08T23:32:40.502907+0000","last_became_peered":"2024-05-08T23:32:40.502907+0000","last_unstale":"2024-05-08T23:32:41.869722+0000","last_undegraded":"2024-05-08T23:32:41.869722+0000","last_fullsized":"2024-05-08T23:32:41.869722+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:43.304902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486163+0000","last_change":"2024-05-08T23:32:40.884604+0000","last_active":"2024-05-08T23:32:42.486163+0000","last_peered":"2024-05-08T23:32:42.486163+0000","last_clean":"2024-05-08T23:32:42.486163+0000","last_became_active":"2024-05-08T23:32:40.884337+0000","last_became_peered":"2024-05-08T23:32:40.884337+0000","last_unstale":"2024-05-08T23:32:42.486163+0000","last_undegraded":"2024-05-08T23:32:42.486163+0000","last_fullsized":"2024-05-08T23:32:42.486163+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:44.757107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.883112+0000","last_change":"2024-05-08T23:32:40.499483+0000","last_active":"2024-05-08T23:32:40.883112+0000","last_peered":"2024-05-08T23:32:40.883112+0000","last_clean":"2024-05-08T23:32:40.883112+0000","last_became_active":"2024-05-08T23:32:40.499187+0000","last_became_peered":"2024-05-08T23:32:40.499187+0000","last_unstale":"2024-05-08T23:32:40.883112+0000","last_undegraded":"2024-05-08T23:32:40.883112+0000","last_fullsized":"2024-05-08T23:32:40.883112+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:41.416678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.484997+0000","last_change":"2024-05-08T23:32:40.880436+0000","last_active":"2024-05-08T23:32:42.484997+0000","last_peered":"2024-05-08T23:32:42.484997+0000","last_clean":"2024-05-08T23:32:42.484997+0000","last_became_active":"2024-05-08T23:32:40.879932+0000","last_became_peered":"2024-05-08T23:32:40.879932+0000","last_unstale":"2024-05-08T23:32:42.484997+0000","last_undegraded":"2024-05-08T23:32:42.484997+0000","last_fullsized":"2024-05-08T23:32:42.484997+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:36.919452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.873854+0000","last_change":"2024-05-08T23:32:40.495823+0000","last_active":"2024-05-08T23:32:40.873854+0000","last_peered":"2024-05-08T23:32:40.873854+0000","last_clean":"2024-05-08T23:32:40.873854+0000","last_became_active":"2024-05-08T23:32:40.495526+0000","last_became_peered":"2024-05-08T23:32:40.495526+0000","last_unstale":"2024-05-08T23:32:40.873854+0000","last_undegraded":"2024-05-08T23:32:40.873854+0000","last_fullsized":"2024-05-08T23:32:40.873854+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:39:14.674732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.487020+0000","last_change":"2024-05-08T23:32:40.885545+0000","last_active":"2024-05-08T23:32:42.487020+0000","last_peered":"2024-05-08T23:32:42.487020+0000","last_clean":"2024-05-08T23:32:42.487020+0000","last_became_active":"2024-05-08T23:32:40.885195+0000","last_became_peered":"2024-05-08T23:32:40.885195+0000","last_unstale":"2024-05-08T23:32:42.487020+0000","last_undegraded":"2024-05-08T23:32:42.487020+0000","last_fullsized":"2024-05-08T23:32:42.487020+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:33.567285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869218+0000","last_change":"2024-05-08T23:32:40.500395+0000","last_active":"2024-05-08T23:32:41.869218+0000","last_peered":"2024-05-08T23:32:41.869218+0000","last_clean":"2024-05-08T23:32:41.869218+0000","last_became_active":"2024-05-08T23:32:40.500130+0000","last_became_peered":"2024-05-08T23:32:40.500130+0000","last_unstale":"2024-05-08T23:32:41.869218+0000","last_undegraded":"2024-05-08T23:32:41.869218+0000","last_fullsized":"2024-05-08T23:32:41.869218+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:08.134598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.872127+0000","last_change":"2024-05-08T23:32:40.888968+0000","last_active":"2024-05-08T23:32:41.872127+0000","last_peered":"2024-05-08T23:32:41.872127+0000","last_clean":"2024-05-08T23:32:41.872127+0000","last_became_active":"2024-05-08T23:32:40.888637+0000","last_became_peered":"2024-05-08T23:32:40.888637+0000","last_unstale":"2024-05-08T23:32:41.872127+0000","last_undegraded":"2024-05-08T23:32:41.872127+0000","last_fullsized":"2024-05-08T23:32:41.872127+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:47:59.478680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.487136+0000","last_change":"2024-05-08T23:32:40.502985+0000","last_active":"2024-05-08T23:32:42.487136+0000","last_peered":"2024-05-08T23:32:42.487136+0000","last_clean":"2024-05-08T23:32:42.487136+0000","last_became_active":"2024-05-08T23:32:40.502677+0000","last_became_peered":"2024-05-08T23:32:40.502677+0000","last_unstale":"2024-05-08T23:32:42.487136+0000","last_undegraded":"2024-05-08T23:32:42.487136+0000","last_fullsized":"2024-05-08T23:32:42.487136+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:26.160105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.487251+0000","last_change":"2024-05-08T23:32:40.880955+0000","last_active":"2024-05-08T23:32:42.487251+0000","last_peered":"2024-05-08T23:32:42.487251+0000","last_clean":"2024-05-08T23:32:42.487251+0000","last_became_active":"2024-05-08T23:32:40.880674+0000","last_became_peered":"2024-05-08T23:32:40.880674+0000","last_unstale":"2024-05-08T23:32:42.487251+0000","last_undegraded":"2024-05-08T23:32:42.487251+0000","last_fullsized":"2024-05-08T23:32:42.487251+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:55.391720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.873114+0000","last_change":"2024-05-08T23:32:40.495101+0000","last_active":"2024-05-08T23:32:40.873114+0000","last_peered":"2024-05-08T23:32:40.873114+0000","last_clean":"2024-05-08T23:32:40.873114+0000","last_became_active":"2024-05-08T23:32:40.494779+0000","last_became_peered":"2024-05-08T23:32:40.494779+0000","last_unstale":"2024-05-08T23:32:40.873114+0000","last_undegraded":"2024-05-08T23:32:40.873114+0000","last_fullsized":"2024-05-08T23:32:40.873114+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:31.833826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870114+0000","last_change":"2024-05-08T23:32:40.877145+0000","last_active":"2024-05-08T23:32:41.870114+0000","last_peered":"2024-05-08T23:32:41.870114+0000","last_clean":"2024-05-08T23:32:41.870114+0000","last_became_active":"2024-05-08T23:32:40.876105+0000","last_became_peered":"2024-05-08T23:32:40.876105+0000","last_unstale":"2024-05-08T23:32:41.870114+0000","last_undegraded":"2024-05-08T23:32:41.870114+0000","last_fullsized":"2024-05-08T23:32:41.870114+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:44.757107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486576+0000","last_change":"2024-05-08T23:32:40.503254+0000","last_active":"2024-05-08T23:32:42.486576+0000","last_peered":"2024-05-08T23:32:42.486576+0000","last_clean":"2024-05-08T23:32:42.486576+0000","last_became_active":"2024-05-08T23:32:40.503019+0000","last_became_peered":"2024-05-08T23:32:40.503019+0000","last_unstale":"2024-05-08T23:32:42.486576+0000","last_undegraded":"2024-05-08T23:32:42.486576+0000","last_fullsized":"2024-05-08T23:32:42.486576+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:32.059412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.872596+0000","last_change":"2024-05-08T23:32:40.872383+0000","last_active":"2024-05-08T23:32:41.872596+0000","last_peered":"2024-05-08T23:32:41.872596+0000","last_clean":"2024-05-08T23:32:41.872596+0000","last_became_active":"2024-05-08T23:32:40.872100+0000","last_became_peered":"2024-05-08T23:32:40.872100+0000","last_unstale":"2024-05-08T23:32:41.872596+0000","last_undegraded":"2024-05-08T23:32:41.872596+0000","last_fullsized":"2024-05-08T23:32:41.872596+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:40.780661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"51'6","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.895170+0000","last_change":"2024-05-08T23:32:40.495716+0000","last_active":"2024-05-08T23:32:40.895170+0000","last_peered":"2024-05-08T23:32:40.895170+0000","last_clean":"2024-05-08T23:32:40.895170+0000","last_became_active":"2024-05-08T23:32:40.494867+0000","last_became_peered":"2024-05-08T23:32:40.494867+0000","last_unstale":"2024-05-08T23:32:40.895170+0000","last_undegraded":"2024-05-08T23:32:40.895170+0000","last_fullsized":"2024-05-08T23:32:40.895170+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:24.848996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485643+0000","last_change":"2024-05-08T23:32:40.879172+0000","last_active":"2024-05-08T23:32:42.485643+0000","last_peered":"2024-05-08T23:32:42.485643+0000","last_clean":"2024-05-08T23:32:42.485643+0000","last_became_active":"2024-05-08T23:32:40.878891+0000","last_became_peered":"2024-05-08T23:32:40.878891+0000","last_unstale":"2024-05-08T23:32:42.485643+0000","last_undegraded":"2024-05-08T23:32:42.485643+0000","last_fullsized":"2024-05-08T23:32:42.485643+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:18:57.346442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869846+0000","last_change":"2024-05-08T23:32:40.501242+0000","last_active":"2024-05-08T23:32:41.869846+0000","last_peered":"2024-05-08T23:32:41.869846+0000","last_clean":"2024-05-08T23:32:41.869846+0000","last_became_active":"2024-05-08T23:32:40.500998+0000","last_became_peered":"2024-05-08T23:32:40.500998+0000","last_unstale":"2024-05-08T23:32:41.869846+0000","last_undegraded":"2024-05-08T23:32:41.869846+0000","last_fullsized":"2024-05-08T23:32:41.869846+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:09.047211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870109+0000","last_change":"2024-05-08T23:32:40.873424+0000","last_active":"2024-05-08T23:32:41.870109+0000","last_peered":"2024-05-08T23:32:41.870109+0000","last_clean":"2024-05-08T23:32:41.870109+0000","last_became_active":"2024-05-08T23:32:40.873161+0000","last_became_peered":"2024-05-08T23:32:40.873161+0000","last_unstale":"2024-05-08T23:32:41.870109+0000","last_undegraded":"2024-05-08T23:32:41.870109+0000","last_fullsized":"2024-05-08T23:32:41.870109+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:14.349815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.874411+0000","last_change":"2024-05-08T23:32:40.504396+0000","last_active":"2024-05-08T23:32:40.874411+0000","last_peered":"2024-05-08T23:32:40.874411+0000","last_clean":"2024-05-08T23:32:40.874411+0000","last_became_active":"2024-05-08T23:32:40.504148+0000","last_became_peered":"2024-05-08T23:32:40.504148+0000","last_unstale":"2024-05-08T23:32:40.874411+0000","last_undegraded":"2024-05-08T23:32:40.874411+0000","last_fullsized":"2024-05-08T23:32:40.874411+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:31.286020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.484833+0000","last_change":"2024-05-08T23:32:40.884363+0000","last_active":"2024-05-08T23:32:42.484833+0000","last_peered":"2024-05-08T23:32:42.484833+0000","last_clean":"2024-05-08T23:32:42.484833+0000","last_became_active":"2024-05-08T23:32:40.881539+0000","last_became_peered":"2024-05-08T23:32:40.881539+0000","last_unstale":"2024-05-08T23:32:42.484833+0000","last_undegraded":"2024-05-08T23:32:42.484833+0000","last_fullsized":"2024-05-08T23:32:42.484833+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:24:04.490840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485852+0000","last_change":"2024-05-08T23:32:40.499587+0000","last_active":"2024-05-08T23:32:42.485852+0000","last_peered":"2024-05-08T23:32:42.485852+0000","last_clean":"2024-05-08T23:32:42.485852+0000","last_became_active":"2024-05-08T23:32:40.499243+0000","last_became_peered":"2024-05-08T23:32:40.499243+0000","last_unstale":"2024-05-08T23:32:42.485852+0000","last_undegraded":"2024-05-08T23:32:42.485852+0000","last_fullsized":"2024-05-08T23:32:42.485852+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:20:53.323381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870508+0000","last_change":"2024-05-08T23:32:40.872864+0000","last_active":"2024-05-08T23:32:41.870508+0000","last_peered":"2024-05-08T23:32:41.870508+0000","last_clean":"2024-05-08T23:32:41.870508+0000","last_became_active":"2024-05-08T23:32:40.872596+0000","last_became_peered":"2024-05-08T23:32:40.872596+0000","last_unstale":"2024-05-08T23:32:41.870508+0000","last_undegraded":"2024-05-08T23:32:41.870508+0000","last_fullsized":"2024-05-08T23:32:41.870508+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:18.771018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"51'8","reported_seq":19,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.895353+0000","last_change":"2024-05-08T23:32:40.499625+0000","last_active":"2024-05-08T23:32:40.895353+0000","last_peered":"2024-05-08T23:32:40.895353+0000","last_clean":"2024-05-08T23:32:40.895353+0000","last_became_active":"2024-05-08T23:32:40.499319+0000","last_became_peered":"2024-05-08T23:32:40.499319+0000","last_unstale":"2024-05-08T23:32:40.895353+0000","last_undegraded":"2024-05-08T23:32:40.895353+0000","last_fullsized":"2024-05-08T23:32:40.895353+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:31.133500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.483995+0000","last_change":"2024-05-08T23:32:40.889185+0000","last_active":"2024-05-08T23:32:42.483995+0000","last_peered":"2024-05-08T23:32:42.483995+0000","last_clean":"2024-05-08T23:32:42.483995+0000","last_became_active":"2024-05-08T23:32:40.888884+0000","last_became_peered":"2024-05-08T23:32:40.888884+0000","last_unstale":"2024-05-08T23:32:42.483995+0000","last_undegraded":"2024-05-08T23:32:42.483995+0000","last_fullsized":"2024-05-08T23:32:42.483995+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:02.020868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870719+0000","last_change":"2024-05-08T23:32:40.499582+0000","last_active":"2024-05-08T23:32:41.870719+0000","last_peered":"2024-05-08T23:32:41.870719+0000","last_clean":"2024-05-08T23:32:41.870719+0000","last_became_active":"2024-05-08T23:32:40.497931+0000","last_became_peered":"2024-05-08T23:32:40.497931+0000","last_unstale":"2024-05-08T23:32:41.870719+0000","last_undegraded":"2024-05-08T23:32:41.870719+0000","last_fullsized":"2024-05-08T23:32:41.870719+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:40.305843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.484484+0000","last_change":"2024-05-08T23:32:40.878703+0000","last_active":"2024-05-08T23:32:42.484484+0000","last_peered":"2024-05-08T23:32:42.484484+0000","last_clean":"2024-05-08T23:32:42.484484+0000","last_became_active":"2024-05-08T23:32:40.878373+0000","last_became_peered":"2024-05-08T23:32:40.878373+0000","last_unstale":"2024-05-08T23:32:42.484484+0000","last_undegraded":"2024-05-08T23:32:42.484484+0000","last_fullsized":"2024-05-08T23:32:42.484484+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:18:57.346442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.887424+0000","last_change":"2024-05-08T23:32:40.493159+0000","last_active":"2024-05-08T23:32:40.887424+0000","last_peered":"2024-05-08T23:32:40.887424+0000","last_clean":"2024-05-08T23:32:40.887424+0000","last_became_active":"2024-05-08T23:32:40.492233+0000","last_became_peered":"2024-05-08T23:32:40.492233+0000","last_unstale":"2024-05-08T23:32:40.887424+0000","last_undegraded":"2024-05-08T23:32:40.887424+0000","last_fullsized":"2024-05-08T23:32:40.887424+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:43:57.594449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.872077+0000","last_change":"2024-05-08T23:32:40.883573+0000","last_active":"2024-05-08T23:32:41.872077+0000","last_peered":"2024-05-08T23:32:41.872077+0000","last_clean":"2024-05-08T23:32:41.872077+0000","last_became_active":"2024-05-08T23:32:40.883276+0000","last_became_peered":"2024-05-08T23:32:40.883276+0000","last_unstale":"2024-05-08T23:32:41.872077+0000","last_undegraded":"2024-05-08T23:32:41.872077+0000","last_fullsized":"2024-05-08T23:32:41.872077+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:55.933403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.884017+0000","last_change":"2024-05-08T23:32:40.501729+0000","last_active":"2024-05-08T23:32:40.884017+0000","last_peered":"2024-05-08T23:32:40.884017+0000","last_clean":"2024-05-08T23:32:40.884017+0000","last_became_active":"2024-05-08T23:32:40.501437+0000","last_became_peered":"2024-05-08T23:32:40.501437+0000","last_unstale":"2024-05-08T23:32:40.884017+0000","last_undegraded":"2024-05-08T23:32:40.884017+0000","last_fullsized":"2024-05-08T23:32:40.884017+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:33:25.871051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485015+0000","last_change":"2024-05-08T23:32:40.874834+0000","last_active":"2024-05-08T23:32:42.485015+0000","last_peered":"2024-05-08T23:32:42.485015+0000","last_clean":"2024-05-08T23:32:42.485015+0000","last_became_active":"2024-05-08T23:32:40.874607+0000","last_became_peered":"2024-05-08T23:32:40.874607+0000","last_unstale":"2024-05-08T23:32:42.485015+0000","last_undegraded":"2024-05-08T23:32:42.485015+0000","last_fullsized":"2024-05-08T23:32:42.485015+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:47.908211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.874128+0000","last_change":"2024-05-08T23:32:40.502954+0000","last_active":"2024-05-08T23:32:40.874128+0000","last_peered":"2024-05-08T23:32:40.874128+0000","last_clean":"2024-05-08T23:32:40.874128+0000","last_became_active":"2024-05-08T23:32:40.502651+0000","last_became_peered":"2024-05-08T23:32:40.502651+0000","last_unstale":"2024-05-08T23:32:40.874128+0000","last_undegraded":"2024-05-08T23:32:40.874128+0000","last_fullsized":"2024-05-08T23:32:40.874128+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:06.250791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485892+0000","last_change":"2024-05-08T23:32:40.876681+0000","last_active":"2024-05-08T23:32:42.485892+0000","last_peered":"2024-05-08T23:32:42.485892+0000","last_clean":"2024-05-08T23:32:42.485892+0000","last_became_active":"2024-05-08T23:32:40.876419+0000","last_became_peered":"2024-05-08T23:32:40.876419+0000","last_unstale":"2024-05-08T23:32:42.485892+0000","last_undegraded":"2024-05-08T23:32:42.485892+0000","last_fullsized":"2024-05-08T23:32:42.485892+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:37.875587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.888146+0000","last_change":"2024-05-08T23:32:40.500995+0000","last_active":"2024-05-08T23:32:40.888146+0000","last_peered":"2024-05-08T23:32:40.888146+0000","last_clean":"2024-05-08T23:32:40.888146+0000","last_became_active":"2024-05-08T23:32:40.500596+0000","last_became_peered":"2024-05-08T23:32:40.500596+0000","last_unstale":"2024-05-08T23:32:40.888146+0000","last_undegraded":"2024-05-08T23:32:40.888146+0000","last_fullsized":"2024-05-08T23:32:40.888146+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:23:54.923508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486064+0000","last_change":"2024-05-08T23:32:40.880472+0000","last_active":"2024-05-08T23:32:42.486064+0000","last_peered":"2024-05-08T23:32:42.486064+0000","last_clean":"2024-05-08T23:32:42.486064+0000","last_became_active":"2024-05-08T23:32:40.880211+0000","last_became_peered":"2024-05-08T23:32:40.880211+0000","last_unstale":"2024-05-08T23:32:42.486064+0000","last_undegraded":"2024-05-08T23:32:42.486064+0000","last_fullsized":"2024-05-08T23:32:42.486064+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:51.735878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.869709+0000","last_change":"2024-05-08T23:32:40.499515+0000","last_active":"2024-05-08T23:32:40.869709+0000","last_peered":"2024-05-08T23:32:40.869709+0000","last_clean":"2024-05-08T23:32:40.869709+0000","last_became_active":"2024-05-08T23:32:40.499229+0000","last_became_peered":"2024-05-08T23:32:40.499229+0000","last_unstale":"2024-05-08T23:32:40.869709+0000","last_undegraded":"2024-05-08T23:32:40.869709+0000","last_fullsized":"2024-05-08T23:32:40.869709+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:18:31.349703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870336+0000","last_change":"2024-05-08T23:32:40.874535+0000","last_active":"2024-05-08T23:32:41.870336+0000","last_peered":"2024-05-08T23:32:41.870336+0000","last_clean":"2024-05-08T23:32:41.870336+0000","last_became_active":"2024-05-08T23:32:40.874316+0000","last_became_peered":"2024-05-08T23:32:40.874316+0000","last_unstale":"2024-05-08T23:32:41.870336+0000","last_undegraded":"2024-05-08T23:32:41.870336+0000","last_fullsized":"2024-05-08T23:32:41.870336+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:06:56.942715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869562+0000","last_change":"2024-05-08T23:32:40.500526+0000","last_active":"2024-05-08T23:32:41.869562+0000","last_peered":"2024-05-08T23:32:41.869562+0000","last_clean":"2024-05-08T23:32:41.869562+0000","last_became_active":"2024-05-08T23:32:40.500219+0000","last_became_peered":"2024-05-08T23:32:40.500219+0000","last_unstale":"2024-05-08T23:32:41.869562+0000","last_undegraded":"2024-05-08T23:32:41.869562+0000","last_fullsized":"2024-05-08T23:32:41.869562+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:56.869056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.484979+0000","last_change":"2024-05-08T23:32:40.885392+0000","last_active":"2024-05-08T23:32:42.484979+0000","last_peered":"2024-05-08T23:32:42.484979+0000","last_clean":"2024-05-08T23:32:42.484979+0000","last_became_active":"2024-05-08T23:32:40.885135+0000","last_became_peered":"2024-05-08T23:32:40.885135+0000","last_unstale":"2024-05-08T23:32:42.484979+0000","last_undegraded":"2024-05-08T23:32:42.484979+0000","last_fullsized":"2024-05-08T23:32:42.484979+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:24.731469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.865251+0000","last_change":"2024-05-08T23:32:40.494606+0000","last_active":"2024-05-08T23:32:40.865251+0000","last_peered":"2024-05-08T23:32:40.865251+0000","last_clean":"2024-05-08T23:32:40.865251+0000","last_became_active":"2024-05-08T23:32:40.494327+0000","last_became_peered":"2024-05-08T23:32:40.494327+0000","last_unstale":"2024-05-08T23:32:40.865251+0000","last_undegraded":"2024-05-08T23:32:40.865251+0000","last_fullsized":"2024-05-08T23:32:40.865251+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:20:31.576039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485198+0000","last_change":"2024-05-08T23:32:40.889195+0000","last_active":"2024-05-08T23:32:42.485198+0000","last_peered":"2024-05-08T23:32:42.485198+0000","last_clean":"2024-05-08T23:32:42.485198+0000","last_became_active":"2024-05-08T23:32:40.888798+0000","last_became_peered":"2024-05-08T23:32:40.888798+0000","last_unstale":"2024-05-08T23:32:42.485198+0000","last_undegraded":"2024-05-08T23:32:42.485198+0000","last_fullsized":"2024-05-08T23:32:42.485198+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:21.065207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.884360+0000","last_change":"2024-05-08T23:32:40.504334+0000","last_active":"2024-05-08T23:32:40.884360+0000","last_peered":"2024-05-08T23:32:40.884360+0000","last_clean":"2024-05-08T23:32:40.884360+0000","last_became_active":"2024-05-08T23:32:40.504073+0000","last_became_peered":"2024-05-08T23:32:40.504073+0000","last_unstale":"2024-05-08T23:32:40.884360+0000","last_undegraded":"2024-05-08T23:32:40.884360+0000","last_fullsized":"2024-05-08T23:32:40.884360+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:21.704760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485612+0000","last_change":"2024-05-08T23:32:40.872908+0000","last_active":"2024-05-08T23:32:42.485612+0000","last_peered":"2024-05-08T23:32:42.485612+0000","last_clean":"2024-05-08T23:32:42.485612+0000","last_became_active":"2024-05-08T23:32:40.872672+0000","last_became_peered":"2024-05-08T23:32:40.872672+0000","last_unstale":"2024-05-08T23:32:42.485612+0000","last_undegraded":"2024-05-08T23:32:42.485612+0000","last_fullsized":"2024-05-08T23:32:42.485612+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:18:32.359598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.486777+0000","last_change":"2024-05-08T23:32:40.499075+0000","last_active":"2024-05-08T23:32:42.486777+0000","last_peered":"2024-05-08T23:32:42.486777+0000","last_clean":"2024-05-08T23:32:42.486777+0000","last_became_active":"2024-05-08T23:32:40.498774+0000","last_became_peered":"2024-05-08T23:32:40.498774+0000","last_unstale":"2024-05-08T23:32:42.486777+0000","last_undegraded":"2024-05-08T23:32:42.486777+0000","last_fullsized":"2024-05-08T23:32:42.486777+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:09.302638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.484189+0000","last_change":"2024-05-08T23:32:40.881831+0000","last_active":"2024-05-08T23:32:42.484189+0000","last_peered":"2024-05-08T23:32:42.484189+0000","last_clean":"2024-05-08T23:32:42.484189+0000","last_became_active":"2024-05-08T23:32:40.881363+0000","last_became_peered":"2024-05-08T23:32:40.881363+0000","last_unstale":"2024-05-08T23:32:42.484189+0000","last_undegraded":"2024-05-08T23:32:42.484189+0000","last_fullsized":"2024-05-08T23:32:42.484189+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:42.333070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869732+0000","last_change":"2024-05-08T23:32:40.499734+0000","last_active":"2024-05-08T23:32:41.869732+0000","last_peered":"2024-05-08T23:32:41.869732+0000","last_clean":"2024-05-08T23:32:41.869732+0000","last_became_active":"2024-05-08T23:32:40.497679+0000","last_became_peered":"2024-05-08T23:32:40.497679+0000","last_unstale":"2024-05-08T23:32:41.869732+0000","last_undegraded":"2024-05-08T23:32:41.869732+0000","last_fullsized":"2024-05-08T23:32:41.869732+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:58.686919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869659+0000","last_change":"2024-05-08T23:32:40.885944+0000","last_active":"2024-05-08T23:32:41.869659+0000","last_peered":"2024-05-08T23:32:41.869659+0000","last_clean":"2024-05-08T23:32:41.869659+0000","last_became_active":"2024-05-08T23:32:40.885687+0000","last_became_peered":"2024-05-08T23:32:40.885687+0000","last_unstale":"2024-05-08T23:32:41.869659+0000","last_undegraded":"2024-05-08T23:32:41.869659+0000","last_fullsized":"2024-05-08T23:32:41.869659+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:33.069307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.868615+0000","last_change":"2024-05-08T23:32:40.495949+0000","last_active":"2024-05-08T23:32:40.868615+0000","last_peered":"2024-05-08T23:32:40.868615+0000","last_clean":"2024-05-08T23:32:40.868615+0000","last_became_active":"2024-05-08T23:32:40.495637+0000","last_became_peered":"2024-05-08T23:32:40.495637+0000","last_unstale":"2024-05-08T23:32:40.868615+0000","last_undegraded":"2024-05-08T23:32:40.868615+0000","last_fullsized":"2024-05-08T23:32:40.868615+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:46.898315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485662+0000","last_change":"2024-05-08T23:32:40.881484+0000","last_active":"2024-05-08T23:32:42.485662+0000","last_peered":"2024-05-08T23:32:42.485662+0000","last_clean":"2024-05-08T23:32:42.485662+0000","last_became_active":"2024-05-08T23:32:40.881221+0000","last_became_peered":"2024-05-08T23:32:40.881221+0000","last_unstale":"2024-05-08T23:32:42.485662+0000","last_undegraded":"2024-05-08T23:32:42.485662+0000","last_fullsized":"2024-05-08T23:32:42.485662+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:24.731469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.869830+0000","last_change":"2024-05-08T23:32:40.504202+0000","last_active":"2024-05-08T23:32:40.869830+0000","last_peered":"2024-05-08T23:32:40.869830+0000","last_clean":"2024-05-08T23:32:40.869830+0000","last_became_active":"2024-05-08T23:32:40.503863+0000","last_became_peered":"2024-05-08T23:32:40.503863+0000","last_unstale":"2024-05-08T23:32:40.869830+0000","last_undegraded":"2024-05-08T23:32:40.869830+0000","last_fullsized":"2024-05-08T23:32:40.869830+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.869988+0000","last_change":"2024-05-08T23:32:40.873427+0000","last_active":"2024-05-08T23:32:41.869988+0000","last_peered":"2024-05-08T23:32:41.869988+0000","last_clean":"2024-05-08T23:32:41.869988+0000","last_became_active":"2024-05-08T23:32:40.873194+0000","last_became_peered":"2024-05-08T23:32:40.873194+0000","last_unstale":"2024-05-08T23:32:41.869988+0000","last_undegraded":"2024-05-08T23:32:41.869988+0000","last_fullsized":"2024-05-08T23:32:41.869988+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:34.323573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.867053+0000","last_change":"2024-05-08T23:32:40.493439+0000","last_active":"2024-05-08T23:32:40.867053+0000","last_peered":"2024-05-08T23:32:40.867053+0000","last_clean":"2024-05-08T23:32:40.867053+0000","last_became_active":"2024-05-08T23:32:40.493190+0000","last_became_peered":"2024-05-08T23:32:40.493190+0000","last_unstale":"2024-05-08T23:32:40.867053+0000","last_undegraded":"2024-05-08T23:32:40.867053+0000","last_fullsized":"2024-05-08T23:32:40.867053+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:19.747747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.484984+0000","last_change":"2024-05-08T23:32:40.881899+0000","last_active":"2024-05-08T23:32:42.484984+0000","last_peered":"2024-05-08T23:32:42.484984+0000","last_clean":"2024-05-08T23:32:42.484984+0000","last_became_active":"2024-05-08T23:32:40.881565+0000","last_became_peered":"2024-05-08T23:32:40.881565+0000","last_unstale":"2024-05-08T23:32:42.484984+0000","last_undegraded":"2024-05-08T23:32:42.484984+0000","last_fullsized":"2024-05-08T23:32:42.484984+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:47:12.738368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.870659+0000","last_change":"2024-05-08T23:32:40.501727+0000","last_active":"2024-05-08T23:32:40.870659+0000","last_peered":"2024-05-08T23:32:40.870659+0000","last_clean":"2024-05-08T23:32:40.870659+0000","last_became_active":"2024-05-08T23:32:40.501413+0000","last_became_peered":"2024-05-08T23:32:40.501413+0000","last_unstale":"2024-05-08T23:32:40.870659+0000","last_undegraded":"2024-05-08T23:32:40.870659+0000","last_fullsized":"2024-05-08T23:32:40.870659+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:36.991964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485149+0000","last_change":"2024-05-08T23:32:40.887961+0000","last_active":"2024-05-08T23:32:42.485149+0000","last_peered":"2024-05-08T23:32:42.485149+0000","last_clean":"2024-05-08T23:32:42.485149+0000","last_became_active":"2024-05-08T23:32:40.887562+0000","last_became_peered":"2024-05-08T23:32:40.887562+0000","last_unstale":"2024-05-08T23:32:42.485149+0000","last_undegraded":"2024-05-08T23:32:42.485149+0000","last_fullsized":"2024-05-08T23:32:42.485149+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:40.471851+0000","last_clean_scrub_stamp":"2024-05-08T23:32:40.471851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:42.426573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:41.870540+0000","last_change":"2024-05-08T23:32:40.503644+0000","last_active":"2024-05-08T23:32:41.870540+0000","last_peered":"2024-05-08T23:32:41.870540+0000","last_clean":"2024-05-08T23:32:41.870540+0000","last_became_active":"2024-05-08T23:32:40.503283+0000","last_became_peered":"2024-05-08T23:32:40.503283+0000","last_unstale":"2024-05-08T23:32:41.870540+0000","last_undegraded":"2024-05-08T23:32:41.870540+0000","last_fullsized":"2024-05-08T23:32:41.870540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:39.461956+0000","last_clean_scrub_stamp":"2024-05-08T23:32:39.461956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:59.435733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":190,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-08T23:32:40.876580+0000","last_change":"2024-05-08T23:30:27.226484+0000","last_active":"2024-05-08T23:32:40.876580+0000","last_peered":"2024-05-08T23:32:40.876580+0000","last_clean":"2024-05-08T23:32:40.876580+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:32:40.876580+0000","last_undegraded":"2024-05-08T23:32:40.876580+0000","last_fullsized":"2024-05-08T23:32:40.876580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_clean_scrub_stamp":"2024-05-08T23:30:26.204118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:14.267287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509473,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":618496,"data_stored":253778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.89400000000000002,"15min":0.90500000000000003},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1719999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.96899999999999997,"15min":0.97899999999999998},"min":{"1min":0.58299999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1910000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.89700000000000002}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.59099999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6579999999999999,"5min":2.004,"15min":2.004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.92600000000000005,"15min":0.92400000000000004},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.327,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.68200000000000005}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88300000000000001,"15min":0.88500000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.95999999999999996,"15min":0.97099999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.218,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.78900000000000003}]},{"osd":3,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.89200000000000002,"15min":0.88400000000000001},"min":{"1min":0.65100000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96899999999999997,"15min":0.97799999999999998},"min":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2290000000000001,"5min":1.272,"15min":1.272},"last":0.85399999999999998}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.71499999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.1439999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0249999999999999,"15min":1.0269999999999999},"min":{"1min":0.67800000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.95299999999999996}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0469999999999999,"15min":1.056},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4450000000000001,"5min":2.109,"15min":2.109},"last":1.006},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1240000000000001,"15min":1.1439999999999999},"min":{"1min":0.75800000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.98099999999999998}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1970000000000001,"15min":1.2070000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5660000000000001,"5min":2.544,"15min":2.544},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.103,"15min":1.109},"min":{"1min":0.81200000000000006,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.0669999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509473,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":253126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2969999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.626},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.97099999999999997,"15min":0.98299999999999998},"min":{"1min":0.63700000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1859999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.46400000000000002}]},{"osd":1,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.90900000000000003,"15min":0.90400000000000003},"min":{"1min":0.67300000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88400000000000001,"15min":0.878},"min":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.54700000000000004}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.91800000000000004,"15min":0.91700000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.385,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.93700000000000006,"15min":0.92500000000000004},"min":{"1min":0.70299999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.454,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.70499999999999996}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.97199999999999998,"15min":0.98399999999999999},"min":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.252,"5min":1.395,"15min":1.395},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.004,"5min":0.96099999999999997,"15min":0.95399999999999996},"min":{"1min":0.63700000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.78300000000000003}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.995,"15min":1.0129999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.1339999999999999,"15min":1.1599999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.484,"5min":2.464,"15min":2.464},"last":0.99099999999999999}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0800000000000001,"15min":1.0820000000000001},"min":{"1min":0.82099999999999995,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.85799999999999998}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1930000000000001,"15min":1.216},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.1759999999999999,"15min":1.1779999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.93700000000000006}]}]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":614400,"data_stored":253708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.84099999999999997,"15min":0.85099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2030000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.3819999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77900000000000003,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.153}]},{"osd":1,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.70999999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1459999999999999,"5min":1.268,"15min":1.268},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.88800000000000001,"15min":0.90000000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.125,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.70299999999999996}]},{"osd":2,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.88800000000000001,"15min":0.89800000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.131},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89900000000000002,"15min":0.90700000000000003},"min":{"1min":0.65400000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0249999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.222}]},{"osd":3,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.85299999999999998,"15min":0.84899999999999998},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1759999999999999,"5min":1.23,"15min":1.23},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2390000000000001,"5min":1.431,"15min":1.431},"last":0.84299999999999997}]},{"osd":5,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3420000000000001,"5min":1.413,"15min":1.413},"last":0.995},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95899999999999996,"15min":0.95099999999999996},"min":{"1min":0.73999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.278,"5min":1.504,"15min":1.504},"last":0.89900000000000002}]},{"osd":6,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0269999999999999,"15min":1.0309999999999999},"min":{"1min":0.73899999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.175},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1419999999999999,"15min":1.1559999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3859999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.4350000000000001}]},{"osd":7,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0800000000000001,"15min":1.0840000000000001},"min":{"1min":0.69899999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4379999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.143,"15min":1.1479999999999999},"min":{"1min":0.94799999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4099999999999999,"5min":1.583,"15min":1.583},"last":1.353}]}]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":254712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.49099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.46300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":1.032,"15min":1.032},"last":0.67700000000000005}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70599999999999996,"15min":0.71099999999999997},"min":{"1min":0.50700000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.89300000000000002,"5min":1,"15min":1},"last":0.98899999999999999}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0469999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80200000000000005,"15min":0.79900000000000004},"min":{"1min":0.56699999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0229999999999999,"5min":1.242,"15min":1.242},"last":0.71499999999999997}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.65600000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.129,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.96699999999999997,"15min":0.97399999999999998},"min":{"1min":0.78000000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.244,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.749}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.014,"15min":1.022},"min":{"1min":0.80300000000000005,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.161,"5min":1.623,"15min":1.623},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.995,"15min":1.004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1359999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.89000000000000001}]},{"osd":6,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.997,"15min":1.0009999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.232,"5min":1.482,"15min":1.482},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.093,"15min":1.111},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3480000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.0209999999999999}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0800000000000001,"15min":1.0860000000000001},"min":{"1min":0.78400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.456,"5min":1.514,"15min":1.514},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0580000000000001,"15min":1.0600000000000001},"min":{"1min":0.75,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.2869999999999999,"5min":1.552,"15min":1.552},"last":0.96099999999999997}]}]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":254690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80400000000000005,"5min":1.099,"15min":1.099},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0660000000000001,"5min":1.129,"15min":1.129},"last":0.60899999999999999}]},{"osd":1,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70999999999999996,"15min":0.70899999999999996},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.879}]},{"osd":3,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0509999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.78500000000000003}]},{"osd":4,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.96799999999999997,"15min":0.97799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.27,"5min":1.367,"15min":1.367},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.95599999999999996,"15min":0.96399999999999997},"min":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.81999999999999995}]},{"osd":5,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98999999999999999,"15min":0.98599999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.137}]},{"osd":6,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0620000000000001,"15min":1.0740000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4239999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.075,"15min":1.075},"min":{"1min":0.85699999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.996}]},{"osd":7,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1220000000000001,"15min":1.1240000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":1.181},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1000000000000001,"15min":1.1020000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.492,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.94399999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509471,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":618496,"data_stored":253286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.51300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.44,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.442}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.628},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.68899999999999995,"15min":0.67000000000000004},"min":{"1min":0.53900000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.39000000000000001}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73499999999999999,"15min":0.72599999999999998},"min":{"1min":0.57999999999999996,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.42799999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.038,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.95099999999999996,"15min":0.95199999999999996},"min":{"1min":0.67600000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.256,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":0.88},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.95499999999999996,"15min":0.94299999999999995},"min":{"1min":0.77300000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.2709999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.70499999999999996}]},{"osd":5,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0109999999999999,"15min":1.0089999999999999},"min":{"1min":0.72299999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.403,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.75},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.077,"15min":1.079},"min":{"1min":0.85499999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3879999999999999,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.81499999999999995}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0369999999999999,"15min":1.034},"min":{"1min":0.80000000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.4330000000000001,"5min":1.633,"15min":1.633},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.095,"5min":1.135,"15min":1.1419999999999999},"min":{"1min":0.80500000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.4750000000000001,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.91800000000000004}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.0209999999999999,"15min":1.0009999999999999},"min":{"1min":0.83099999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.371,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.081,"15min":1.0820000000000001},"min":{"1min":0.84999999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.306,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.80100000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542175,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":843618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.88300000000000001,"15min":0.89700000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.84699999999999998}]},{"osd":1,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.86399999999999999,"15min":0.86699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1379999999999999,"5min":1.359,"15min":1.359},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91200000000000003,"15min":0.91600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.23,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.747}]},{"osd":2,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88,"15min":0.873},"min":{"1min":0.57499999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88,"15min":0.88500000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.133,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.97699999999999998}]},{"osd":3,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.91300000000000003,"15min":0.91100000000000003},"min":{"1min":0.60699999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.99299999999999999,"15min":0.99099999999999999},"min":{"1min":0.80100000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2470000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.94999999999999996}]},{"osd":4,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.90400000000000003,"15min":0.91200000000000003},"min":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.73999999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.1899999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.1080000000000001}]},{"osd":5,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.028,"15min":1.028},"min":{"1min":0.64100000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.405,"5min":1.714,"15min":1.714},"last":1.069},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.1950000000000001,"5min":1.325,"15min":1.325},"last":1.165}]},{"osd":6,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.048,"15min":1.048},"min":{"1min":0.86599999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.4430000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.105,"15min":1.107},"min":{"1min":0.83999999999999997,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.359,"5min":1.496,"15min":1.496},"last":1.151}]}]},{"osd":0,"up_from":14,"seq":60129542175,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1212416,"data_stored":843550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.47499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":1.351},"last":0.71799999999999997}]},{"osd":2,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.51500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.238,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.546,"5min":1.869,"15min":1.869},"last":0.80100000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.84199999999999997,"15min":0.85599999999999998},"min":{"1min":0.55200000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.94199999999999995,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.89300000000000002}]},{"osd":4,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97199999999999998,"15min":0.97299999999999998},"min":{"1min":0.71999999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.357,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.90800000000000003,"15min":0.89600000000000002},"min":{"1min":0.69899999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.746}]},{"osd":5,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.052,"15min":1.0600000000000001},"min":{"1min":0.76800000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3819999999999999,"5min":1.474,"15min":1.474},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.92500000000000004,"15min":0.91400000000000003},"min":{"1min":0.72699999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.98099999999999998}]},{"osd":6,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.056,"15min":1.0720000000000001},"min":{"1min":0.76900000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.171,"5min":1.736,"15min":1.736},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1180000000000001,"15min":1.121},"min":{"1min":0.89000000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4830000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.0009999999999999}]},{"osd":7,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.113,"15min":1.117},"min":{"1min":0.66600000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.397,"5min":1.72,"15min":1.72},"last":1.115},{"interface":"front","average":{"1min":1.097,"5min":1.109,"15min":1.111},"min":{"1min":0.86399999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.1439999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:32:49.655 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T23:32:49.655 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T23:32:49.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.783+0000 7f24fb59e700 1 Processor -- start 2024-05-08T23:32:49.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.783+0000 7f24fb59e700 1 -- start start 2024-05-08T23:32:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.783+0000 7f24fb59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24fc06d8c0 0x7f24fc15e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.783+0000 7f24fb59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24fc071a50 0x7f24fc156d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.783+0000 7f24fb59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24fc0721c0 con 0x7f24fc0b20f0 2024-05-08T23:32:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.783+0000 7f24fb59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24fc15e650 con 0x7f24fc071a50 2024-05-08T23:32:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.783+0000 7f24fb59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc0ad700 con 0x7f24fc06d8c0 2024-05-08T23:32:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.783+0000 7f24fad9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24fc071a50 0x7f24fc156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.783+0000 7f24fa59c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24fc06d8c0 0x7f24fc15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.783+0000 7f24fad9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24fc071a50 0x7f24fc156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:40182/0 (socket says 172.21.15.81:40182) 2024-05-08T23:32:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.783+0000 7f24f9d9b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f24fc0b20f0 0x7f24fc071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:60598/0 (socket says 172.21.15.81:60598) 2024-05-08T23:32:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.783+0000 7f24fad9d700 1 -- 172.21.15.81:0/1708665469 learned_addr learned my addr 172.21.15.81:0/1708665469 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24fad9d700 1 -- 172.21.15.81:0/1708665469 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24fc06d8c0 msgr2=0x7f24fc15e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24fad9d700 1 --2- 172.21.15.81:0/1708665469 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24fc06d8c0 0x7f24fc15e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24fad9d700 1 -- 172.21.15.81:0/1708665469 >> v1:172.21.15.117:6789/0 conn(0x7f24fc0b20f0 legacy=0x7f24fc071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24fad9d700 1 -- 172.21.15.81:0/1708665469 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc0b2580 con 0x7f24fc071a50 2024-05-08T23:32:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24fad9d700 1 --2- 172.21.15.81:0/1708665469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24fc071a50 0x7f24fc156d80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009e50 tx=0x7f24f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=849eba2e0f29aee0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24f959a700 1 -- 172.21.15.81:0/1708665469 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f400e070 con 0x7f24fc071a50 2024-05-08T23:32:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24f959a700 1 -- 172.21.15.81:0/1708665469 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f400b910 con 0x7f24fc071a50 2024-05-08T23:32:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24f959a700 1 -- 172.21.15.81:0/1708665469 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f400bcc0 con 0x7f24fc071a50 2024-05-08T23:32:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24fb59e700 1 -- 172.21.15.81:0/1708665469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24fc071a50 msgr2=0x7f24fc156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24fb59e700 1 --2- 172.21.15.81:0/1708665469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24fc071a50 0x7f24fc156d80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009e50 tx=0x7f24f400b330 comp rx=0 tx=0).stop 2024-05-08T23:32:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24fb59e700 1 -- 172.21.15.81:0/1708665469 shutdown_connections 2024-05-08T23:32:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24fb59e700 1 --2- 172.21.15.81:0/1708665469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24fc071a50 0x7f24fc156d80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24fb59e700 1 --2- 172.21.15.81:0/1708665469 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24fc06d8c0 0x7f24fc15e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24fb59e700 1 -- 172.21.15.81:0/1708665469 >> 172.21.15.81:0/1708665469 conn(0x7f24fc0b0de0 msgr2=0x7f24fc0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24fb59e700 1 -- 172.21.15.81:0/1708665469 shutdown_connections 2024-05-08T23:32:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24fb59e700 1 -- 172.21.15.81:0/1708665469 wait complete. 2024-05-08T23:32:49.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24fb59e700 1 Processor -- start 2024-05-08T23:32:49.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.787+0000 7f24fb59e700 1 -- start start 2024-05-08T23:32:49.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fb59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24fc06d8c0 0x7f24fc159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:49.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fb59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24fc071a50 0x7f24fc156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:49.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fb59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24fc099a70 0x7f24fc15feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:49.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fb59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc09a250 con 0x7f24fc06d8c0 2024-05-08T23:32:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fb59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc09a650 con 0x7f24fc071a50 2024-05-08T23:32:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fb59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc09a930 con 0x7f24fc099a70 2024-05-08T23:32:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fa59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24fc06d8c0 0x7f24fc159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fa59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24fc06d8c0 0x7f24fc159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56460/0 (socket says 172.21.15.81:56460) 2024-05-08T23:32:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fa59c700 1 -- 172.21.15.81:0/4113749773 learned_addr learned my addr 172.21.15.81:0/4113749773 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24f9d9b700 1 --2- 172.21.15.81:0/4113749773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24fc071a50 0x7f24fc156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fad9d700 1 --2- 172.21.15.81:0/4113749773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24fc099a70 0x7f24fc15feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fa59c700 1 -- 172.21.15.81:0/4113749773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24fc099a70 msgr2=0x7f24fc15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fa59c700 1 --2- 172.21.15.81:0/4113749773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24fc099a70 0x7f24fc15feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fa59c700 1 -- 172.21.15.81:0/4113749773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24fc071a50 msgr2=0x7f24fc156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fa59c700 1 --2- 172.21.15.81:0/4113749773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24fc071a50 0x7f24fc156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fa59c700 1 -- 172.21.15.81:0/4113749773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f4009b30 con 0x7f24fc06d8c0 2024-05-08T23:32:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24f9d9b700 1 --2- 172.21.15.81:0/4113749773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24fc071a50 0x7f24fc156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fa59c700 1 --2- 172.21.15.81:0/4113749773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24fc06d8c0 0x7f24fc159950 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f24e400e9d0 tx=0x7f24e400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:49.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24f2ffd700 1 -- 172.21.15.81:0/4113749773 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e4009720 con 0x7f24fc06d8c0 2024-05-08T23:32:49.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24f2ffd700 1 -- 172.21.15.81:0/4113749773 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e400ee70 con 0x7f24fc06d8c0 2024-05-08T23:32:49.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fb59e700 1 -- 172.21.15.81:0/4113749773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24fc09ac10 con 0x7f24fc06d8c0 2024-05-08T23:32:49.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24f2ffd700 1 -- 172.21.15.81:0/4113749773 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e400ca50 con 0x7f24fc06d8c0 2024-05-08T23:32:49.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.791+0000 7f24fb59e700 1 -- 172.21.15.81:0/4113749773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24fc1603f0 con 0x7f24fc06d8c0 2024-05-08T23:32:49.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.795+0000 7f24fb59e700 1 -- 172.21.15.81:0/4113749773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24fc0684b0 con 0x7f24fc06d8c0 2024-05-08T23:32:49.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.795+0000 7f24f2ffd700 1 -- 172.21.15.81:0/4113749773 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f24e400cbb0 con 0x7f24fc06d8c0 2024-05-08T23:32:49.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.795+0000 7f24f2ffd700 1 --2- 172.21.15.81:0/4113749773 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f24dc066730 0x7f24dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:49.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.795+0000 7f24f9d9b700 1 --2- 172.21.15.81:0/4113749773 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f24dc066730 0x7f24dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:49.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.795+0000 7f24f2ffd700 1 -- 172.21.15.81:0/4113749773 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24e4084110 con 0x7f24fc06d8c0 2024-05-08T23:32:49.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.795+0000 7f24f9d9b700 1 --2- 172.21.15.81:0/4113749773 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f24dc066730 0x7f24dc068bf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f24fc158440 tx=0x7f24ec007ef0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:49.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:49.799+0000 7f24f2ffd700 1 -- 172.21.15.81:0/4113749773 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f24e4051480 con 0x7f24fc06d8c0 2024-05-08T23:32:50.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.031+0000 7f24fb59e700 1 -- 172.21.15.81:0/4113749773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f24fc06cf00 con 0x7f24fc06d8c0 2024-05-08T23:32:50.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.035+0000 7f24f2ffd700 1 -- 172.21.15.81:0/4113749773 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f24e400ce60 con 0x7f24fc06d8c0 2024-05-08T23:32:50.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.039+0000 7f24fb59e700 1 -- 172.21.15.81:0/4113749773 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f24dc066730 msgr2=0x7f24dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.039+0000 7f24fb59e700 1 --2- 172.21.15.81:0/4113749773 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f24dc066730 0x7f24dc068bf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f24fc158440 tx=0x7f24ec007ef0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.039+0000 7f24fb59e700 1 -- 172.21.15.81:0/4113749773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24fc06d8c0 msgr2=0x7f24fc159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.039+0000 7f24fb59e700 1 --2- 172.21.15.81:0/4113749773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24fc06d8c0 0x7f24fc159950 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f24e400e9d0 tx=0x7f24e400ece0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.039+0000 7f24fb59e700 1 -- 172.21.15.81:0/4113749773 shutdown_connections 2024-05-08T23:32:50.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.039+0000 7f24fb59e700 1 --2- 172.21.15.81:0/4113749773 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f24dc066730 0x7f24dc068bf0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.039+0000 7f24fb59e700 1 --2- 172.21.15.81:0/4113749773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24fc099a70 0x7f24fc15feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.039+0000 7f24fb59e700 1 --2- 172.21.15.81:0/4113749773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24fc071a50 0x7f24fc156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.039+0000 7f24fb59e700 1 --2- 172.21.15.81:0/4113749773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24fc06d8c0 0x7f24fc159950 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.039+0000 7f24fb59e700 1 -- 172.21.15.81:0/4113749773 >> 172.21.15.81:0/4113749773 conn(0x7f24fc0b0de0 msgr2=0x7f24fc15a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:50.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.039+0000 7f24fb59e700 1 -- 172.21.15.81:0/4113749773 shutdown_connections 2024-05-08T23:32:50.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.039+0000 7f24fb59e700 1 -- 172.21.15.81:0/4113749773 wait complete. 2024-05-08T23:32:50.044 INFO:teuthology.orchestra.run.smithi081.stderr:updated caps for client.0 2024-05-08T23:32:50.057 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:50.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38e9ba700 1 Processor -- start 2024-05-08T23:32:50.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38e9ba700 1 -- start start 2024-05-08T23:32:50.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38e9ba700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa388071350 0x7fa3880b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38e9ba700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa388071730 0x7fa388150a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38e9ba700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3880b4890 con 0x7fa388150f40 2024-05-08T23:32:50.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38e9ba700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa38815b470 con 0x7fa388071350 2024-05-08T23:32:50.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38e9ba700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3880b38b0 con 0x7fa388071730 2024-05-08T23:32:50.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38d1b7700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa388071730 0x7fa388150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38d9b8700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa388071350 0x7fa3880b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38d9b8700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa388071350 0x7fa3880b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:40190/0 (socket says 172.21.15.81:40190) 2024-05-08T23:32:50.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38d9b8700 1 -- 172.21.15.81:0/882792534 learned_addr learned my addr 172.21.15.81:0/882792534 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:50.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38e1b9700 1 --1- 172.21.15.81:0/882792534 >> v1:172.21.15.117:6789/0 conn(0x7fa388150f40 0x7fa38815a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:60610/0 (socket says 172.21.15.81:60610) 2024-05-08T23:32:50.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38d9b8700 1 -- 172.21.15.81:0/882792534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa388071730 msgr2=0x7fa388150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:50.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38d9b8700 1 --2- 172.21.15.81:0/882792534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa388071730 0x7fa388150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38d9b8700 1 -- 172.21.15.81:0/882792534 >> v1:172.21.15.117:6789/0 conn(0x7fa388150f40 legacy=0x7fa38815a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38c9b6700 1 -- 172.21.15.81:0/882792534 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540436123 0 0) 0x7fa380001490 con 0x7fa388150f40 2024-05-08T23:32:50.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38d9b8700 1 -- 172.21.15.81:0/882792534 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa388151370 con 0x7fa388071350 2024-05-08T23:32:50.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.159+0000 7fa38d1b7700 1 --2- 172.21.15.81:0/882792534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa388071730 0x7fa388150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:50.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.163+0000 7fa38d9b8700 1 --2- 172.21.15.81:0/882792534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa388071350 0x7fa3880b4350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa378009910 tx=0x7fa378009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79996cb9a8fe60be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.163+0000 7fa38c9b6700 1 -- 172.21.15.81:0/882792534 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378011830 con 0x7fa388071350 2024-05-08T23:32:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.163+0000 7fa38c9b6700 1 -- 172.21.15.81:0/882792534 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa378011990 con 0x7fa388071350 2024-05-08T23:32:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.163+0000 7fa38c9b6700 1 -- 172.21.15.81:0/882792534 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378011d40 con 0x7fa388071350 2024-05-08T23:32:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.163+0000 7fa38e9ba700 1 -- 172.21.15.81:0/882792534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa388071350 msgr2=0x7fa3880b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.163+0000 7fa38e9ba700 1 --2- 172.21.15.81:0/882792534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa388071350 0x7fa3880b4350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa378009910 tx=0x7fa378009c20 comp rx=0 tx=0).stop 2024-05-08T23:32:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.163+0000 7fa38e9ba700 1 -- 172.21.15.81:0/882792534 shutdown_connections 2024-05-08T23:32:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.163+0000 7fa38e9ba700 1 --2- 172.21.15.81:0/882792534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa388071730 0x7fa388150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.163+0000 7fa38e9ba700 1 --2- 172.21.15.81:0/882792534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa388071350 0x7fa3880b4350 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.163+0000 7fa38e9ba700 1 -- 172.21.15.81:0/882792534 >> 172.21.15.81:0/882792534 conn(0x7fa38806b470 msgr2=0x7fa38806b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:50.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.163+0000 7fa38e9ba700 1 -- 172.21.15.81:0/882792534 shutdown_connections 2024-05-08T23:32:50.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.163+0000 7fa38e9ba700 1 -- 172.21.15.81:0/882792534 wait complete. 2024-05-08T23:32:50.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.163+0000 7fa38e9ba700 1 Processor -- start 2024-05-08T23:32:50.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.163+0000 7fa38e9ba700 1 -- start start 2024-05-08T23:32:50.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38e9ba700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa388071350 0x7fa3881f0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38e9ba700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa388071730 0x7fa388147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38e9ba700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa388150f40 0x7fa388156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38e9ba700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3881f0b70 con 0x7fa388071350 2024-05-08T23:32:50.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38e9ba700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3881528b0 con 0x7fa388150f40 2024-05-08T23:32:50.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38e9ba700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa388151ca0 con 0x7fa388071730 2024-05-08T23:32:50.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38d9b8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa388071350 0x7fa3881f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38d9b8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa388071350 0x7fa3881f0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56462/0 (socket says 172.21.15.81:56462) 2024-05-08T23:32:50.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38d9b8700 1 -- 172.21.15.81:0/3330334594 learned_addr learned my addr 172.21.15.81:0/3330334594 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:50.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38e1b9700 1 --2- 172.21.15.81:0/3330334594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa388150f40 0x7fa388156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38d1b7700 1 --2- 172.21.15.81:0/3330334594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa388071730 0x7fa388147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38d9b8700 1 -- 172.21.15.81:0/3330334594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa388071730 msgr2=0x7fa388147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38d9b8700 1 --2- 172.21.15.81:0/3330334594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa388071730 0x7fa388147fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38d9b8700 1 -- 172.21.15.81:0/3330334594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa388150f40 msgr2=0x7fa388156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38d9b8700 1 --2- 172.21.15.81:0/3330334594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa388150f40 0x7fa388156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38d9b8700 1 -- 172.21.15.81:0/3330334594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3840094b0 con 0x7fa388071350 2024-05-08T23:32:50.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38d9b8700 1 --2- 172.21.15.81:0/3330334594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa388071350 0x7fa3881f0630 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa378009f70 tx=0x7fa378004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:50.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa37e7fc700 1 -- 172.21.15.81:0/3330334594 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37801a900 con 0x7fa388071350 2024-05-08T23:32:50.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa37e7fc700 1 -- 172.21.15.81:0/3330334594 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37801f070 con 0x7fa388071350 2024-05-08T23:32:50.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38e9ba700 1 -- 172.21.15.81:0/3330334594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa378009570 con 0x7fa388071350 2024-05-08T23:32:50.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa37e7fc700 1 -- 172.21.15.81:0/3330334594 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37801acc0 con 0x7fa388071350 2024-05-08T23:32:50.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38e9ba700 1 -- 172.21.15.81:0/3330334594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa388152260 con 0x7fa388071350 2024-05-08T23:32:50.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.167+0000 7fa38e9ba700 1 -- 172.21.15.81:0/3330334594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3880672a0 con 0x7fa388071350 2024-05-08T23:32:50.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.171+0000 7fa37e7fc700 1 -- 172.21.15.81:0/3330334594 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa37801b430 con 0x7fa388071350 2024-05-08T23:32:50.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.171+0000 7fa37e7fc700 1 --2- 172.21.15.81:0/3330334594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa36c066730 0x7fa36c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.171+0000 7fa37e7fc700 1 -- 172.21.15.81:0/3330334594 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa378021070 con 0x7fa388071350 2024-05-08T23:32:50.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.171+0000 7fa38d1b7700 1 --2- 172.21.15.81:0/3330334594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa36c066730 0x7fa36c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.171+0000 7fa38d1b7700 1 --2- 172.21.15.81:0/3330334594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa36c066730 0x7fa36c068bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa388146ec0 tx=0x7fa384009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:50.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.175+0000 7fa37e7fc700 1 -- 172.21.15.81:0/3330334594 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa378052da0 con 0x7fa388071350 2024-05-08T23:32:50.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.383+0000 7fa38e9ba700 1 -- 172.21.15.81:0/3330334594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3880cc3a0 con 0x7fa388071350 2024-05-08T23:32:50.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.387+0000 7fa37e7fc700 1 -- 172.21.15.81:0/3330334594 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4743 (secure 0 0 0) 0x7fa378015020 con 0x7fa388071350 2024-05-08T23:32:50.387 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:50.387 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/2105335102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":2105335102},{"type":"v1","addr":"172.21.15.117:6834","nonce":2105335102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2900127785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2900127785},{"type":"v1","addr":"172.21.15.81:6835","nonce":2900127785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1899281779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1899281779},{"type":"v1","addr":"172.21.15.81:6837","nonce":1899281779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:32:40.852853+0000","modified":"2024-05-08T23:32:43.867414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":15311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_15311":{"gid":15311,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/3401411259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3401411259},{"type":"v1","addr":"172.21.15.117:6835","nonce":3401411259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:32:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.387+0000 7fa38e9ba700 1 -- 172.21.15.81:0/3330334594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa36c066730 msgr2=0x7fa36c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.387+0000 7fa38e9ba700 1 --2- 172.21.15.81:0/3330334594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa36c066730 0x7fa36c068bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa388146ec0 tx=0x7fa384009240 comp rx=0 tx=0).stop 2024-05-08T23:32:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.387+0000 7fa38e9ba700 1 -- 172.21.15.81:0/3330334594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa388071350 msgr2=0x7fa3881f0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.387+0000 7fa38e9ba700 1 --2- 172.21.15.81:0/3330334594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa388071350 0x7fa3881f0630 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa378009f70 tx=0x7fa378004770 comp rx=0 tx=0).stop 2024-05-08T23:32:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.387+0000 7fa38e9ba700 1 -- 172.21.15.81:0/3330334594 shutdown_connections 2024-05-08T23:32:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.387+0000 7fa38e9ba700 1 --2- 172.21.15.81:0/3330334594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa36c066730 0x7fa36c068bf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.387+0000 7fa38e9ba700 1 --2- 172.21.15.81:0/3330334594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa388150f40 0x7fa388156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.387+0000 7fa38e9ba700 1 --2- 172.21.15.81:0/3330334594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa388071730 0x7fa388147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.387+0000 7fa38e9ba700 1 --2- 172.21.15.81:0/3330334594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa388071350 0x7fa3881f0630 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.387+0000 7fa38e9ba700 1 -- 172.21.15.81:0/3330334594 >> 172.21.15.81:0/3330334594 conn(0x7fa38806b470 msgr2=0x7fa3880ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.387+0000 7fa38e9ba700 1 -- 172.21.15.81:0/3330334594 shutdown_connections 2024-05-08T23:32:50.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.387+0000 7fa38e9ba700 1 -- 172.21.15.81:0/3330334594 wait complete. 2024-05-08T23:32:50.391 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 23 2024-05-08T23:32:50.803 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:32:40.852853+0000', 'modified': '2024-05-08T23:32:43.867414+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 15311}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_15311': {'gid': 15311, 'name': 'b', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6835/3401411259', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6832', 'nonce': 3401411259}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 3401411259}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:32:50.803 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T23:32:50.803 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T23:32:50.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.926+0000 7fa913fff700 1 Processor -- start 2024-05-08T23:32:50.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.926+0000 7fa913fff700 1 -- start start 2024-05-08T23:32:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa913fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa9140d37c0 0x7fa9140d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa913fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa9140b2ea0 0x7fa9140b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa913fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9140d3c20 con 0x7fa9140d1690 2024-05-08T23:32:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa913fff700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9140d3e50 con 0x7fa9140b2ea0 2024-05-08T23:32:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa913fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9140d1460 con 0x7fa9140d37c0 2024-05-08T23:32:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa9137fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa9140b2ea0 0x7fa9140b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa9137fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa9140b2ea0 0x7fa9140b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56464/0 (socket says 172.21.15.81:56464) 2024-05-08T23:32:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa9137fe700 1 -- 172.21.15.81:0/3180989862 learned_addr learned my addr 172.21.15.81:0/3180989862 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa90a7fc700 1 --2- 172.21.15.81:0/3180989862 >> v2:172.21.15.117:3300/0 conn(0x7fa9140d37c0 0x7fa9140d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa9137fe700 1 -- 172.21.15.81:0/3180989862 >> v2:172.21.15.117:3300/0 conn(0x7fa9140d37c0 msgr2=0x7fa9140d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa9137fe700 1 --2- 172.21.15.81:0/3180989862 >> v2:172.21.15.117:3300/0 conn(0x7fa9140d37c0 0x7fa9140d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa9137fe700 1 -- 172.21.15.81:0/3180989862 >> v1:172.21.15.117:6789/0 conn(0x7fa9140d1690 legacy=0x7fa9140d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa9137fe700 1 -- 172.21.15.81:0/3180989862 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9140ce870 con 0x7fa9140b2ea0 2024-05-08T23:32:50.934 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa9137fe700 1 --2- 172.21.15.81:0/3180989862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9140b2ea0 0x7fa9140b3330 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa91408d360 tx=0x7fa90c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2851333134ab8bd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:50.934 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa9127fc700 1 -- 172.21.15.81:0/3180989862 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00e070 con 0x7fa9140b2ea0 2024-05-08T23:32:50.934 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa9127fc700 1 -- 172.21.15.81:0/3180989862 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c003a40 con 0x7fa9140b2ea0 2024-05-08T23:32:50.934 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa9127fc700 1 -- 172.21.15.81:0/3180989862 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c003df0 con 0x7fa9140b2ea0 2024-05-08T23:32:50.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa913fff700 1 -- 172.21.15.81:0/3180989862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9140b2ea0 msgr2=0x7fa9140b3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa913fff700 1 --2- 172.21.15.81:0/3180989862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9140b2ea0 0x7fa9140b3330 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa91408d360 tx=0x7fa90c00b330 comp rx=0 tx=0).stop 2024-05-08T23:32:50.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa913fff700 1 -- 172.21.15.81:0/3180989862 shutdown_connections 2024-05-08T23:32:50.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa913fff700 1 --2- 172.21.15.81:0/3180989862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9140b2ea0 0x7fa9140b3330 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa913fff700 1 --2- 172.21.15.81:0/3180989862 >> v2:172.21.15.117:3300/0 conn(0x7fa9140d37c0 0x7fa9140d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa913fff700 1 -- 172.21.15.81:0/3180989862 >> 172.21.15.81:0/3180989862 conn(0x7fa91409c3c0 msgr2=0x7fa9140d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:50.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa913fff700 1 -- 172.21.15.81:0/3180989862 shutdown_connections 2024-05-08T23:32:50.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.930+0000 7fa913fff700 1 -- 172.21.15.81:0/3180989862 wait complete. 2024-05-08T23:32:50.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa913fff700 1 Processor -- start 2024-05-08T23:32:50.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa913fff700 1 -- start start 2024-05-08T23:32:50.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa913fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9140b2ea0 0x7fa9141f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa913fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9140d1690 0x7fa91415fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa913fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9140d37c0 0x7fa914159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa913fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140cba70 con 0x7fa9140d37c0 2024-05-08T23:32:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa913fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140b87a0 con 0x7fa9140b2ea0 2024-05-08T23:32:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa913fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140b8fb0 con 0x7fa9140d1690 2024-05-08T23:32:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa9137fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9140d37c0 0x7fa914159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa9137fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9140d37c0 0x7fa914159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56476/0 (socket says 172.21.15.81:56476) 2024-05-08T23:32:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa9137fe700 1 -- 172.21.15.81:0/2285853846 learned_addr learned my addr 172.21.15.81:0/2285853846 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa912ffd700 1 --2- 172.21.15.81:0/2285853846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9140b2ea0 0x7fa9141f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa90a7fc700 1 --2- 172.21.15.81:0/2285853846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9140d1690 0x7fa91415fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa9137fe700 1 -- 172.21.15.81:0/2285853846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9140d1690 msgr2=0x7fa91415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa9137fe700 1 --2- 172.21.15.81:0/2285853846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9140d1690 0x7fa91415fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa9137fe700 1 -- 172.21.15.81:0/2285853846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9140b2ea0 msgr2=0x7fa9141f6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa9137fe700 1 --2- 172.21.15.81:0/2285853846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9140b2ea0 0x7fa9141f6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa9137fe700 1 -- 172.21.15.81:0/2285853846 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c009b30 con 0x7fa9140d37c0 2024-05-08T23:32:50.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa9137fe700 1 --2- 172.21.15.81:0/2285853846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9140d37c0 0x7fa914159620 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00b370 tx=0x7fa90c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:50.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa90bfff700 1 -- 172.21.15.81:0/2285853846 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c004440 con 0x7fa9140d37c0 2024-05-08T23:32:50.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa90bfff700 1 -- 172.21.15.81:0/2285853846 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c0045c0 con 0x7fa9140d37c0 2024-05-08T23:32:50.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa90bfff700 1 -- 172.21.15.81:0/2285853846 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c004970 con 0x7fa9140d37c0 2024-05-08T23:32:50.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa913fff700 1 -- 172.21.15.81:0/2285853846 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa914160430 con 0x7fa9140d37c0 2024-05-08T23:32:50.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.934+0000 7fa913fff700 1 -- 172.21.15.81:0/2285853846 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9140b8cd0 con 0x7fa9140d37c0 2024-05-08T23:32:50.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa90bfff700 1 -- 172.21.15.81:0/2285853846 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa90c012430 con 0x7fa9140d37c0 2024-05-08T23:32:50.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa91c8af740 1 Processor -- start 2024-05-08T23:32:50.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa91c8af740 1 -- start start 2024-05-08T23:32:50.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa90bfff700 1 --2- 172.21.15.81:0/2285853846 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa8f4066760 0x7fa8f4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa90a7fc700 1 --2- 172.21.15.81:0/2285853846 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa8f4066760 0x7fa8f4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa90bfff700 1 -- 172.21.15.81:0/2285853846 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa90c0854f0 con 0x7fa9140d37c0 2024-05-08T23:32:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa91c8af740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2940b00 0x2940ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa91c8af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x2908470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa91c8af740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2941420 0x29089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa91c8af740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x2867d80 con 0x2940b00 2024-05-08T23:32:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa91c8af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x2865510 con 0x2937be0 2024-05-08T23:32:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa91c8af740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x2860e00 con 0x2941420 2024-05-08T23:32:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa912ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2940b00 0x2940ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa912ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2940b00 0x2940ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56492/0 (socket says 172.21.15.81:56492) 2024-05-08T23:32:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa912ffd700 1 -- 172.21.15.81:0/2165718157 learned_addr learned my addr 172.21.15.81:0/2165718157 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa90a7fc700 1 --2- 172.21.15.81:0/2165718157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x2908470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa9137fe700 1 --2- 172.21.15.81:0/2165718157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2941420 0x29089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa912ffd700 1 -- 172.21.15.81:0/2165718157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2941420 msgr2=0x29089b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa912ffd700 1 --2- 172.21.15.81:0/2165718157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2941420 0x29089b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa912ffd700 1 -- 172.21.15.81:0/2165718157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 msgr2=0x2908470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa912ffd700 1 --2- 172.21.15.81:0/2165718157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x2908470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa912ffd700 1 -- 172.21.15.81:0/2165718157 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x26d3960 con 0x2940b00 2024-05-08T23:32:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.938+0000 7fa90a7fc700 1 --2- 172.21.15.81:0/2285853846 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa8f4066760 0x7fa8f4068c20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa9140affa0 tx=0x7fa8fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.942+0000 7fa912ffd700 1 --2- 172.21.15.81:0/2165718157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2940b00 0x2940ee0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa904009a40 tx=0x7fa904009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247551fb12ddcc40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.942+0000 7fa908ff9700 1 -- 172.21.15.81:0/2165718157 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904009540 con 0x2940b00 2024-05-08T23:32:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.942+0000 7fa908ff9700 1 -- 172.21.15.81:0/2165718157 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90400e8d0 con 0x2940b00 2024-05-08T23:32:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.942+0000 7fa908ff9700 1 -- 172.21.15.81:0/2165718157 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90400ec40 con 0x2940b00 2024-05-08T23:32:50.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.942+0000 7fa91c8af740 1 -- 172.21.15.81:0/2165718157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2940b00 msgr2=0x2940ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.942+0000 7fa91c8af740 1 --2- 172.21.15.81:0/2165718157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2940b00 0x2940ee0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa904009a40 tx=0x7fa904009d50 comp rx=0 tx=0).stop 2024-05-08T23:32:50.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.942+0000 7fa91c8af740 1 -- 172.21.15.81:0/2165718157 shutdown_connections 2024-05-08T23:32:50.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.942+0000 7fa91c8af740 1 --2- 172.21.15.81:0/2165718157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2941420 0x29089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.942+0000 7fa91c8af740 1 --2- 172.21.15.81:0/2165718157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x2908470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.942+0000 7fa91c8af740 1 --2- 172.21.15.81:0/2165718157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2940b00 0x2940ee0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.942+0000 7fa91c8af740 1 -- 172.21.15.81:0/2165718157 >> 172.21.15.81:0/2165718157 conn(0x26884d0 msgr2=0x2922250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:50.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.942+0000 7fa91c8af740 1 -- 172.21.15.81:0/2165718157 shutdown_connections 2024-05-08T23:32:50.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.942+0000 7fa91c8af740 1 -- 172.21.15.81:0/2165718157 wait complete. 2024-05-08T23:32:50.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.942+0000 7fa91c8af740 1 -- start start 2024-05-08T23:32:50.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.942+0000 7fa91c8af740 1 Processor -- start 2024-05-08T23:32:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x2946ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 0x2949ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2941420 0x293f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x281e730 con 0x2941420 2024-05-08T23:32:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x28245b0 con 0x2937be0 2024-05-08T23:32:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x270caa0 con 0x2940b00 2024-05-08T23:32:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 10 client.0 _ll_register_callbacks cb 0x285a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:32:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa9137fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2941420 0x293f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa9137fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2941420 0x293f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56498/0 (socket says 172.21.15.81:56498) 2024-05-08T23:32:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa90a7fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 0x2949ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa912ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x2946ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa9137fe700 1 -- 172.21.15.81:0/3300258440 learned_addr learned my addr 172.21.15.81:0/3300258440 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa9137fe700 1 -- 172.21.15.81:0/3300258440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 msgr2=0x2949ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa9137fe700 1 --2- 172.21.15.81:0/3300258440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 0x2949ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa9137fe700 1 -- 172.21.15.81:0/3300258440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 msgr2=0x2946ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa9137fe700 1 --2- 172.21.15.81:0/3300258440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x2946ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa9137fe700 1 -- 172.21.15.81:0/3300258440 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9040096a0 con 0x2941420 2024-05-08T23:32:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa9137fe700 1 --2- 172.21.15.81:0/3300258440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2941420 0x293f960 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00bfd0 tx=0x7fa90c056f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757fbb8253e1bbfe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa8f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-08T23:32:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa8f1ffb700 1 -- 172.21.15.81:0/3300258440 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c05eac0 con 0x2941420 2024-05-08T23:32:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa8f1ffb700 1 -- 172.21.15.81:0/3300258440 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa90c083e40 con 0x2941420 2024-05-08T23:32:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 1 -- 172.21.15.81:0/3300258440 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x26951e0 con 0x2941420 2024-05-08T23:32:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa8f1ffb700 1 -- 172.21.15.81:0/3300258440 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa90c05a170 con 0x2941420 2024-05-08T23:32:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 10 client.6072 fetch_fsmap learned FSMap version 23 2024-05-08T23:32:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 1 -- 172.21.15.81:0/3300258440 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x2827220 con 0x2941420 2024-05-08T23:32:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa8f1ffb700 1 -- 172.21.15.81:0/3300258440 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x26951e0 con 0x2941420 2024-05-08T23:32:50.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa8f1ffb700 1 -- 172.21.15.81:0/3300258440 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa90c055bc0 con 0x2941420 2024-05-08T23:32:50.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 10 client.6072 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:32:50.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 10 client.6072 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:32:50.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 20 client.6072 populate_metadata read hostname 'smithi081' 2024-05-08T23:32:50.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 1 --2- 172.21.15.81:0/3300258440 >> [v2:172.21.15.117:6832/3401411259,v1:172.21.15.117:6835/3401411259] conn(0x2939140 0x2939520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 4 client.6072 mds_command: new command op to 15311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:32:50.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa91c8af740 1 -- 172.21.15.81:0/3300258440 --> [v2:172.21.15.117:6832/3401411259,v1:172.21.15.117:6835/3401411259] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x26df460 con 0x2939140 2024-05-08T23:32:50.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.946+0000 7fa912ffd700 1 --2- 172.21.15.81:0/3300258440 >> [v2:172.21.15.117:6832/3401411259,v1:172.21.15.117:6835/3401411259] conn(0x2939140 0x2939520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa8f1ffb700 10 client.6072 ms_handle_connect on v2:172.21.15.117:6832/3401411259 2024-05-08T23:32:50.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa912ffd700 1 --2- 172.21.15.81:0/3300258440 >> [v2:172.21.15.117:6832/3401411259,v1:172.21.15.117:6835/3401411259] conn(0x2939140 0x2939520 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a37543d5c19e9112 server_cookie=abb00aa33657396a in_seq=0 out_seq=0 2024-05-08T23:32:50.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa8f1ffb700 1 -- 172.21.15.81:0/3300258440 <== mds.0 v2:172.21.15.117:6832/3401411259 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x26df460 con 0x2939140 2024-05-08T23:32:50.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa8f1ffb700 10 client.6072 handle_command_reply: tid=0 2024-05-08T23:32:50.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa91c8af740 1 client.6072 shutdown 2024-05-08T23:32:50.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa91c8af740 1 -- 172.21.15.81:0/3300258440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2941420 msgr2=0x293f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa91c8af740 1 --2- 172.21.15.81:0/3300258440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2941420 0x293f960 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00bfd0 tx=0x7fa90c056f10 comp rx=0 tx=0).stop 2024-05-08T23:32:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa91c8af740 1 -- 172.21.15.81:0/3300258440 shutdown_connections 2024-05-08T23:32:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa91c8af740 1 --2- 172.21.15.81:0/3300258440 >> [v2:172.21.15.117:6832/3401411259,v1:172.21.15.117:6835/3401411259] conn(0x2939140 0x2939520 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa91c8af740 1 --2- 172.21.15.81:0/3300258440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2941420 0x293f960 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa8f1ffb700 0 client.6072 ms_handle_reset on v2:172.21.15.117:6832/3401411259 2024-05-08T23:32:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa91c8af740 1 --2- 172.21.15.81:0/3300258440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 0x2949ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa91c8af740 1 --2- 172.21.15.81:0/3300258440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x2946ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa91c8af740 1 -- 172.21.15.81:0/3300258440 >> 172.21.15.81:0/3300258440 conn(0x26884d0 msgr2=0x2921990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa91c8af740 1 -- 172.21.15.81:0/3300258440 shutdown_connections 2024-05-08T23:32:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa91c8af740 1 -- 172.21.15.81:0/3300258440 wait complete. 2024-05-08T23:32:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.950+0000 7fa91c8af740 20 client.6072 trim_cache size 0 max 16384 2024-05-08T23:32:50.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.966+0000 7fa91c8af740 1 Processor -- start 2024-05-08T23:32:50.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.966+0000 7fa91c8af740 1 -- start start 2024-05-08T23:32:50.972 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.966+0000 7fa91c8af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x28f8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.972 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.966+0000 7fa91c8af740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2939140 0x28f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.972 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.966+0000 7fa91c8af740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 0x294f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.972 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.966+0000 7fa91c8af740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x28fa2f0 con 0x2939140 2024-05-08T23:32:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.966+0000 7fa91c8af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x281d690 con 0x2937be0 2024-05-08T23:32:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.966+0000 7fa91c8af740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x2937a20 con 0x2940b00 2024-05-08T23:32:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.966+0000 7fa90a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2939140 0x28f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.966+0000 7fa90a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2939140 0x28f8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56504/0 (socket says 172.21.15.81:56504) 2024-05-08T23:32:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa90a7fc700 1 -- 172.21.15.81:0/3899430602 learned_addr learned my addr 172.21.15.81:0/3899430602 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa912ffd700 1 --2- 172.21.15.81:0/3899430602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x28f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa9137fe700 1 --2- 172.21.15.81:0/3899430602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 0x294f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa90a7fc700 1 -- 172.21.15.81:0/3899430602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 msgr2=0x294f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa90a7fc700 1 --2- 172.21.15.81:0/3899430602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 0x294f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa90a7fc700 1 -- 172.21.15.81:0/3899430602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 msgr2=0x28f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa90a7fc700 1 --2- 172.21.15.81:0/3899430602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x28f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa90a7fc700 1 -- 172.21.15.81:0/3899430602 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x280d450 con 0x2939140 2024-05-08T23:32:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa9137fe700 1 --2- 172.21.15.81:0/3899430602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 0x294f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa90a7fc700 1 --2- 172.21.15.81:0/3899430602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2939140 0x28f8d50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc010960 tx=0x7fa8fc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef9f2de1b88d3b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:50.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa9097fa700 1 -- 172.21.15.81:0/3899430602 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc004300 con 0x2939140 2024-05-08T23:32:50.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa9097fa700 1 -- 172.21.15.81:0/3899430602 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc004460 con 0x2939140 2024-05-08T23:32:50.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa9097fa700 1 -- 172.21.15.81:0/3899430602 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0045d0 con 0x2939140 2024-05-08T23:32:50.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa91c8af740 1 -- 172.21.15.81:0/3899430602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2939140 msgr2=0x28f8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa91c8af740 1 --2- 172.21.15.81:0/3899430602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2939140 0x28f8d50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc010960 tx=0x7fa8fc018000 comp rx=0 tx=0).stop 2024-05-08T23:32:50.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa91c8af740 1 -- 172.21.15.81:0/3899430602 shutdown_connections 2024-05-08T23:32:50.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa91c8af740 1 --2- 172.21.15.81:0/3899430602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 0x294f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa91c8af740 1 --2- 172.21.15.81:0/3899430602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2939140 0x28f8d50 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa91c8af740 1 --2- 172.21.15.81:0/3899430602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x28f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa91c8af740 1 -- 172.21.15.81:0/3899430602 >> 172.21.15.81:0/3899430602 conn(0x26884d0 msgr2=0x29224a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:50.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa91c8af740 1 -- 172.21.15.81:0/3899430602 shutdown_connections 2024-05-08T23:32:50.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa91c8af740 1 -- 172.21.15.81:0/3899430602 wait complete. 2024-05-08T23:32:50.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa91c8af740 1 -- start start 2024-05-08T23:32:50.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.970+0000 7fa91c8af740 1 Processor -- start 2024-05-08T23:32:50.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa91c8af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x295e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa91c8af740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2939140 0x28f7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa91c8af740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 0x28f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa91c8af740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x2949180 con 0x2939140 2024-05-08T23:32:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa91c8af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x28fafb0 con 0x2937be0 2024-05-08T23:32:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa91c8af740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x2922db0 con 0x2940b00 2024-05-08T23:32:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa91c8af740 10 client.0 _ll_register_callbacks cb 0x7fa904009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:32:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa90a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2939140 0x28f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa90a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2939140 0x28f7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56508/0 (socket says 172.21.15.81:56508) 2024-05-08T23:32:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa90a7fc700 1 -- 172.21.15.81:0/144531130 learned_addr learned my addr 172.21.15.81:0/144531130 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa912ffd700 1 --2- 172.21.15.81:0/144531130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x295e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa9137fe700 1 --2- 172.21.15.81:0/144531130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 0x28f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa90a7fc700 1 -- 172.21.15.81:0/144531130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 msgr2=0x28f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa90a7fc700 1 --2- 172.21.15.81:0/144531130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 0x28f7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa90a7fc700 1 -- 172.21.15.81:0/144531130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 msgr2=0x295e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:50.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa90a7fc700 1 --2- 172.21.15.81:0/144531130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x295e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa90a7fc700 1 -- 172.21.15.81:0/144531130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8fc00e040 con 0x2939140 2024-05-08T23:32:50.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa90a7fc700 1 --2- 172.21.15.81:0/144531130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2939140 0x28f7760 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc007840 tx=0x7fa8fc0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1f1e12976539ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:50.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa8f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-08T23:32:50.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa8f1ffb700 1 -- 172.21.15.81:0/144531130 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc032070 con 0x2939140 2024-05-08T23:32:50.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa8f1ffb700 1 -- 172.21.15.81:0/144531130 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8fc00f040 con 0x2939140 2024-05-08T23:32:50.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa91c8af740 1 -- 172.21.15.81:0/144531130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa90400e8d0 con 0x2939140 2024-05-08T23:32:50.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa8f1ffb700 1 -- 172.21.15.81:0/144531130 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8fc007040 con 0x2939140 2024-05-08T23:32:50.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa91c8af740 10 client.6078 fetch_fsmap learned FSMap version 23 2024-05-08T23:32:50.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa91c8af740 1 -- 172.21.15.81:0/144531130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x2801720 con 0x2939140 2024-05-08T23:32:50.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.974+0000 7fa8f1ffb700 1 -- 172.21.15.81:0/144531130 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90400e8d0 con 0x2939140 2024-05-08T23:32:50.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa8f1ffb700 1 -- 172.21.15.81:0/144531130 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa8fc016790 con 0x2939140 2024-05-08T23:32:50.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 10 client.6078 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T23:32:50.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 10 client.6078 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:32:50.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 20 client.6078 populate_metadata read hostname 'smithi081' 2024-05-08T23:32:50.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 1 --2- 172.21.15.81:0/144531130 >> [v2:172.21.15.117:6832/3401411259,v1:172.21.15.117:6835/3401411259] conn(0x2941420 0x28f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 4 client.6078 mds_command: new command op to 15311 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:32:50.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 1 -- 172.21.15.81:0/144531130 --> [v2:172.21.15.117:6832/3401411259,v1:172.21.15.117:6835/3401411259] -- command(tid 0: {"prefix": "status"}) v1 -- 0x27041e0 con 0x2941420 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa912ffd700 1 --2- 172.21.15.81:0/144531130 >> [v2:172.21.15.117:6832/3401411259,v1:172.21.15.117:6835/3401411259] conn(0x2941420 0x28f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa8f1ffb700 10 client.6078 ms_handle_connect on v2:172.21.15.117:6832/3401411259 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa912ffd700 1 --2- 172.21.15.81:0/144531130 >> [v2:172.21.15.117:6832/3401411259,v1:172.21.15.117:6835/3401411259] conn(0x2941420 0x28f81e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ca2c1b83068d37 server_cookie=ce7ca10d7673dab2 in_seq=0 out_seq=0 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa8f1ffb700 1 -- 172.21.15.81:0/144531130 <== mds.0 v2:172.21.15.117:6832/3401411259 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x27041e0 con 0x2941420 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa8f1ffb700 10 client.6078 handle_command_reply: tid=0 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 1 client.6078 shutdown 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 1 -- 172.21.15.81:0/144531130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2939140 msgr2=0x28f7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 1 --2- 172.21.15.81:0/144531130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2939140 0x28f7760 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc007840 tx=0x7fa8fc0047e0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 1 -- 172.21.15.81:0/144531130 shutdown_connections 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 1 --2- 172.21.15.81:0/144531130 >> [v2:172.21.15.117:6832/3401411259,v1:172.21.15.117:6835/3401411259] conn(0x2941420 0x28f81e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 1 --2- 172.21.15.81:0/144531130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x2940b00 0x28f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 1 --2- 172.21.15.81:0/144531130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x2939140 0x28f7760 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa8f1ffb700 0 client.6078 ms_handle_reset on v2:172.21.15.117:6832/3401411259 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 1 --2- 172.21.15.81:0/144531130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x2937be0 0x295e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 1 -- 172.21.15.81:0/144531130 >> 172.21.15.81:0/144531130 conn(0x26884d0 msgr2=0x2945820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 1 -- 172.21.15.81:0/144531130 shutdown_connections 2024-05-08T23:32:50.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 1 -- 172.21.15.81:0/144531130 wait complete. 2024-05-08T23:32:50.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa91c8af740 20 client.6078 trim_cache size 0 max 16384 2024-05-08T23:32:50.984 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:32:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_fsid": "dee3de5e-bc15-439a-9578-b5583aefc890", 2024-05-08T23:32:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "whoami": 0, 2024-05-08T23:32:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 15311, 2024-05-08T23:32:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "want_state": "up:active", 2024-05-08T23:32:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "state": "up:active", 2024-05-08T23:32:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs", 2024-05-08T23:32:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "rank_uptime": 10.124867455, 2024-05-08T23:32:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: "mdsmap_epoch": 23, 2024-05-08T23:32:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch": 52, 2024-05-08T23:32:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch_barrier": 52, 2024-05-08T23:32:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: "uptime": 14.572588850000001 2024-05-08T23:32:50.985 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:32:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa913fff700 1 -- 172.21.15.81:0/2285853846 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa8f4066760 msgr2=0x7fa8f4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa913fff700 1 --2- 172.21.15.81:0/2285853846 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa8f4066760 0x7fa8f4068c20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa9140affa0 tx=0x7fa8fc008040 comp rx=0 tx=0).stop 2024-05-08T23:32:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa913fff700 1 -- 172.21.15.81:0/2285853846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9140d37c0 msgr2=0x7fa914159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa913fff700 1 --2- 172.21.15.81:0/2285853846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9140d37c0 0x7fa914159620 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00b370 tx=0x7fa90c004af0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa913fff700 1 -- 172.21.15.81:0/2285853846 shutdown_connections 2024-05-08T23:32:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa913fff700 1 --2- 172.21.15.81:0/2285853846 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa8f4066760 0x7fa8f4068c20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa913fff700 1 --2- 172.21.15.81:0/2285853846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9140d37c0 0x7fa914159620 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa913fff700 1 --2- 172.21.15.81:0/2285853846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9140d1690 0x7fa91415fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa913fff700 1 --2- 172.21.15.81:0/2285853846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9140b2ea0 0x7fa9141f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa913fff700 1 -- 172.21.15.81:0/2285853846 >> 172.21.15.81:0/2285853846 conn(0x7fa91409c3c0 msgr2=0x7fa9140d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa913fff700 1 -- 172.21.15.81:0/2285853846 shutdown_connections 2024-05-08T23:32:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:50.978+0000 7fa913fff700 1 -- 172.21.15.81:0/2285853846 wait complete. 2024-05-08T23:32:50.996 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:32:51.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa35a764700 1 Processor -- start 2024-05-08T23:32:51.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa35a764700 1 -- start start 2024-05-08T23:32:51.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa35a764700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa354159f10 0x7fa35406d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa35a764700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35415a730 con 0x7fa35406efa0 2024-05-08T23:32:51.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa35a764700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3540d0cd0 con 0x7fa35415a2f0 2024-05-08T23:32:51.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa35a764700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35406d8b0 con 0x7fa354159f10 2024-05-08T23:32:51.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa359762700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa354159f10 0x7fa35406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa359f63700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa35406efa0 0x7fa3540b2eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49550/0 (socket says 172.21.15.81:49550) 2024-05-08T23:32:51.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa359f63700 1 -- 172.21.15.81:0/2188167558 learned_addr learned my addr 172.21.15.81:0/2188167558 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:51.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa359762700 1 -- 172.21.15.81:0/2188167558 >> v1:172.21.15.117:6789/0 conn(0x7fa35415a2f0 legacy=0x7fa35406e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa359762700 1 -- 172.21.15.81:0/2188167558 >> v1:172.21.15.81:6789/0 conn(0x7fa35406efa0 legacy=0x7fa3540b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa359762700 1 -- 172.21.15.81:0/2188167558 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35406db90 con 0x7fa354159f10 2024-05-08T23:32:51.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa359762700 1 --2- 172.21.15.81:0/2188167558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa354159f10 0x7fa35406d220 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa350009850 tx=0x7fa350009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f7278adcb099c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:51.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa34bfff700 1 -- 172.21.15.81:0/2188167558 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350011830 con 0x7fa354159f10 2024-05-08T23:32:51.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa34bfff700 1 -- 172.21.15.81:0/2188167558 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa350011990 con 0x7fa354159f10 2024-05-08T23:32:51.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa34bfff700 1 -- 172.21.15.81:0/2188167558 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350011d20 con 0x7fa354159f10 2024-05-08T23:32:51.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa35a764700 1 -- 172.21.15.81:0/2188167558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa354159f10 msgr2=0x7fa35406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa35a764700 1 --2- 172.21.15.81:0/2188167558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa354159f10 0x7fa35406d220 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa350009850 tx=0x7fa350009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:51.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa35a764700 1 -- 172.21.15.81:0/2188167558 shutdown_connections 2024-05-08T23:32:51.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa35a764700 1 --2- 172.21.15.81:0/2188167558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa354159f10 0x7fa35406d220 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa35a764700 1 -- 172.21.15.81:0/2188167558 >> 172.21.15.81:0/2188167558 conn(0x7fa3540cfda0 msgr2=0x7fa3540d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:51.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa35a764700 1 -- 172.21.15.81:0/2188167558 shutdown_connections 2024-05-08T23:32:51.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.106+0000 7fa35a764700 1 -- 172.21.15.81:0/2188167558 wait complete. 2024-05-08T23:32:51.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa35a764700 1 Processor -- start 2024-05-08T23:32:51.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa35a764700 1 -- start start 2024-05-08T23:32:51.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa35a764700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa35406efa0 0x7fa354140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa35a764700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa354159f10 0x7fa35414e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa35a764700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa35415a2f0 0x7fa3540b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa35a764700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35414fe20 con 0x7fa35406efa0 2024-05-08T23:32:51.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa35a764700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35414f1a0 con 0x7fa354159f10 2024-05-08T23:32:51.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa35a764700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35414f480 con 0x7fa35415a2f0 2024-05-08T23:32:51.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa359762700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa35406efa0 0x7fa354140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa359762700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa35406efa0 0x7fa354140750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56530/0 (socket says 172.21.15.81:56530) 2024-05-08T23:32:51.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa359762700 1 -- 172.21.15.81:0/312839249 learned_addr learned my addr 172.21.15.81:0/312839249 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:51.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa358f61700 1 --2- 172.21.15.81:0/312839249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa354159f10 0x7fa35414e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa359f63700 1 --2- 172.21.15.81:0/312839249 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa35415a2f0 0x7fa3540b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa359762700 1 -- 172.21.15.81:0/312839249 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa35415a2f0 msgr2=0x7fa3540b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa359762700 1 --2- 172.21.15.81:0/312839249 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa35415a2f0 0x7fa3540b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa359762700 1 -- 172.21.15.81:0/312839249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa354159f10 msgr2=0x7fa35414e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa359762700 1 --2- 172.21.15.81:0/312839249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa354159f10 0x7fa35414e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa359762700 1 -- 172.21.15.81:0/312839249 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3500094b0 con 0x7fa35406efa0 2024-05-08T23:32:51.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa359762700 1 --2- 172.21.15.81:0/312839249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa35406efa0 0x7fa354140750 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa35415a8c0 tx=0x7fa350004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:51.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa349ffb700 1 -- 172.21.15.81:0/312839249 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3500045d0 con 0x7fa35406efa0 2024-05-08T23:32:51.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa349ffb700 1 -- 172.21.15.81:0/312839249 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35001f070 con 0x7fa35406efa0 2024-05-08T23:32:51.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa35a764700 1 -- 172.21.15.81:0/312839249 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35414f760 con 0x7fa35406efa0 2024-05-08T23:32:51.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa349ffb700 1 -- 172.21.15.81:0/312839249 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350003900 con 0x7fa35406efa0 2024-05-08T23:32:51.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.110+0000 7fa35a764700 1 -- 172.21.15.81:0/312839249 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa354140c90 con 0x7fa35406efa0 2024-05-08T23:32:51.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.114+0000 7fa35a764700 1 -- 172.21.15.81:0/312839249 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35414ff60 con 0x7fa35406efa0 2024-05-08T23:32:51.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.114+0000 7fa349ffb700 1 -- 172.21.15.81:0/312839249 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa350003a60 con 0x7fa35406efa0 2024-05-08T23:32:51.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.114+0000 7fa349ffb700 1 --2- 172.21.15.81:0/312839249 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa338066730 0x7fa338068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.114+0000 7fa349ffb700 1 -- 172.21.15.81:0/312839249 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa350021070 con 0x7fa35406efa0 2024-05-08T23:32:51.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.114+0000 7fa358f61700 1 --2- 172.21.15.81:0/312839249 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa338066730 0x7fa338068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.114+0000 7fa358f61700 1 --2- 172.21.15.81:0/312839249 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa338066730 0x7fa338068bf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa3541f66a0 tx=0x7fa344009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:51.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.122+0000 7fa349ffb700 1 -- 172.21.15.81:0/312839249 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa35414ff60 con 0x7fa35406efa0 2024-05-08T23:32:51.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.314+0000 7fa35a764700 1 -- 172.21.15.81:0/312839249 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa35414ff60 con 0x7fa35406efa0 2024-05-08T23:32:51.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.314+0000 7fa349ffb700 1 -- 172.21.15.81:0/312839249 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa35414ff60 con 0x7fa35406efa0 2024-05-08T23:32:51.317 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:32:51.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.314+0000 7fa35a764700 1 -- 172.21.15.81:0/312839249 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa338066730 msgr2=0x7fa338068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.314+0000 7fa35a764700 1 --2- 172.21.15.81:0/312839249 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa338066730 0x7fa338068bf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa3541f66a0 tx=0x7fa344009040 comp rx=0 tx=0).stop 2024-05-08T23:32:51.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.314+0000 7fa35a764700 1 -- 172.21.15.81:0/312839249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa35406efa0 msgr2=0x7fa354140750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.314+0000 7fa35a764700 1 --2- 172.21.15.81:0/312839249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa35406efa0 0x7fa354140750 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa35415a8c0 tx=0x7fa350004770 comp rx=0 tx=0).stop 2024-05-08T23:32:51.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.318+0000 7fa35a764700 1 -- 172.21.15.81:0/312839249 shutdown_connections 2024-05-08T23:32:51.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.318+0000 7fa35a764700 1 --2- 172.21.15.81:0/312839249 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa338066730 0x7fa338068bf0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.318+0000 7fa35a764700 1 --2- 172.21.15.81:0/312839249 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa35415a2f0 0x7fa3540b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.318+0000 7fa35a764700 1 --2- 172.21.15.81:0/312839249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa354159f10 0x7fa35414e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.318+0000 7fa35a764700 1 --2- 172.21.15.81:0/312839249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa35406efa0 0x7fa354140750 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.318+0000 7fa35a764700 1 -- 172.21.15.81:0/312839249 >> 172.21.15.81:0/312839249 conn(0x7fa3540cfda0 msgr2=0x7fa35415e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:51.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.318+0000 7fa35a764700 1 -- 172.21.15.81:0/312839249 shutdown_connections 2024-05-08T23:32:51.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:51.318+0000 7fa35a764700 1 -- 172.21.15.81:0/312839249 wait complete. 2024-05-08T23:32:51.331 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:32:51.331 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:32:51.331 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2024-05-08T23:32:51.331 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:32:51.331 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:51.331 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:32:51.331 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:32:51.331 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:32:51.332 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:32:51.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:51.332 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2024-05-08T23:32:51.340 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:32:51.340 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:32:51.340 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:32:51.340 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:51.340 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-05-08T23:32:51.341 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:51.341 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:32:51.341 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:51.341 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:32:51.341 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:51.341 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:32:51.341 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:51.341 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:32:51.341 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:51.341 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T23:32:51.341 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42444sec preferred_lft 42444sec 2024-05-08T23:32:51.341 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2024-05-08T23:32:51.341 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:51.341 INFO:teuthology.orchestra.run.smithi081.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T23:32:51.341 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 02:42:22:62:7a:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:51.342 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T23:32:51.342 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:51.342 INFO:teuthology.orchestra.run.smithi081.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:32:51.342 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:2b:10:6b:08:e3 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:51.342 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:32:51.342 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:51.342 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::48fc:c7ff:fe41:6d94/64 scope link 2024-05-08T23:32:51.342 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:51.342 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:32:51.342 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:2b:10:6b:08:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:32:51.342 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::702b:10ff:fe6b:8e3/64 scope link 2024-05-08T23:32:51.342 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:51.342 INFO:teuthology.orchestra.run.smithi081.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:32:51.342 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:9d:1b:4e:6b:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:32:51.342 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::709d:1bff:fe4e:6bd5/64 scope link 2024-05-08T23:32:51.343 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:51.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:51.343 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:32:51.357 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-05-08T23:32:51.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:51.358 DEBUG:teuthology.orchestra.run.smithi081:> route 2024-05-08T23:32:51.408 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2024-05-08T23:32:51.408 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:32:51.408 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T23:32:51.408 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T23:32:51.408 INFO:teuthology.orchestra.run.smithi081.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T23:32:51.408 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T23:32:51.408 INFO:teuthology.orchestra.run.smithi081.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T23:32:51.408 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:32:51.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:51.409 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-05-08T23:32:51.409 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-05-08T23:32:51.409 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T23:32:51.409 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T23:32:51.409 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T23:32:51.409 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-05-08T23:32:51.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:51.443 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-05-08T23:32:51.449 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T23:32:51.449 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:32:51.449 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:32:51.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:51.449 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2024-05-08T23:32:51.462 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T23:32:51.462 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:32:51.462 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:51.469 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:32:51.469 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:32:51.469 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:51.517 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:32:51.533 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:32:51.533 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:32:51.548 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:32:51.549 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:32:51.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:51.550 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:32:51.556 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T23:32:51.557 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T23:32:51.559 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:32:51.573 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:32:51.573 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:32:51.630 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:32:51.631 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:32:51.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:51.631 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:32:51.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2024-05-08T23:32:51.642+0000 7f7ac264b640 -1 init, newargv = 0x562a6ea84db0 newargc=15 2024-05-08T23:32:51.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[39193]: starting ceph client 2024-05-08T23:32:51.655 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[39193]: starting fuse 2024-05-08T23:32:52.737 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:32:52.756 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:32:52.756 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:32:52.815 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:32:52.816 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:32:52.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:52.816 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:32:52.823 INFO:teuthology.orchestra.run.smithi081.stdout:52 2024-05-08T23:32:52.824 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T23:32:52.824 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:32:52.824 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2024-05-08T23:32:52.824 DEBUG:teuthology.orchestra.run.smithi081:> import re 2024-05-08T23:32:52.824 DEBUG:teuthology.orchestra.run.smithi081:> import os 2024-05-08T23:32:52.824 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2024-05-08T23:32:52.824 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:32:52.824 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2024-05-08T23:32:52.824 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> return f 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T23:32:52.825 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:32:52.826 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2024-05-08T23:32:52.826 DEBUG:teuthology.orchestra.run.smithi081:> ' 2024-05-08T23:32:52.910 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.39193.asok 2024-05-08T23:32:53.093 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39193.asok 2024-05-08T23:32:53.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:53.094 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39193.asok status 2024-05-08T23:32:53.238 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:32:53.238 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2024-05-08T23:32:53.238 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T23:32:53.238 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:32:53.238 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2024-05-08T23:32:53.238 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2024-05-08T23:32:53.238 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "39193", 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 6093, 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 6093 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 112846689 2024-05-08T23:32:53.239 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:32:53.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:53.240 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:32:53.240 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:32:53.240 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:32:53.240 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 112846689 2024-05-08T23:32:53.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:32:53.240 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.6093 192.168.0.1:0/112846689", 2024-05-08T23:32:53.240 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/112846689", 2024-05-08T23:32:53.240 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2024-05-08T23:32:53.240 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 23, 2024-05-08T23:32:53.240 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 52, 2024-05-08T23:32:53.240 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2024-05-08T23:32:53.240 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2024-05-08T23:32:53.240 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2024-05-08T23:32:53.240 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:32:53.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:53.248 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2024-05-08T23:32:53.249 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:53.256 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2024-05-08T23:32:53.257 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:53.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:53.257 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:53.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:53.274 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file) 2024-05-08T23:32:53.335 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow r pool=cephfs_data' 2024-05-08T23:32:53.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f12a6137700 1 Processor -- start 2024-05-08T23:32:53.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f12a6137700 1 -- start start 2024-05-08T23:32:53.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f12a6137700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f12a0140080 0x7f12a00b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:53.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f12a6137700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f12a00b4310 0x7f12a015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:53.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f12a6137700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a015a6d0 con 0x7f12a013d050 2024-05-08T23:32:53.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f12a6137700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a0140500 con 0x7f12a00b4310 2024-05-08T23:32:53.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f12a6137700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a00b3940 con 0x7f12a0140080 2024-05-08T23:32:53.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f129ffff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f12a00b4310 0x7f12a015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:53.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f129f7fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f12a013d050 0x7f12a013f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:60638/0 (socket says 172.21.15.81:60638) 2024-05-08T23:32:53.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f129ffff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f12a00b4310 0x7f12a015dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56544/0 (socket says 172.21.15.81:56544) 2024-05-08T23:32:53.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f129f7fe700 1 -- 172.21.15.81:0/1381443717 learned_addr learned my addr 172.21.15.81:0/1381443717 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:53.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f129effd700 1 --2- 172.21.15.81:0/1381443717 >> v2:172.21.15.117:3300/0 conn(0x7f12a0140080 0x7f12a00b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:53.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f129ffff700 1 -- 172.21.15.81:0/1381443717 >> v2:172.21.15.117:3300/0 conn(0x7f12a0140080 msgr2=0x7f12a00b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:53.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f129ffff700 1 --2- 172.21.15.81:0/1381443717 >> v2:172.21.15.117:3300/0 conn(0x7f12a0140080 0x7f12a00b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:53.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f129ffff700 1 -- 172.21.15.81:0/1381443717 >> v1:172.21.15.117:6789/0 conn(0x7f12a013d050 legacy=0x7f12a013f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:53.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f129e7fc700 1 -- 172.21.15.81:0/1381443717 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593509609 0 0) 0x7f1290001110 con 0x7f12a013d050 2024-05-08T23:32:53.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f129ffff700 1 -- 172.21.15.81:0/1381443717 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a00b3010 con 0x7f12a00b4310 2024-05-08T23:32:53.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.466+0000 7f129ffff700 1 --2- 172.21.15.81:0/1381443717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00b4310 0x7f12a015dd10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f12a00721e0 tx=0x7f129800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4dac6021acba53 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:53.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.470+0000 7f129e7fc700 1 -- 172.21.15.81:0/1381443717 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129800e070 con 0x7f12a00b4310 2024-05-08T23:32:53.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.470+0000 7f129e7fc700 1 -- 172.21.15.81:0/1381443717 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129800bb20 con 0x7f12a00b4310 2024-05-08T23:32:53.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.470+0000 7f129e7fc700 1 -- 172.21.15.81:0/1381443717 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298012430 con 0x7f12a00b4310 2024-05-08T23:32:53.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.470+0000 7f12a6137700 1 -- 172.21.15.81:0/1381443717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00b4310 msgr2=0x7f12a015dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:53.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.470+0000 7f12a6137700 1 --2- 172.21.15.81:0/1381443717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00b4310 0x7f12a015dd10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f12a00721e0 tx=0x7f129800b330 comp rx=0 tx=0).stop 2024-05-08T23:32:53.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.470+0000 7f12a6137700 1 -- 172.21.15.81:0/1381443717 shutdown_connections 2024-05-08T23:32:53.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.470+0000 7f12a6137700 1 --2- 172.21.15.81:0/1381443717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00b4310 0x7f12a015dd10 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:53.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.470+0000 7f12a6137700 1 --2- 172.21.15.81:0/1381443717 >> v2:172.21.15.117:3300/0 conn(0x7f12a0140080 0x7f12a00b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:53.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.470+0000 7f12a6137700 1 -- 172.21.15.81:0/1381443717 >> 172.21.15.81:0/1381443717 conn(0x7f12a006b470 msgr2=0x7f12a006b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:53.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.470+0000 7f12a6137700 1 -- 172.21.15.81:0/1381443717 shutdown_connections 2024-05-08T23:32:53.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.470+0000 7f12a6137700 1 -- 172.21.15.81:0/1381443717 wait complete. 2024-05-08T23:32:53.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.470+0000 7f12a6137700 1 Processor -- start 2024-05-08T23:32:53.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.470+0000 7f12a6137700 1 -- start start 2024-05-08T23:32:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f12a6137700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12a00b4310 0x7f12a01557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f12a6137700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12a013d050 0x7f12a0158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f12a6137700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0140080 0x7f12a0156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f12a6137700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0155e80 con 0x7f12a0140080 2024-05-08T23:32:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f12a6137700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0156160 con 0x7f12a013d050 2024-05-08T23:32:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f12a6137700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0156440 con 0x7f12a00b4310 2024-05-08T23:32:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f129ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0140080 0x7f12a0156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f129ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0140080 0x7f12a0156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56554/0 (socket says 172.21.15.81:56554) 2024-05-08T23:32:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f129ffff700 1 -- 172.21.15.81:0/3328352889 learned_addr learned my addr 172.21.15.81:0/3328352889 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f129effd700 1 --2- 172.21.15.81:0/3328352889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12a013d050 0x7f12a0158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:53.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f129f7fe700 1 --2- 172.21.15.81:0/3328352889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12a00b4310 0x7f12a01557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:53.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f129ffff700 1 -- 172.21.15.81:0/3328352889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12a00b4310 msgr2=0x7f12a01557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:53.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f129ffff700 1 --2- 172.21.15.81:0/3328352889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12a00b4310 0x7f12a01557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:53.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f129ffff700 1 -- 172.21.15.81:0/3328352889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12a013d050 msgr2=0x7f12a0158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:53.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f129ffff700 1 --2- 172.21.15.81:0/3328352889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12a013d050 0x7f12a0158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:53.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f129ffff700 1 -- 172.21.15.81:0/3328352889 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1298009a70 con 0x7f12a0140080 2024-05-08T23:32:53.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f129ffff700 1 --2- 172.21.15.81:0/3328352889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0140080 0x7f12a0156f20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f12a0070c20 tx=0x7f1298004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:53.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f1297fff700 1 -- 172.21.15.81:0/3328352889 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298012430 con 0x7f12a0140080 2024-05-08T23:32:53.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f1297fff700 1 -- 172.21.15.81:0/3328352889 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1298020070 con 0x7f12a0140080 2024-05-08T23:32:53.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f1297fff700 1 -- 172.21.15.81:0/3328352889 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298004280 con 0x7f12a0140080 2024-05-08T23:32:53.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f12a6137700 1 -- 172.21.15.81:0/3328352889 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a0156720 con 0x7f12a0140080 2024-05-08T23:32:53.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.474+0000 7f12a6137700 1 -- 172.21.15.81:0/3328352889 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a01591c0 con 0x7f12a0140080 2024-05-08T23:32:53.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.478+0000 7f12a6137700 1 -- 172.21.15.81:0/3328352889 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a0156c40 con 0x7f12a0140080 2024-05-08T23:32:53.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.478+0000 7f1297fff700 1 -- 172.21.15.81:0/3328352889 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f12980043e0 con 0x7f12a0140080 2024-05-08T23:32:53.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.478+0000 7f1297fff700 1 --2- 172.21.15.81:0/3328352889 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1284066730 0x7f1284068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:53.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.478+0000 7f129f7fe700 1 --2- 172.21.15.81:0/3328352889 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1284066730 0x7f1284068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:53.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.478+0000 7f1297fff700 1 -- 172.21.15.81:0/3328352889 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1298022070 con 0x7f12a0140080 2024-05-08T23:32:53.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.478+0000 7f129f7fe700 1 --2- 172.21.15.81:0/3328352889 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1284066730 0x7f1284068bf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1290000f60 tx=0x7f1290007c40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:53.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.486+0000 7f1297fff700 1 -- 172.21.15.81:0/3328352889 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f12a0156c40 con 0x7f12a0140080 2024-05-08T23:32:53.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.714+0000 7f12a6137700 1 -- 172.21.15.81:0/3328352889 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]} v 0) v1 -- 0x7f12a0156c40 con 0x7f12a0140080 2024-05-08T23:32:53.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.722+0000 7f1297fff700 1 -- 172.21.15.81:0/3328352889 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]}]=0 updated caps for client.0 v15) v1 ==== 183+0+0 (secure 0 0 0) 0x7f12a0156c40 con 0x7f12a0140080 2024-05-08T23:32:53.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.722+0000 7f12a6137700 1 -- 172.21.15.81:0/3328352889 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1284066730 msgr2=0x7f1284068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:53.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.722+0000 7f12a6137700 1 --2- 172.21.15.81:0/3328352889 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1284066730 0x7f1284068bf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1290000f60 tx=0x7f1290007c40 comp rx=0 tx=0).stop 2024-05-08T23:32:53.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.722+0000 7f12a6137700 1 -- 172.21.15.81:0/3328352889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0140080 msgr2=0x7f12a0156f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:53.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.722+0000 7f12a6137700 1 --2- 172.21.15.81:0/3328352889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0140080 0x7f12a0156f20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f12a0070c20 tx=0x7f1298004af0 comp rx=0 tx=0).stop 2024-05-08T23:32:53.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.722+0000 7f12a6137700 1 -- 172.21.15.81:0/3328352889 shutdown_connections 2024-05-08T23:32:53.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.722+0000 7f12a6137700 1 --2- 172.21.15.81:0/3328352889 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1284066730 0x7f1284068bf0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:53.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.722+0000 7f12a6137700 1 --2- 172.21.15.81:0/3328352889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0140080 0x7f12a0156f20 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:53.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.722+0000 7f12a6137700 1 --2- 172.21.15.81:0/3328352889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12a013d050 0x7f12a0158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:53.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.722+0000 7f12a6137700 1 --2- 172.21.15.81:0/3328352889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12a00b4310 0x7f12a01557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:53.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.722+0000 7f12a6137700 1 -- 172.21.15.81:0/3328352889 >> 172.21.15.81:0/3328352889 conn(0x7f12a006b470 msgr2=0x7f12a015d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:53.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.726+0000 7f12a6137700 1 -- 172.21.15.81:0/3328352889 shutdown_connections 2024-05-08T23:32:53.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.726+0000 7f12a6137700 1 -- 172.21.15.81:0/3328352889 wait complete. 2024-05-08T23:32:53.728 INFO:teuthology.orchestra.run.smithi081.stderr:updated caps for client.0 2024-05-08T23:32:53.739 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:32:53.740 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:53.751 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febda035700 1 Processor -- start 2024-05-08T23:32:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febda035700 1 -- start start 2024-05-08T23:32:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febda035700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7febd4140130 0x7febd40b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febda035700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415e430 0x7febd40b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febda035700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd40b4800 con 0x7febd4142880 2024-05-08T23:32:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febda035700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd4140510 con 0x7febd4140130 2024-05-08T23:32:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febda035700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd40b24a0 con 0x7febd415e430 2024-05-08T23:32:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febd9033700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7febd4140130 0x7febd40b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febd8832700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415e430 0x7febd40b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febd9033700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7febd4140130 0x7febd40b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56560/0 (socket says 172.21.15.81:56560) 2024-05-08T23:32:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febd8832700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415e430 0x7febd40b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:59634/0 (socket says 172.21.15.81:59634) 2024-05-08T23:32:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febd9033700 1 -- 172.21.15.81:0/2574478541 learned_addr learned my addr 172.21.15.81:0/2574478541 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febd9834700 1 --1- 172.21.15.81:0/2574478541 >> v1:172.21.15.117:6789/0 conn(0x7febd4142880 0x7febd4160d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:60642/0 (socket says 172.21.15.81:60642) 2024-05-08T23:32:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febd8832700 1 -- 172.21.15.81:0/2574478541 >> v2:172.21.15.81:3300/0 conn(0x7febd4140130 msgr2=0x7febd40b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febd8832700 1 --2- 172.21.15.81:0/2574478541 >> v2:172.21.15.81:3300/0 conn(0x7febd4140130 0x7febd40b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febd8832700 1 -- 172.21.15.81:0/2574478541 >> v1:172.21.15.117:6789/0 conn(0x7febd4142880 legacy=0x7febd4160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febd8832700 1 -- 172.21.15.81:0/2574478541 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd40b3ed0 con 0x7febd415e430 2024-05-08T23:32:53.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febd9033700 1 --2- 172.21.15.81:0/2574478541 >> v2:172.21.15.81:3300/0 conn(0x7febd4140130 0x7febd40b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:53.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.906+0000 7febd8832700 1 --2- 172.21.15.81:0/2574478541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415e430 0x7febd40b1f60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febc4009850 tx=0x7febc4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=787765faebb4af5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:53.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.910+0000 7febcbfff700 1 -- 172.21.15.81:0/2574478541 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4011830 con 0x7febd415e430 2024-05-08T23:32:53.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.910+0000 7febcbfff700 1 -- 172.21.15.81:0/2574478541 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc4011990 con 0x7febd415e430 2024-05-08T23:32:53.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.910+0000 7febcbfff700 1 -- 172.21.15.81:0/2574478541 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4011d40 con 0x7febd415e430 2024-05-08T23:32:53.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.910+0000 7febda035700 1 -- 172.21.15.81:0/2574478541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415e430 msgr2=0x7febd40b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:53.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.910+0000 7febda035700 1 --2- 172.21.15.81:0/2574478541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415e430 0x7febd40b1f60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febc4009850 tx=0x7febc4009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:53.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.910+0000 7febda035700 1 -- 172.21.15.81:0/2574478541 shutdown_connections 2024-05-08T23:32:53.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.910+0000 7febda035700 1 --2- 172.21.15.81:0/2574478541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415e430 0x7febd40b1f60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:53.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.910+0000 7febda035700 1 --2- 172.21.15.81:0/2574478541 >> v2:172.21.15.81:3300/0 conn(0x7febd4140130 0x7febd40b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:53.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.910+0000 7febda035700 1 -- 172.21.15.81:0/2574478541 >> 172.21.15.81:0/2574478541 conn(0x7febd406c330 msgr2=0x7febd406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:53.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.910+0000 7febda035700 1 -- 172.21.15.81:0/2574478541 shutdown_connections 2024-05-08T23:32:53.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febda035700 1 -- 172.21.15.81:0/2574478541 wait complete. 2024-05-08T23:32:53.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febda035700 1 Processor -- start 2024-05-08T23:32:53.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febda035700 1 -- start start 2024-05-08T23:32:53.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febda035700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febd4140130 0x7febd41548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:53.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febda035700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd4142880 0x7febd4070a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:53.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febda035700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415e430 0x7febd413d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:53.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febda035700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd4071d10 con 0x7febd4140130 2024-05-08T23:32:53.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febda035700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd4141f30 con 0x7febd4142880 2024-05-08T23:32:53.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febda035700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd4154e80 con 0x7febd415e430 2024-05-08T23:32:53.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febd9033700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febd4140130 0x7febd41548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:53.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febd9033700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febd4140130 0x7febd41548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56568/0 (socket says 172.21.15.81:56568) 2024-05-08T23:32:53.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febd9033700 1 -- 172.21.15.81:0/4177485811 learned_addr learned my addr 172.21.15.81:0/4177485811 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:53.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febd8832700 1 --2- 172.21.15.81:0/4177485811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd4142880 0x7febd4070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:53.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febd9834700 1 --2- 172.21.15.81:0/4177485811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415e430 0x7febd413d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:53.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febd9033700 1 -- 172.21.15.81:0/4177485811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415e430 msgr2=0x7febd413d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:53.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febd9033700 1 --2- 172.21.15.81:0/4177485811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415e430 0x7febd413d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:53.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febd9033700 1 -- 172.21.15.81:0/4177485811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd4142880 msgr2=0x7febd4070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:53.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febd9033700 1 --2- 172.21.15.81:0/4177485811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd4142880 0x7febd4070a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:53.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febd9033700 1 -- 172.21.15.81:0/4177485811 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc009b30 con 0x7febd4140130 2024-05-08T23:32:53.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.914+0000 7febd8832700 1 --2- 172.21.15.81:0/4177485811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd4142880 0x7febd4070a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:53.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.918+0000 7febd9033700 1 --2- 172.21.15.81:0/4177485811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febd4140130 0x7febd41548a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7febcc00efc0 tx=0x7febcc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:53.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.918+0000 7febc9ffb700 1 -- 172.21.15.81:0/4177485811 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc00cc70 con 0x7febd4140130 2024-05-08T23:32:53.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.918+0000 7febc9ffb700 1 -- 172.21.15.81:0/4177485811 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc00f040 con 0x7febd4140130 2024-05-08T23:32:53.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.918+0000 7febda035700 1 -- 172.21.15.81:0/4177485811 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc40094b0 con 0x7febd4140130 2024-05-08T23:32:53.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.918+0000 7febc9ffb700 1 -- 172.21.15.81:0/4177485811 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc014480 con 0x7febd4140130 2024-05-08T23:32:53.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.918+0000 7febda035700 1 -- 172.21.15.81:0/4177485811 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd41424d0 con 0x7febd4140130 2024-05-08T23:32:53.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.918+0000 7febda035700 1 -- 172.21.15.81:0/4177485811 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd40684b0 con 0x7febd4140130 2024-05-08T23:32:53.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.918+0000 7febc9ffb700 1 -- 172.21.15.81:0/4177485811 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7febcc0145e0 con 0x7febd4140130 2024-05-08T23:32:53.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.918+0000 7febc9ffb700 1 --2- 172.21.15.81:0/4177485811 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febb8066730 0x7febb8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:53.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.918+0000 7febc9ffb700 1 -- 172.21.15.81:0/4177485811 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febcc084830 con 0x7febd4140130 2024-05-08T23:32:53.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.918+0000 7febd8832700 1 --2- 172.21.15.81:0/4177485811 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febb8066730 0x7febb8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:53.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.922+0000 7febd8832700 1 --2- 172.21.15.81:0/4177485811 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febb8066730 0x7febb8068bf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7febc40090b0 tx=0x7febc4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:53.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:53.926+0000 7febc9ffb700 1 -- 172.21.15.81:0/4177485811 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7febcc051ba0 con 0x7febd4140130 2024-05-08T23:32:54.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.094+0000 7febda035700 1 -- 172.21.15.81:0/4177485811 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febd406dfe0 con 0x7febd4140130 2024-05-08T23:32:54.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.098+0000 7febc9ffb700 1 -- 172.21.15.81:0/4177485811 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16101 (secure 0 0 0) 0x7febcc0551c0 con 0x7febd4140130 2024-05-08T23:32:54.102 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:54.102 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":52,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:32:41.855853+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:39.161796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:39.853963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:54.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.102+0000 7febda035700 1 -- 172.21.15.81:0/4177485811 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febb8066730 msgr2=0x7febb8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.102+0000 7febda035700 1 --2- 172.21.15.81:0/4177485811 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febb8066730 0x7febb8068bf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7febc40090b0 tx=0x7febc4008030 comp rx=0 tx=0).stop 2024-05-08T23:32:54.104 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.102+0000 7febda035700 1 -- 172.21.15.81:0/4177485811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febd4140130 msgr2=0x7febd41548a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.102+0000 7febda035700 1 --2- 172.21.15.81:0/4177485811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febd4140130 0x7febd41548a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7febcc00efc0 tx=0x7febcc00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.102+0000 7febda035700 1 -- 172.21.15.81:0/4177485811 shutdown_connections 2024-05-08T23:32:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.102+0000 7febda035700 1 --2- 172.21.15.81:0/4177485811 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febb8066730 0x7febb8068bf0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.102+0000 7febda035700 1 --2- 172.21.15.81:0/4177485811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febd415e430 0x7febd413d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.102+0000 7febda035700 1 --2- 172.21.15.81:0/4177485811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febd4142880 0x7febd4070a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.102+0000 7febda035700 1 --2- 172.21.15.81:0/4177485811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febd4140130 0x7febd41548a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.102+0000 7febda035700 1 -- 172.21.15.81:0/4177485811 >> 172.21.15.81:0/4177485811 conn(0x7febd406c330 msgr2=0x7febd406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.102+0000 7febda035700 1 -- 172.21.15.81:0/4177485811 shutdown_connections 2024-05-08T23:32:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.102+0000 7febda035700 1 -- 172.21.15.81:0/4177485811 wait complete. 2024-05-08T23:32:54.117 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:32:39.161796+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:32:39.853963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:32:54.117 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:54.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.234+0000 7efe24db1700 1 Processor -- start 2024-05-08T23:32:54.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.234+0000 7efe24db1700 1 -- start start 2024-05-08T23:32:54.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.234+0000 7efe24db1700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efe20071dd0 0x7efe2006a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.234+0000 7efe24db1700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efe2015c0f0 0x7efe2006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.234+0000 7efe24db1700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe200cf720 con 0x7efe20158760 2024-05-08T23:32:54.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.234+0000 7efe24db1700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe200cfc50 con 0x7efe2015c0f0 2024-05-08T23:32:54.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.234+0000 7efe24db1700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe2015b400 con 0x7efe20071dd0 2024-05-08T23:32:54.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.234+0000 7efe1ffff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efe2015c0f0 0x7efe2006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.240 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.234+0000 7efe1effd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efe20071dd0 0x7efe2006a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.240 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe1f7fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7efe20158760 0x7efe200cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:60652/0 (socket says 172.21.15.81:60652) 2024-05-08T23:32:54.240 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe1f7fe700 1 -- 172.21.15.81:0/3200054313 learned_addr learned my addr 172.21.15.81:0/3200054313 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:54.240 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe1ffff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efe2015c0f0 0x7efe2006a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56576/0 (socket says 172.21.15.81:56576) 2024-05-08T23:32:54.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe1ffff700 1 -- 172.21.15.81:0/3200054313 >> v2:172.21.15.117:3300/0 conn(0x7efe20071dd0 msgr2=0x7efe2006a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:54.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe1ffff700 1 --2- 172.21.15.81:0/3200054313 >> v2:172.21.15.117:3300/0 conn(0x7efe20071dd0 0x7efe2006a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe1ffff700 1 -- 172.21.15.81:0/3200054313 >> v1:172.21.15.117:6789/0 conn(0x7efe20158760 legacy=0x7efe200cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe1ffff700 1 -- 172.21.15.81:0/3200054313 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe200cf970 con 0x7efe2015c0f0 2024-05-08T23:32:54.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe1ffff700 1 --2- 172.21.15.81:0/3200054313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe2015c0f0 0x7efe2006a8a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7efe200af200 tx=0x7efe1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa254ba8449b372 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:54.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe1e7fc700 1 -- 172.21.15.81:0/3200054313 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1800e070 con 0x7efe2015c0f0 2024-05-08T23:32:54.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe1e7fc700 1 -- 172.21.15.81:0/3200054313 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1800bb20 con 0x7efe2015c0f0 2024-05-08T23:32:54.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe1e7fc700 1 -- 172.21.15.81:0/3200054313 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18012430 con 0x7efe2015c0f0 2024-05-08T23:32:54.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe24db1700 1 -- 172.21.15.81:0/3200054313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe2015c0f0 msgr2=0x7efe2006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe24db1700 1 --2- 172.21.15.81:0/3200054313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe2015c0f0 0x7efe2006a8a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7efe200af200 tx=0x7efe1800b330 comp rx=0 tx=0).stop 2024-05-08T23:32:54.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe24db1700 1 -- 172.21.15.81:0/3200054313 shutdown_connections 2024-05-08T23:32:54.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe24db1700 1 --2- 172.21.15.81:0/3200054313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe2015c0f0 0x7efe2006a8a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe24db1700 1 --2- 172.21.15.81:0/3200054313 >> v2:172.21.15.117:3300/0 conn(0x7efe20071dd0 0x7efe2006a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe24db1700 1 -- 172.21.15.81:0/3200054313 >> 172.21.15.81:0/3200054313 conn(0x7efe2006b470 msgr2=0x7efe2006b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:54.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe24db1700 1 -- 172.21.15.81:0/3200054313 shutdown_connections 2024-05-08T23:32:54.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.238+0000 7efe24db1700 1 -- 172.21.15.81:0/3200054313 wait complete. 2024-05-08T23:32:54.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe24db1700 1 Processor -- start 2024-05-08T23:32:54.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe24db1700 1 -- start start 2024-05-08T23:32:54.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe24db1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe20071dd0 0x7efe2014a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe24db1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe20158760 0x7efe2015d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe24db1700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe2015c0f0 0x7efe2009a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe24db1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2015d520 con 0x7efe20071dd0 2024-05-08T23:32:54.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe24db1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2009cbe0 con 0x7efe20158760 2024-05-08T23:32:54.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe24db1700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2009c2f0 con 0x7efe2015c0f0 2024-05-08T23:32:54.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe1f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe20071dd0 0x7efe2014a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe1f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe20071dd0 0x7efe2014a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56584/0 (socket says 172.21.15.81:56584) 2024-05-08T23:32:54.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe1f7fe700 1 -- 172.21.15.81:0/1941995590 learned_addr learned my addr 172.21.15.81:0/1941995590 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:54.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe1f7fe700 1 -- 172.21.15.81:0/1941995590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe2015c0f0 msgr2=0x7efe2009a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe1ffff700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe2015c0f0 0x7efe2009a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe1effd700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe20158760 0x7efe2015d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe1f7fe700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe2015c0f0 0x7efe2009a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe1f7fe700 1 -- 172.21.15.81:0/1941995590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe20158760 msgr2=0x7efe2015d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe1f7fe700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe20158760 0x7efe2015d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe1f7fe700 1 -- 172.21.15.81:0/1941995590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe18009a70 con 0x7efe20071dd0 2024-05-08T23:32:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe1effd700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe20158760 0x7efe2015d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe1ffff700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe2015c0f0 0x7efe2009a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe1f7fe700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe20071dd0 0x7efe2014a210 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7efe0800c530 tx=0x7efe0800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.242+0000 7efe17fff700 1 -- 172.21.15.81:0/1941995590 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08005d90 con 0x7efe20071dd0 2024-05-08T23:32:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.246+0000 7efe17fff700 1 -- 172.21.15.81:0/1941995590 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe080070d0 con 0x7efe20071dd0 2024-05-08T23:32:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.246+0000 7efe24db1700 1 -- 172.21.15.81:0/1941995590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2015d240 con 0x7efe20071dd0 2024-05-08T23:32:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.246+0000 7efe17fff700 1 -- 172.21.15.81:0/1941995590 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08007460 con 0x7efe20071dd0 2024-05-08T23:32:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.246+0000 7efe24db1700 1 -- 172.21.15.81:0/1941995590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2009c870 con 0x7efe20071dd0 2024-05-08T23:32:54.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.246+0000 7efe24db1700 1 -- 172.21.15.81:0/1941995590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe200672a0 con 0x7efe20071dd0 2024-05-08T23:32:54.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.246+0000 7efe17fff700 1 -- 172.21.15.81:0/1941995590 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efe080075c0 con 0x7efe20071dd0 2024-05-08T23:32:54.251 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.246+0000 7efe17fff700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efe00066730 0x7efe00068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.251 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.246+0000 7efe17fff700 1 -- 172.21.15.81:0/1941995590 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe08083670 con 0x7efe20071dd0 2024-05-08T23:32:54.251 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.246+0000 7efe1effd700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efe00066730 0x7efe00068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.251 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.250+0000 7efe1effd700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efe00066730 0x7efe00068bf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efe200b0fb0 tx=0x7efe10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:54.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.254+0000 7efe17fff700 1 -- 172.21.15.81:0/1941995590 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efe080509e0 con 0x7efe20071dd0 2024-05-08T23:32:54.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.426+0000 7efe24db1700 1 -- 172.21.15.81:0/1941995590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe200b3790 con 0x7efe20071dd0 2024-05-08T23:32:54.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.426+0000 7efe17fff700 1 -- 172.21.15.81:0/1941995590 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe08054000 con 0x7efe20071dd0 2024-05-08T23:32:54.431 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:32:54.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.430+0000 7efe24db1700 1 -- 172.21.15.81:0/1941995590 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efe00066730 msgr2=0x7efe00068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.430+0000 7efe24db1700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efe00066730 0x7efe00068bf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efe200b0fb0 tx=0x7efe10008040 comp rx=0 tx=0).stop 2024-05-08T23:32:54.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.430+0000 7efe24db1700 1 -- 172.21.15.81:0/1941995590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe20071dd0 msgr2=0x7efe2014a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.430+0000 7efe24db1700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe20071dd0 0x7efe2014a210 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7efe0800c530 tx=0x7efe0800c8f0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.430+0000 7efe24db1700 1 -- 172.21.15.81:0/1941995590 shutdown_connections 2024-05-08T23:32:54.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.430+0000 7efe24db1700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efe00066730 0x7efe00068bf0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.430+0000 7efe24db1700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe2015c0f0 0x7efe2009a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.430+0000 7efe24db1700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe20158760 0x7efe2015d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.430+0000 7efe24db1700 1 --2- 172.21.15.81:0/1941995590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe20071dd0 0x7efe2014a210 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.430+0000 7efe24db1700 1 -- 172.21.15.81:0/1941995590 >> 172.21.15.81:0/1941995590 conn(0x7efe2006b470 msgr2=0x7efe2015a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:54.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.430+0000 7efe24db1700 1 -- 172.21.15.81:0/1941995590 shutdown_connections 2024-05-08T23:32:54.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.430+0000 7efe24db1700 1 -- 172.21.15.81:0/1941995590 wait complete. 2024-05-08T23:32:54.445 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:32:54.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f63495ae700 1 Processor -- start 2024-05-08T23:32:54.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f63495ae700 1 -- start start 2024-05-08T23:32:54.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f63495ae700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63440cb3b0 0x7f63441478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f63495ae700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6344144d40 0x7f63441451b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f63495ae700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63441656a0 con 0x7f6344165240 2024-05-08T23:32:54.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f63495ae700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6344161c50 con 0x7f63440cb3b0 2024-05-08T23:32:54.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f63495ae700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6344166210 con 0x7f6344144d40 2024-05-08T23:32:54.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f63427fc700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f6344165240 0x7f6344162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49558/0 (socket says 172.21.15.81:49558) 2024-05-08T23:32:54.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f63427fc700 1 -- 172.21.15.81:0/1685124682 learned_addr learned my addr 172.21.15.81:0/1685124682 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:54.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f6342ffd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63440cb3b0 0x7f63441478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f63437fe700 1 --2- 172.21.15.81:0/1685124682 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6344144d40 0x7f63441451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f6341ffb700 1 -- 172.21.15.81:0/1685124682 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532745590 0 0) 0x7f63441656a0 con 0x7f6344165240 2024-05-08T23:32:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f6341ffb700 1 -- 172.21.15.81:0/1685124682 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f632c0035d0 con 0x7f6344165240 2024-05-08T23:32:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f6342ffd700 1 -- 172.21.15.81:0/1685124682 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6344144d40 msgr2=0x7f63441451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f6342ffd700 1 --2- 172.21.15.81:0/1685124682 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6344144d40 0x7f63441451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f6342ffd700 1 -- 172.21.15.81:0/1685124682 >> v1:172.21.15.81:6789/0 conn(0x7f6344165240 legacy=0x7f6344162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f6341ffb700 1 -- 172.21.15.81:0/1685124682 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1149244363 0 0) 0x7f632c0035d0 con 0x7f6344165240 2024-05-08T23:32:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.558+0000 7f6342ffd700 1 -- 172.21.15.81:0/1685124682 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6344165970 con 0x7f63440cb3b0 2024-05-08T23:32:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.562+0000 7f6342ffd700 1 --2- 172.21.15.81:0/1685124682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63440cb3b0 0x7f63441478c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6338009850 tx=0x7f6338009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa43d3f1ed13dc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.562+0000 7f6341ffb700 1 -- 172.21.15.81:0/1685124682 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334001670 con 0x7f63440cb3b0 2024-05-08T23:32:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.562+0000 7f6341ffb700 1 -- 172.21.15.81:0/1685124682 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6338004030 con 0x7f63440cb3b0 2024-05-08T23:32:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.562+0000 7f6341ffb700 1 -- 172.21.15.81:0/1685124682 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63380043e0 con 0x7f63440cb3b0 2024-05-08T23:32:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.562+0000 7f63495ae700 1 -- 172.21.15.81:0/1685124682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63440cb3b0 msgr2=0x7f63441478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.562+0000 7f63495ae700 1 --2- 172.21.15.81:0/1685124682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63440cb3b0 0x7f63441478c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6338009850 tx=0x7f6338009b60 comp rx=0 tx=0).stop 2024-05-08T23:32:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.562+0000 7f63495ae700 1 -- 172.21.15.81:0/1685124682 shutdown_connections 2024-05-08T23:32:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.562+0000 7f63495ae700 1 --2- 172.21.15.81:0/1685124682 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6344144d40 0x7f63441451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.562+0000 7f63495ae700 1 --2- 172.21.15.81:0/1685124682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63440cb3b0 0x7f63441478c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.562+0000 7f63495ae700 1 -- 172.21.15.81:0/1685124682 >> 172.21.15.81:0/1685124682 conn(0x7f634406a360 msgr2=0x7f634406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:54.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.562+0000 7f63495ae700 1 -- 172.21.15.81:0/1685124682 shutdown_connections 2024-05-08T23:32:54.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.562+0000 7f63495ae700 1 -- 172.21.15.81:0/1685124682 wait complete. 2024-05-08T23:32:54.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.562+0000 7f63495ae700 1 Processor -- start 2024-05-08T23:32:54.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.562+0000 7f63495ae700 1 -- start start 2024-05-08T23:32:54.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f63495ae700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63440cb3b0 0x7f6344137ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f63495ae700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6344144d40 0x7f634414a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f63495ae700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6344165240 0x7f6344158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f63495ae700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634415c6a0 con 0x7f63440cb3b0 2024-05-08T23:32:54.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f63495ae700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634415be50 con 0x7f6344165240 2024-05-08T23:32:54.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f63495ae700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6344164940 con 0x7f6344144d40 2024-05-08T23:32:54.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f6342ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63440cb3b0 0x7f6344137ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f6342ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63440cb3b0 0x7f6344137ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56590/0 (socket says 172.21.15.81:56590) 2024-05-08T23:32:54.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f6342ffd700 1 -- 172.21.15.81:0/2082531654 learned_addr learned my addr 172.21.15.81:0/2082531654 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:54.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f63427fc700 1 --2- 172.21.15.81:0/2082531654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6344144d40 0x7f634414a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f63437fe700 1 --2- 172.21.15.81:0/2082531654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6344165240 0x7f6344158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f6342ffd700 1 -- 172.21.15.81:0/2082531654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6344144d40 msgr2=0x7f634414a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f6342ffd700 1 --2- 172.21.15.81:0/2082531654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6344144d40 0x7f634414a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f6342ffd700 1 -- 172.21.15.81:0/2082531654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6344165240 msgr2=0x7f6344158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f6342ffd700 1 --2- 172.21.15.81:0/2082531654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6344165240 0x7f6344158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f6342ffd700 1 -- 172.21.15.81:0/2082531654 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63380094b0 con 0x7f63440cb3b0 2024-05-08T23:32:54.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f6342ffd700 1 --2- 172.21.15.81:0/2082531654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63440cb3b0 0x7f6344137ac0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f63380090b0 tx=0x7f6338004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:54.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f632b7fe700 1 -- 172.21.15.81:0/2082531654 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6338003d10 con 0x7f63440cb3b0 2024-05-08T23:32:54.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f632b7fe700 1 -- 172.21.15.81:0/2082531654 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633801f070 con 0x7f63440cb3b0 2024-05-08T23:32:54.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f632b7fe700 1 -- 172.21.15.81:0/2082531654 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6338011870 con 0x7f63440cb3b0 2024-05-08T23:32:54.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f63495ae700 1 -- 172.21.15.81:0/2082531654 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634415b8c0 con 0x7f63440cb3b0 2024-05-08T23:32:54.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.566+0000 7f63495ae700 1 -- 172.21.15.81:0/2082531654 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6344149a30 con 0x7f63440cb3b0 2024-05-08T23:32:54.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.570+0000 7f63495ae700 1 -- 172.21.15.81:0/2082531654 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63440684b0 con 0x7f63440cb3b0 2024-05-08T23:32:54.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.570+0000 7f632b7fe700 1 -- 172.21.15.81:0/2082531654 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f63380119d0 con 0x7f63440cb3b0 2024-05-08T23:32:54.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.570+0000 7f632b7fe700 1 --2- 172.21.15.81:0/2082531654 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6330066730 0x7f6330068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.570+0000 7f632b7fe700 1 -- 172.21.15.81:0/2082531654 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6338021070 con 0x7f63440cb3b0 2024-05-08T23:32:54.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.570+0000 7f63427fc700 1 --2- 172.21.15.81:0/2082531654 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6330066730 0x7f6330068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.570+0000 7f63427fc700 1 --2- 172.21.15.81:0/2082531654 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6330066730 0x7f6330068bf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6344138000 tx=0x7f6334001e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:54.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.574+0000 7f632b7fe700 1 -- 172.21.15.81:0/2082531654 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6338052cc0 con 0x7f63440cb3b0 2024-05-08T23:32:54.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.750+0000 7f63495ae700 1 -- 172.21.15.81:0/2082531654 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f634406d0c0 con 0x7f63440cb3b0 2024-05-08T23:32:54.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.750+0000 7f632b7fe700 1 -- 172.21.15.81:0/2082531654 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7f633802b020 con 0x7f63440cb3b0 2024-05-08T23:32:54.752 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:32:54.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.750+0000 7f63495ae700 1 -- 172.21.15.81:0/2082531654 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6330066730 msgr2=0x7f6330068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.750+0000 7f63495ae700 1 --2- 172.21.15.81:0/2082531654 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6330066730 0x7f6330068bf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6344138000 tx=0x7f6334001e00 comp rx=0 tx=0).stop 2024-05-08T23:32:54.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.750+0000 7f63495ae700 1 -- 172.21.15.81:0/2082531654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63440cb3b0 msgr2=0x7f6344137ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.750+0000 7f63495ae700 1 --2- 172.21.15.81:0/2082531654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63440cb3b0 0x7f6344137ac0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f63380090b0 tx=0x7f6338004770 comp rx=0 tx=0).stop 2024-05-08T23:32:54.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.750+0000 7f63495ae700 1 -- 172.21.15.81:0/2082531654 shutdown_connections 2024-05-08T23:32:54.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.750+0000 7f63495ae700 1 --2- 172.21.15.81:0/2082531654 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6330066730 0x7f6330068bf0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.750+0000 7f63495ae700 1 --2- 172.21.15.81:0/2082531654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6344165240 0x7f6344158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.750+0000 7f63495ae700 1 --2- 172.21.15.81:0/2082531654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6344144d40 0x7f634414a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.750+0000 7f63495ae700 1 --2- 172.21.15.81:0/2082531654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63440cb3b0 0x7f6344137ac0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.750+0000 7f63495ae700 1 -- 172.21.15.81:0/2082531654 >> 172.21.15.81:0/2082531654 conn(0x7f634406a360 msgr2=0x7f6344163810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:54.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.750+0000 7f63495ae700 1 -- 172.21.15.81:0/2082531654 shutdown_connections 2024-05-08T23:32:54.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.750+0000 7f63495ae700 1 -- 172.21.15.81:0/2082531654 wait complete. 2024-05-08T23:32:54.766 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:32:54.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec1ef37700 1 Processor -- start 2024-05-08T23:32:54.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec1ef37700 1 -- start start 2024-05-08T23:32:54.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec1ef37700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fec180b3210 0x7fec180b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec1ef37700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec180b3ba0 con 0x7fec1815c400 2024-05-08T23:32:54.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec1ef37700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec180b4660 con 0x7fec180d07e0 2024-05-08T23:32:54.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec1ef37700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1815f5b0 con 0x7fec180b3210 2024-05-08T23:32:54.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec17fff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fec1815c400 0x7fec180cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49562/0 (socket says 172.21.15.81:49562) 2024-05-08T23:32:54.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec1d4d4700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fec180b3210 0x7fec180b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec17fff700 1 -- 172.21.15.81:0/3918957624 learned_addr learned my addr 172.21.15.81:0/3918957624 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:54.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec1d4d4700 1 -- 172.21.15.81:0/3918957624 >> v1:172.21.15.117:6789/0 conn(0x7fec180d07e0 legacy=0x7fec1815ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec1d4d4700 1 -- 172.21.15.81:0/3918957624 >> v1:172.21.15.81:6789/0 conn(0x7fec1815c400 legacy=0x7fec180cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec177fe700 1 -- 172.21.15.81:0/3918957624 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915901982 0 0) 0x7fec180b4660 con 0x7fec180d07e0 2024-05-08T23:32:54.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec1d4d4700 1 -- 172.21.15.81:0/3918957624 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1815f890 con 0x7fec180b3210 2024-05-08T23:32:54.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec1d4d4700 1 --2- 172.21.15.81:0/3918957624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec180b3210 0x7fec180b3660 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fec10009d90 tx=0x7fec1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693ee9521d961be7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:54.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec177fe700 1 -- 172.21.15.81:0/3918957624 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1000e070 con 0x7fec180b3210 2024-05-08T23:32:54.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec177fe700 1 -- 172.21.15.81:0/3918957624 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1000bad0 con 0x7fec180b3210 2024-05-08T23:32:54.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.886+0000 7fec177fe700 1 -- 172.21.15.81:0/3918957624 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1000be80 con 0x7fec180b3210 2024-05-08T23:32:54.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.890+0000 7fec1ef37700 1 -- 172.21.15.81:0/3918957624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec180b3210 msgr2=0x7fec180b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.890+0000 7fec1ef37700 1 --2- 172.21.15.81:0/3918957624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec180b3210 0x7fec180b3660 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fec10009d90 tx=0x7fec1000b330 comp rx=0 tx=0).stop 2024-05-08T23:32:54.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.890+0000 7fec1ef37700 1 -- 172.21.15.81:0/3918957624 shutdown_connections 2024-05-08T23:32:54.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.890+0000 7fec1ef37700 1 --2- 172.21.15.81:0/3918957624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec180b3210 0x7fec180b3660 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.890+0000 7fec1ef37700 1 -- 172.21.15.81:0/3918957624 >> 172.21.15.81:0/3918957624 conn(0x7fec18073ab0 msgr2=0x7fec18073ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:54.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.890+0000 7fec1ef37700 1 -- 172.21.15.81:0/3918957624 shutdown_connections 2024-05-08T23:32:54.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.890+0000 7fec1ef37700 1 -- 172.21.15.81:0/3918957624 wait complete. 2024-05-08T23:32:54.894 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.890+0000 7fec1ef37700 1 Processor -- start 2024-05-08T23:32:54.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.890+0000 7fec1ef37700 1 -- start start 2024-05-08T23:32:54.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec1ef37700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec180b3210 0x7fec1813f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec1ef37700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec180d07e0 0x7fec18145e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec1ef37700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec1815c400 0x7fec180c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec1ef37700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1813ff20 con 0x7fec180d07e0 2024-05-08T23:32:54.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec1ef37700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec18159d50 con 0x7fec180b3210 2024-05-08T23:32:54.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec1ef37700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1815a030 con 0x7fec1815c400 2024-05-08T23:32:54.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec17fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec180d07e0 0x7fec18145e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec17fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec180d07e0 0x7fec18145e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56604/0 (socket says 172.21.15.81:56604) 2024-05-08T23:32:54.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec17fff700 1 -- 172.21.15.81:0/132106176 learned_addr learned my addr 172.21.15.81:0/132106176 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:54.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec1ccd3700 1 --2- 172.21.15.81:0/132106176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec180b3210 0x7fec1813f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec1d4d4700 1 --2- 172.21.15.81:0/132106176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec1815c400 0x7fec180c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec17fff700 1 -- 172.21.15.81:0/132106176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec1815c400 msgr2=0x7fec180c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec17fff700 1 --2- 172.21.15.81:0/132106176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec1815c400 0x7fec180c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec17fff700 1 -- 172.21.15.81:0/132106176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec180b3210 msgr2=0x7fec1813f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:54.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec17fff700 1 --2- 172.21.15.81:0/132106176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec180b3210 0x7fec1813f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:54.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec17fff700 1 -- 172.21.15.81:0/132106176 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec10009a70 con 0x7fec180d07e0 2024-05-08T23:32:54.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec1ccd3700 1 --2- 172.21.15.81:0/132106176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec180b3210 0x7fec1813f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:54.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec17fff700 1 --2- 172.21.15.81:0/132106176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec180d07e0 0x7fec18145e70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fec0800c840 tx=0x7fec0800cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:54.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec157fa700 1 -- 172.21.15.81:0/132106176 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec08004ca0 con 0x7fec180d07e0 2024-05-08T23:32:54.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec157fa700 1 -- 172.21.15.81:0/132106176 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec08004e00 con 0x7fec180d07e0 2024-05-08T23:32:54.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec1ef37700 1 -- 172.21.15.81:0/132106176 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1815a310 con 0x7fec180d07e0 2024-05-08T23:32:54.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec157fa700 1 -- 172.21.15.81:0/132106176 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec08007310 con 0x7fec180d07e0 2024-05-08T23:32:54.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.894+0000 7fec1ef37700 1 -- 172.21.15.81:0/132106176 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1815a8d0 con 0x7fec180d07e0 2024-05-08T23:32:54.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.898+0000 7fec1ef37700 1 -- 172.21.15.81:0/132106176 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec180684b0 con 0x7fec180d07e0 2024-05-08T23:32:54.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.898+0000 7fec157fa700 1 -- 172.21.15.81:0/132106176 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fec08007470 con 0x7fec180d07e0 2024-05-08T23:32:54.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.898+0000 7fec157fa700 1 --2- 172.21.15.81:0/132106176 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febf8066760 0x7febf8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:54.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.898+0000 7fec157fa700 1 -- 172.21.15.81:0/132106176 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec08083600 con 0x7fec180d07e0 2024-05-08T23:32:54.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.898+0000 7fec1ccd3700 1 --2- 172.21.15.81:0/132106176 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febf8066760 0x7febf8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:54.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.898+0000 7fec1ccd3700 1 --2- 172.21.15.81:0/132106176 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febf8066760 0x7febf8068c20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fec00000f60 tx=0x7fec00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:54.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:54.902+0000 7fec157fa700 1 -- 172.21.15.81:0/132106176 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fec080508d0 con 0x7fec180d07e0 2024-05-08T23:32:55.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.070+0000 7fec1ef37700 1 -- 172.21.15.81:0/132106176 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fec180b3ba0 con 0x7fec180d07e0 2024-05-08T23:32:55.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.074+0000 7fec157fa700 1 -- 172.21.15.81:0/132106176 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7fec08018020 con 0x7fec180d07e0 2024-05-08T23:32:55.076 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:32:55.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.074+0000 7fec1ef37700 1 -- 172.21.15.81:0/132106176 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febf8066760 msgr2=0x7febf8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.074+0000 7fec1ef37700 1 --2- 172.21.15.81:0/132106176 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febf8066760 0x7febf8068c20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fec00000f60 tx=0x7fec00009040 comp rx=0 tx=0).stop 2024-05-08T23:32:55.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.074+0000 7fec1ef37700 1 -- 172.21.15.81:0/132106176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec180d07e0 msgr2=0x7fec18145e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.074+0000 7fec1ef37700 1 --2- 172.21.15.81:0/132106176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec180d07e0 0x7fec18145e70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fec0800c840 tx=0x7fec0800cb50 comp rx=0 tx=0).stop 2024-05-08T23:32:55.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.074+0000 7fec1ef37700 1 -- 172.21.15.81:0/132106176 shutdown_connections 2024-05-08T23:32:55.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.074+0000 7fec1ef37700 1 --2- 172.21.15.81:0/132106176 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7febf8066760 0x7febf8068c20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.074+0000 7fec1ef37700 1 --2- 172.21.15.81:0/132106176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec1815c400 0x7fec180c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.074+0000 7fec1ef37700 1 --2- 172.21.15.81:0/132106176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec180d07e0 0x7fec18145e70 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.074+0000 7fec1ef37700 1 --2- 172.21.15.81:0/132106176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec180b3210 0x7fec1813f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.074+0000 7fec1ef37700 1 -- 172.21.15.81:0/132106176 >> 172.21.15.81:0/132106176 conn(0x7fec18073ab0 msgr2=0x7fec18073e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:55.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.074+0000 7fec1ef37700 1 -- 172.21.15.81:0/132106176 shutdown_connections 2024-05-08T23:32:55.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.074+0000 7fec1ef37700 1 -- 172.21.15.81:0/132106176 wait complete. 2024-05-08T23:32:55.091 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:32:55.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.198+0000 7f129e1a2700 1 Processor -- start 2024-05-08T23:32:55.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.198+0000 7f129e1a2700 1 -- start start 2024-05-08T23:32:55.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129e1a2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f129815afa0 0x7f129815b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129e1a2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12980ce990 0x7f12980b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129e1a2700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129815eda0 con 0x7f129815b8c0 2024-05-08T23:32:55.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129e1a2700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129815ea70 con 0x7f129815afa0 2024-05-08T23:32:55.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129e1a2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129815bd40 con 0x7f12980ce990 2024-05-08T23:32:55.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129d1a0700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f129815afa0 0x7f129815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129d1a0700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f129815afa0 0x7f129815b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56608/0 (socket says 172.21.15.81:56608) 2024-05-08T23:32:55.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129d1a0700 1 -- 172.21.15.81:0/3870595806 learned_addr learned my addr 172.21.15.81:0/3870595806 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:55.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129d9a1700 1 --2- 172.21.15.81:0/3870595806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12980ce990 0x7f12980b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129d1a0700 1 -- 172.21.15.81:0/3870595806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12980ce990 msgr2=0x7f12980b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:55.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129d1a0700 1 --2- 172.21.15.81:0/3870595806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12980ce990 0x7f12980b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129d1a0700 1 -- 172.21.15.81:0/3870595806 >> v1:172.21.15.117:6789/0 conn(0x7f129815b8c0 legacy=0x7f129815f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129d1a0700 1 -- 172.21.15.81:0/3870595806 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129815c020 con 0x7f129815afa0 2024-05-08T23:32:55.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129d1a0700 1 --2- 172.21.15.81:0/3870595806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f129815afa0 0x7f129815b380 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1288009910 tx=0x7f1288009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b0f63ae92a76a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:55.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f128ffff700 1 -- 172.21.15.81:0/3870595806 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288011830 con 0x7f129815afa0 2024-05-08T23:32:55.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f128ffff700 1 -- 172.21.15.81:0/3870595806 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1288011990 con 0x7f129815afa0 2024-05-08T23:32:55.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f128ffff700 1 -- 172.21.15.81:0/3870595806 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288011d40 con 0x7f129815afa0 2024-05-08T23:32:55.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129e1a2700 1 -- 172.21.15.81:0/3870595806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f129815afa0 msgr2=0x7f129815b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129e1a2700 1 --2- 172.21.15.81:0/3870595806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f129815afa0 0x7f129815b380 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1288009910 tx=0x7f1288009c20 comp rx=0 tx=0).stop 2024-05-08T23:32:55.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129e1a2700 1 -- 172.21.15.81:0/3870595806 shutdown_connections 2024-05-08T23:32:55.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129e1a2700 1 --2- 172.21.15.81:0/3870595806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12980ce990 0x7f12980b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129e1a2700 1 --2- 172.21.15.81:0/3870595806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f129815afa0 0x7f129815b380 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129e1a2700 1 -- 172.21.15.81:0/3870595806 >> 172.21.15.81:0/3870595806 conn(0x7f129806c330 msgr2=0x7f129806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:55.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129e1a2700 1 -- 172.21.15.81:0/3870595806 shutdown_connections 2024-05-08T23:32:55.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.202+0000 7f129e1a2700 1 -- 172.21.15.81:0/3870595806 wait complete. 2024-05-08T23:32:55.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129e1a2700 1 Processor -- start 2024-05-08T23:32:55.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129e1a2700 1 -- start start 2024-05-08T23:32:55.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129e1a2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12980ce990 0x7f129815a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129e1a2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f129815afa0 0x7f129814b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129e1a2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f129815b8c0 0x7f1298154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129e1a2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129814bd30 con 0x7f12980ce990 2024-05-08T23:32:55.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129e1a2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1298159690 con 0x7f129815b8c0 2024-05-08T23:32:55.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129e1a2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1298159970 con 0x7f129815afa0 2024-05-08T23:32:55.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129d1a0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12980ce990 0x7f129815a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129d1a0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12980ce990 0x7f129815a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56622/0 (socket says 172.21.15.81:56622) 2024-05-08T23:32:55.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129d9a1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f129815b8c0 0x7f1298154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129d1a0700 1 -- 172.21.15.81:0/1744955786 learned_addr learned my addr 172.21.15.81:0/1744955786 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:55.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129c99f700 1 --2- 172.21.15.81:0/1744955786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f129815afa0 0x7f129814b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129d1a0700 1 -- 172.21.15.81:0/1744955786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f129815afa0 msgr2=0x7f129814b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129d1a0700 1 --2- 172.21.15.81:0/1744955786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f129815afa0 0x7f129814b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129d1a0700 1 -- 172.21.15.81:0/1744955786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f129815b8c0 msgr2=0x7f1298154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129d1a0700 1 --2- 172.21.15.81:0/1744955786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f129815b8c0 0x7f1298154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129d1a0700 1 -- 172.21.15.81:0/1744955786 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1288009570 con 0x7f12980ce990 2024-05-08T23:32:55.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129d1a0700 1 --2- 172.21.15.81:0/1744955786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12980ce990 0x7f129815a450 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1288009c60 tx=0x7f1288004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:55.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f128dffb700 1 -- 172.21.15.81:0/1744955786 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288004300 con 0x7f12980ce990 2024-05-08T23:32:55.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f128dffb700 1 -- 172.21.15.81:0/1744955786 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128801f070 con 0x7f12980ce990 2024-05-08T23:32:55.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129e1a2700 1 -- 172.21.15.81:0/1744955786 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1298159c50 con 0x7f12980ce990 2024-05-08T23:32:55.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f128dffb700 1 -- 172.21.15.81:0/1744955786 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12880036c0 con 0x7f12980ce990 2024-05-08T23:32:55.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.206+0000 7f129e1a2700 1 -- 172.21.15.81:0/1744955786 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1298159360 con 0x7f12980ce990 2024-05-08T23:32:55.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.210+0000 7f129e1a2700 1 -- 172.21.15.81:0/1744955786 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12980684b0 con 0x7f12980ce990 2024-05-08T23:32:55.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.210+0000 7f128dffb700 1 -- 172.21.15.81:0/1744955786 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1288021030 con 0x7f12980ce990 2024-05-08T23:32:55.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.210+0000 7f128dffb700 1 --2- 172.21.15.81:0/1744955786 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1284066730 0x7f1284068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.210+0000 7f128dffb700 1 -- 172.21.15.81:0/1744955786 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1288085570 con 0x7f12980ce990 2024-05-08T23:32:55.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.210+0000 7f129c99f700 1 --2- 172.21.15.81:0/1744955786 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1284066730 0x7f1284068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.214+0000 7f129c99f700 1 --2- 172.21.15.81:0/1744955786 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1284066730 0x7f1284068bf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f129814c350 tx=0x7f1294009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:55.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.222+0000 7f128dffb700 1 -- 172.21.15.81:0/1744955786 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1288052970 con 0x7f12980ce990 2024-05-08T23:32:55.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.402+0000 7f129e1a2700 1 -- 172.21.15.81:0/1744955786 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f129806e220 con 0x7f12980ce990 2024-05-08T23:32:55.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.402+0000 7f128dffb700 1 -- 172.21.15.81:0/1744955786 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1288055f90 con 0x7f12980ce990 2024-05-08T23:32:55.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.406+0000 7f129e1a2700 1 -- 172.21.15.81:0/1744955786 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1284066730 msgr2=0x7f1284068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.411 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.406+0000 7f129e1a2700 1 --2- 172.21.15.81:0/1744955786 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1284066730 0x7f1284068bf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f129814c350 tx=0x7f1294009040 comp rx=0 tx=0).stop 2024-05-08T23:32:55.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.406+0000 7f129e1a2700 1 -- 172.21.15.81:0/1744955786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12980ce990 msgr2=0x7f129815a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.406+0000 7f129e1a2700 1 --2- 172.21.15.81:0/1744955786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12980ce990 0x7f129815a450 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1288009c60 tx=0x7f1288004770 comp rx=0 tx=0).stop 2024-05-08T23:32:55.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.406+0000 7f129e1a2700 1 -- 172.21.15.81:0/1744955786 shutdown_connections 2024-05-08T23:32:55.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.406+0000 7f129e1a2700 1 --2- 172.21.15.81:0/1744955786 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1284066730 0x7f1284068bf0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.406+0000 7f129e1a2700 1 --2- 172.21.15.81:0/1744955786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f129815b8c0 0x7f1298154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.406+0000 7f129e1a2700 1 --2- 172.21.15.81:0/1744955786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f129815afa0 0x7f129814b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.406+0000 7f129e1a2700 1 --2- 172.21.15.81:0/1744955786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12980ce990 0x7f129815a450 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.406+0000 7f129e1a2700 1 -- 172.21.15.81:0/1744955786 >> 172.21.15.81:0/1744955786 conn(0x7f129806c330 msgr2=0x7f129806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:55.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.410+0000 7f129e1a2700 1 -- 172.21.15.81:0/1744955786 shutdown_connections 2024-05-08T23:32:55.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.410+0000 7f129e1a2700 1 -- 172.21.15.81:0/1744955786 wait complete. 2024-05-08T23:32:55.413 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:32:55.426 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:32:55.427 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:55.433 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:55.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.538+0000 7ffa93fff700 1 Processor -- start 2024-05-08T23:32:55.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.538+0000 7ffa93fff700 1 -- start start 2024-05-08T23:32:55.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa93fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0e11d0 0x7ffa8c0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa93fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffa8c038820 0x7ffa8c039f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa93fff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa8c0078e0 con 0x7ffa8c051010 2024-05-08T23:32:55.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa93fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8c006f10 con 0x7ffa8c038820 2024-05-08T23:32:55.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa93fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c0e15b0 con 0x7ffa8c0e11d0 2024-05-08T23:32:55.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa927fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffa8c038820 0x7ffa8c039f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa927fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffa8c038820 0x7ffa8c039f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:40352/0 (socket says 172.21.15.81:40352) 2024-05-08T23:32:55.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa927fc700 1 -- 172.21.15.81:0/2966203360 learned_addr learned my addr 172.21.15.81:0/2966203360 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:55.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa937fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ffa8c051010 0x7ffa8c04bc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49570/0 (socket says 172.21.15.81:49570) 2024-05-08T23:32:55.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa92ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0e11d0 0x7ffa8c0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa927fc700 1 -- 172.21.15.81:0/2966203360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0e11d0 msgr2=0x7ffa8c0073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:55.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa927fc700 1 --2- 172.21.15.81:0/2966203360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0e11d0 0x7ffa8c0073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa927fc700 1 -- 172.21.15.81:0/2966203360 >> v1:172.21.15.81:6789/0 conn(0x7ffa8c051010 legacy=0x7ffa8c04bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa927fc700 1 -- 172.21.15.81:0/2966203360 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c006300 con 0x7ffa8c038820 2024-05-08T23:32:55.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa927fc700 1 --2- 172.21.15.81:0/2966203360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c038820 0x7ffa8c039f70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009910 tx=0x7ffa7c009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55a706eebad5ac8b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:55.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa91ffb700 1 -- 172.21.15.81:0/2966203360 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c011830 con 0x7ffa8c038820 2024-05-08T23:32:55.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa91ffb700 1 -- 172.21.15.81:0/2966203360 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7c011990 con 0x7ffa8c038820 2024-05-08T23:32:55.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa91ffb700 1 -- 172.21.15.81:0/2966203360 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c011d40 con 0x7ffa8c038820 2024-05-08T23:32:55.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa93fff700 1 -- 172.21.15.81:0/2966203360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c038820 msgr2=0x7ffa8c039f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa93fff700 1 --2- 172.21.15.81:0/2966203360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c038820 0x7ffa8c039f70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009910 tx=0x7ffa7c009c20 comp rx=0 tx=0).stop 2024-05-08T23:32:55.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa93fff700 1 -- 172.21.15.81:0/2966203360 shutdown_connections 2024-05-08T23:32:55.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa93fff700 1 --2- 172.21.15.81:0/2966203360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c038820 0x7ffa8c039f70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa93fff700 1 --2- 172.21.15.81:0/2966203360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c0e11d0 0x7ffa8c0073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.547 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa93fff700 1 -- 172.21.15.81:0/2966203360 >> 172.21.15.81:0/2966203360 conn(0x7ffa8c054b70 msgr2=0x7ffa8c054f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:55.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa93fff700 1 -- 172.21.15.81:0/2966203360 shutdown_connections 2024-05-08T23:32:55.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.542+0000 7ffa93fff700 1 -- 172.21.15.81:0/2966203360 wait complete. 2024-05-08T23:32:55.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.546+0000 7ffa93fff700 1 Processor -- start 2024-05-08T23:32:55.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.546+0000 7ffa93fff700 1 -- start start 2024-05-08T23:32:55.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa93fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c038820 0x7ffa8c049480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa93fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c051010 0x7ffa8c0d5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa93fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c0e11d0 0x7ffa8c04af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa93fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c0db960 con 0x7ffa8c038820 2024-05-08T23:32:55.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa93fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c0dab90 con 0x7ffa8c0e11d0 2024-05-08T23:32:55.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa93fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c0dae70 con 0x7ffa8c051010 2024-05-08T23:32:55.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa92ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c038820 0x7ffa8c049480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa92ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c038820 0x7ffa8c049480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56632/0 (socket says 172.21.15.81:56632) 2024-05-08T23:32:55.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa92ffd700 1 -- 172.21.15.81:0/2040787266 learned_addr learned my addr 172.21.15.81:0/2040787266 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:55.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa937fe700 1 --2- 172.21.15.81:0/2040787266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c0e11d0 0x7ffa8c04af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa927fc700 1 --2- 172.21.15.81:0/2040787266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c051010 0x7ffa8c0d5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa92ffd700 1 -- 172.21.15.81:0/2040787266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c051010 msgr2=0x7ffa8c0d5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa92ffd700 1 --2- 172.21.15.81:0/2040787266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c051010 0x7ffa8c0d5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa92ffd700 1 -- 172.21.15.81:0/2040787266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c0e11d0 msgr2=0x7ffa8c04af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa92ffd700 1 --2- 172.21.15.81:0/2040787266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c0e11d0 0x7ffa8c04af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa92ffd700 1 -- 172.21.15.81:0/2040787266 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c009570 con 0x7ffa8c038820 2024-05-08T23:32:55.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa92ffd700 1 --2- 172.21.15.81:0/2040787266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c038820 0x7ffa8c049480 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400e9d0 tx=0x7ffa8400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:55.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa8b7fe700 1 -- 172.21.15.81:0/2040787266 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84009720 con 0x7ffa8c038820 2024-05-08T23:32:55.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa8b7fe700 1 -- 172.21.15.81:0/2040787266 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8400ee60 con 0x7ffa8c038820 2024-05-08T23:32:55.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa93fff700 1 -- 172.21.15.81:0/2040787266 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa8c0db150 con 0x7ffa8c038820 2024-05-08T23:32:55.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa8b7fe700 1 -- 172.21.15.81:0/2040787266 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400ca50 con 0x7ffa8c038820 2024-05-08T23:32:55.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa93fff700 1 -- 172.21.15.81:0/2040787266 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8c0499c0 con 0x7ffa8c038820 2024-05-08T23:32:55.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.550+0000 7ffa93fff700 1 -- 172.21.15.81:0/2040787266 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8c0dbad0 con 0x7ffa8c038820 2024-05-08T23:32:55.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.554+0000 7ffa8b7fe700 1 -- 172.21.15.81:0/2040787266 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffa8400cbb0 con 0x7ffa8c038820 2024-05-08T23:32:55.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.554+0000 7ffa8b7fe700 1 --2- 172.21.15.81:0/2040787266 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa74066760 0x7ffa74068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.554+0000 7ffa927fc700 1 --2- 172.21.15.81:0/2040787266 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa74066760 0x7ffa74068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.554+0000 7ffa8b7fe700 1 -- 172.21.15.81:0/2040787266 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa840841c0 con 0x7ffa8c038820 2024-05-08T23:32:55.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.554+0000 7ffa927fc700 1 --2- 172.21.15.81:0/2040787266 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa74066760 0x7ffa74068c20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c014040 tx=0x7ffa7c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:55.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.562+0000 7ffa8b7fe700 1 -- 172.21.15.81:0/2040787266 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffa840514b0 con 0x7ffa8c038820 2024-05-08T23:32:55.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.730+0000 7ffa93fff700 1 -- 172.21.15.81:0/2040787266 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa8c005440 con 0x7ffa8c038820 2024-05-08T23:32:55.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.730+0000 7ffa8b7fe700 1 -- 172.21.15.81:0/2040787266 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16101 (secure 0 0 0) 0x7ffa8400ce60 con 0x7ffa8c038820 2024-05-08T23:32:55.734 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:32:55.734 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":52,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:32:41.855853+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:39.161796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:39.853963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:55.736 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.734+0000 7ffa93fff700 1 -- 172.21.15.81:0/2040787266 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa74066760 msgr2=0x7ffa74068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.737 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.734+0000 7ffa93fff700 1 --2- 172.21.15.81:0/2040787266 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa74066760 0x7ffa74068c20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c014040 tx=0x7ffa7c008030 comp rx=0 tx=0).stop 2024-05-08T23:32:55.737 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.734+0000 7ffa93fff700 1 -- 172.21.15.81:0/2040787266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c038820 msgr2=0x7ffa8c049480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.737 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.734+0000 7ffa93fff700 1 --2- 172.21.15.81:0/2040787266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c038820 0x7ffa8c049480 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400e9d0 tx=0x7ffa8400ece0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.737 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.734+0000 7ffa93fff700 1 -- 172.21.15.81:0/2040787266 shutdown_connections 2024-05-08T23:32:55.737 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.734+0000 7ffa93fff700 1 --2- 172.21.15.81:0/2040787266 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ffa74066760 0x7ffa74068c20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.737 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.734+0000 7ffa93fff700 1 --2- 172.21.15.81:0/2040787266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c0e11d0 0x7ffa8c04af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.737 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.734+0000 7ffa93fff700 1 --2- 172.21.15.81:0/2040787266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa8c051010 0x7ffa8c0d5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.737 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.734+0000 7ffa93fff700 1 --2- 172.21.15.81:0/2040787266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffa8c038820 0x7ffa8c049480 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.737 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.734+0000 7ffa93fff700 1 -- 172.21.15.81:0/2040787266 >> 172.21.15.81:0/2040787266 conn(0x7ffa8c054b70 msgr2=0x7ffa8c0e1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:55.737 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.734+0000 7ffa93fff700 1 -- 172.21.15.81:0/2040787266 shutdown_connections 2024-05-08T23:32:55.737 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.734+0000 7ffa93fff700 1 -- 172.21.15.81:0/2040787266 wait complete. 2024-05-08T23:32:55.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:32:39.161796+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:32:39.853963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:32:55.749 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:55.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fea31700 1 Processor -- start 2024-05-08T23:32:55.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fea31700 1 -- start start 2024-05-08T23:32:55.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fea31700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb6f815afa0 0x7fb6f80ac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fea31700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80ce6c0 0x7fb6f806ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fea31700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f80acc50 con 0x7fb6f815eab0 2024-05-08T23:32:55.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fea31700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f815b380 con 0x7fb6f815afa0 2024-05-08T23:32:55.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fea31700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f80ae0d0 con 0x7fb6f80ce6c0 2024-05-08T23:32:55.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fd22e700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb6f815eab0 0x7fb6f80cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49572/0 (socket says 172.21.15.81:49572) 2024-05-08T23:32:55.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fd22e700 1 -- 172.21.15.81:0/3295208971 learned_addr learned my addr 172.21.15.81:0/3295208971 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:55.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fda2f700 1 --2- 172.21.15.81:0/3295208971 >> v2:172.21.15.81:3300/0 conn(0x7fb6f815afa0 0x7fb6f80ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fe230700 1 --2- 172.21.15.81:0/3295208971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80ce6c0 0x7fb6f806ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fca2d700 1 -- 172.21.15.81:0/3295208971 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612455581 0 0) 0x7fb6f80acc50 con 0x7fb6f815eab0 2024-05-08T23:32:55.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fca2d700 1 -- 172.21.15.81:0/3295208971 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6e40035d0 con 0x7fb6f815eab0 2024-05-08T23:32:55.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fda2f700 1 -- 172.21.15.81:0/3295208971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80ce6c0 msgr2=0x7fb6f806ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:55.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fda2f700 1 --2- 172.21.15.81:0/3295208971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80ce6c0 0x7fb6f806ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fda2f700 1 -- 172.21.15.81:0/3295208971 >> v1:172.21.15.81:6789/0 conn(0x7fb6f815eab0 legacy=0x7fb6f80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fda2f700 1 -- 172.21.15.81:0/3295208971 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f80acf30 con 0x7fb6f815afa0 2024-05-08T23:32:55.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.870+0000 7fb6fda2f700 1 --2- 172.21.15.81:0/3295208971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6f815afa0 0x7fb6f80ac420 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009880 tx=0x7fb6e8009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9b08e2d05e1f2e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:55.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.874+0000 7fb6fca2d700 1 -- 172.21.15.81:0/3295208971 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8010a90 con 0x7fb6f815afa0 2024-05-08T23:32:55.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.874+0000 7fb6fca2d700 1 -- 172.21.15.81:0/3295208971 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e8010bf0 con 0x7fb6f815afa0 2024-05-08T23:32:55.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.874+0000 7fb6fca2d700 1 -- 172.21.15.81:0/3295208971 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8018440 con 0x7fb6f815afa0 2024-05-08T23:32:55.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.874+0000 7fb6fea31700 1 -- 172.21.15.81:0/3295208971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6f815afa0 msgr2=0x7fb6f80ac420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.874+0000 7fb6fea31700 1 --2- 172.21.15.81:0/3295208971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6f815afa0 0x7fb6f80ac420 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009880 tx=0x7fb6e8009b90 comp rx=0 tx=0).stop 2024-05-08T23:32:55.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.874+0000 7fb6fea31700 1 -- 172.21.15.81:0/3295208971 shutdown_connections 2024-05-08T23:32:55.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.874+0000 7fb6fea31700 1 --2- 172.21.15.81:0/3295208971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80ce6c0 0x7fb6f806ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.874+0000 7fb6fea31700 1 --2- 172.21.15.81:0/3295208971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6f815afa0 0x7fb6f80ac420 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.874+0000 7fb6fea31700 1 -- 172.21.15.81:0/3295208971 >> 172.21.15.81:0/3295208971 conn(0x7fb6f80b0de0 msgr2=0x7fb6f80b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:55.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.874+0000 7fb6fea31700 1 -- 172.21.15.81:0/3295208971 shutdown_connections 2024-05-08T23:32:55.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.874+0000 7fb6fea31700 1 -- 172.21.15.81:0/3295208971 wait complete. 2024-05-08T23:32:55.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.874+0000 7fb6fea31700 1 Processor -- start 2024-05-08T23:32:55.878 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.874+0000 7fb6fea31700 1 -- start start 2024-05-08T23:32:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fea31700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6f80ce6c0 0x7fb6f814a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fea31700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6f815eab0 0x7fb6f81eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fea31700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f81ef080 0x7fb6f815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fea31700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f809b000 con 0x7fb6f815eab0 2024-05-08T23:32:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fea31700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f809acd0 con 0x7fb6f80ce6c0 2024-05-08T23:32:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fea31700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f81ef6a0 con 0x7fb6f81ef080 2024-05-08T23:32:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fd22e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6f815eab0 0x7fb6f81eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fd22e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6f815eab0 0x7fb6f81eeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56658/0 (socket says 172.21.15.81:56658) 2024-05-08T23:32:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fd22e700 1 -- 172.21.15.81:0/4221100163 learned_addr learned my addr 172.21.15.81:0/4221100163 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fda2f700 1 --2- 172.21.15.81:0/4221100163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6f80ce6c0 0x7fb6f814a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fe230700 1 --2- 172.21.15.81:0/4221100163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f81ef080 0x7fb6f815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fd22e700 1 -- 172.21.15.81:0/4221100163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f81ef080 msgr2=0x7fb6f815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fd22e700 1 --2- 172.21.15.81:0/4221100163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f81ef080 0x7fb6f815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fd22e700 1 -- 172.21.15.81:0/4221100163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6f80ce6c0 msgr2=0x7fb6f814a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:55.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fd22e700 1 --2- 172.21.15.81:0/4221100163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6f80ce6c0 0x7fb6f814a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:55.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fd22e700 1 -- 172.21.15.81:0/4221100163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e80094b0 con 0x7fb6f815eab0 2024-05-08T23:32:55.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fda2f700 1 --2- 172.21.15.81:0/4221100163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6f80ce6c0 0x7fb6f814a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:55.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fd22e700 1 --2- 172.21.15.81:0/4221100163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6f815eab0 0x7fb6f81eeb40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400c4d0 tx=0x7fb6f400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:55.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6ee7fc700 1 -- 172.21.15.81:0/4221100163 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f40011d0 con 0x7fb6f815eab0 2024-05-08T23:32:55.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6ee7fc700 1 -- 172.21.15.81:0/4221100163 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f4005d90 con 0x7fb6f815eab0 2024-05-08T23:32:55.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fea31700 1 -- 172.21.15.81:0/4221100163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f81ef980 con 0x7fb6f815eab0 2024-05-08T23:32:55.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6ee7fc700 1 -- 172.21.15.81:0/4221100163 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f40072d0 con 0x7fb6f815eab0 2024-05-08T23:32:55.882 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fea31700 1 -- 172.21.15.81:0/4221100163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f8159e30 con 0x7fb6f815eab0 2024-05-08T23:32:55.883 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.878+0000 7fb6fea31700 1 -- 172.21.15.81:0/4221100163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f81ef460 con 0x7fb6f815eab0 2024-05-08T23:32:55.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.882+0000 7fb6ee7fc700 1 -- 172.21.15.81:0/4221100163 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb6f4007430 con 0x7fb6f815eab0 2024-05-08T23:32:55.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.882+0000 7fb6ee7fc700 1 --2- 172.21.15.81:0/4221100163 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6dc066730 0x7fb6dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:55.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.882+0000 7fb6ee7fc700 1 -- 172.21.15.81:0/4221100163 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6f40836a0 con 0x7fb6f815eab0 2024-05-08T23:32:55.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.882+0000 7fb6fda2f700 1 --2- 172.21.15.81:0/4221100163 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6dc066730 0x7fb6dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:55.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.882+0000 7fb6fda2f700 1 --2- 172.21.15.81:0/4221100163 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6dc066730 0x7fb6dc068bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8072410 tx=0x7fb6e8014000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:55.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:55.890+0000 7fb6ee7fc700 1 -- 172.21.15.81:0/4221100163 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb6f81ef460 con 0x7fb6f815eab0 2024-05-08T23:32:56.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.058+0000 7fb6fea31700 1 -- 172.21.15.81:0/4221100163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6f81ef460 con 0x7fb6f815eab0 2024-05-08T23:32:56.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.062+0000 7fb6ee7fc700 1 -- 172.21.15.81:0/4221100163 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6f81ef460 con 0x7fb6f815eab0 2024-05-08T23:32:56.064 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:32:56.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.062+0000 7fb6fea31700 1 -- 172.21.15.81:0/4221100163 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6dc066730 msgr2=0x7fb6dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.062+0000 7fb6fea31700 1 --2- 172.21.15.81:0/4221100163 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6dc066730 0x7fb6dc068bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8072410 tx=0x7fb6e8014000 comp rx=0 tx=0).stop 2024-05-08T23:32:56.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.062+0000 7fb6fea31700 1 -- 172.21.15.81:0/4221100163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6f815eab0 msgr2=0x7fb6f81eeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.062+0000 7fb6fea31700 1 --2- 172.21.15.81:0/4221100163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6f815eab0 0x7fb6f81eeb40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400c4d0 tx=0x7fb6f400c890 comp rx=0 tx=0).stop 2024-05-08T23:32:56.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.062+0000 7fb6fea31700 1 -- 172.21.15.81:0/4221100163 shutdown_connections 2024-05-08T23:32:56.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.062+0000 7fb6fea31700 1 --2- 172.21.15.81:0/4221100163 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb6dc066730 0x7fb6dc068bf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.062+0000 7fb6fea31700 1 --2- 172.21.15.81:0/4221100163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f81ef080 0x7fb6f815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.062+0000 7fb6fea31700 1 --2- 172.21.15.81:0/4221100163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6f815eab0 0x7fb6f81eeb40 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.062+0000 7fb6fea31700 1 --2- 172.21.15.81:0/4221100163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6f80ce6c0 0x7fb6f814a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.062+0000 7fb6fea31700 1 -- 172.21.15.81:0/4221100163 >> 172.21.15.81:0/4221100163 conn(0x7fb6f80b0de0 msgr2=0x7fb6f80ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:56.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.062+0000 7fb6fea31700 1 -- 172.21.15.81:0/4221100163 shutdown_connections 2024-05-08T23:32:56.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.062+0000 7fb6fea31700 1 -- 172.21.15.81:0/4221100163 wait complete. 2024-05-08T23:32:56.078 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:32:56.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.194+0000 7efca6f06700 1 Processor -- start 2024-05-08T23:32:56.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca6f06700 1 -- start start 2024-05-08T23:32:56.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca6f06700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca00abda0 0x7efca015e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca6f06700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efca00adf30 0x7efca00b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca6f06700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca015a350 con 0x7efca0157b30 2024-05-08T23:32:56.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca6f06700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca015e980 con 0x7efca00adf30 2024-05-08T23:32:56.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca6f06700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca015b370 con 0x7efca00abda0 2024-05-08T23:32:56.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca6705700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efca00adf30 0x7efca00b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca5703700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca00abda0 0x7efca015e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca6705700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efca00adf30 0x7efca00b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:56518/0 (socket says 172.21.15.81:56518) 2024-05-08T23:32:56.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca5703700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca00abda0 0x7efca015e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:49470/0 (socket says 172.21.15.81:49470) 2024-05-08T23:32:56.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca6705700 1 -- 172.21.15.81:0/3710012960 learned_addr learned my addr 172.21.15.81:0/3710012960 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:56.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca5f04700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7efca0157b30 0x7efca00ad820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:59710/0 (socket says 172.21.15.81:59710) 2024-05-08T23:32:56.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca6705700 1 -- 172.21.15.81:0/3710012960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca00abda0 msgr2=0x7efca015e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:56.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca6705700 1 --2- 172.21.15.81:0/3710012960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca00abda0 0x7efca015e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca6705700 1 -- 172.21.15.81:0/3710012960 >> v1:172.21.15.117:6789/0 conn(0x7efca0157b30 legacy=0x7efca00ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca6705700 1 -- 172.21.15.81:0/3710012960 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca015a5b0 con 0x7efca00adf30 2024-05-08T23:32:56.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca5703700 1 --2- 172.21.15.81:0/3710012960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca00abda0 0x7efca015e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:56.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca6705700 1 --2- 172.21.15.81:0/3710012960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca00adf30 0x7efca00b2fe0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efc9800d150 tx=0x7efc9800d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fae45a1860206812 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:56.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca4f02700 1 -- 172.21.15.81:0/3710012960 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98010070 con 0x7efca00adf30 2024-05-08T23:32:56.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca4f02700 1 -- 172.21.15.81:0/3710012960 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9800dd10 con 0x7efca00adf30 2024-05-08T23:32:56.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.198+0000 7efca4f02700 1 -- 172.21.15.81:0/3710012960 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9800b310 con 0x7efca00adf30 2024-05-08T23:32:56.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.202+0000 7efca6f06700 1 -- 172.21.15.81:0/3710012960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca00adf30 msgr2=0x7efca00b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.202+0000 7efca6f06700 1 --2- 172.21.15.81:0/3710012960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca00adf30 0x7efca00b2fe0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efc9800d150 tx=0x7efc9800d460 comp rx=0 tx=0).stop 2024-05-08T23:32:56.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.202+0000 7efca6f06700 1 -- 172.21.15.81:0/3710012960 shutdown_connections 2024-05-08T23:32:56.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.202+0000 7efca6f06700 1 --2- 172.21.15.81:0/3710012960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca00adf30 0x7efca00b2fe0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.202+0000 7efca6f06700 1 --2- 172.21.15.81:0/3710012960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca00abda0 0x7efca015e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.202+0000 7efca6f06700 1 -- 172.21.15.81:0/3710012960 >> 172.21.15.81:0/3710012960 conn(0x7efca00b0de0 msgr2=0x7efca00b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:56.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.202+0000 7efca6f06700 1 -- 172.21.15.81:0/3710012960 shutdown_connections 2024-05-08T23:32:56.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.202+0000 7efca6f06700 1 -- 172.21.15.81:0/3710012960 wait complete. 2024-05-08T23:32:56.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.202+0000 7efca6f06700 1 Processor -- start 2024-05-08T23:32:56.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.202+0000 7efca6f06700 1 -- start start 2024-05-08T23:32:56.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca6f06700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efca00abda0 0x7efca01edf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca6f06700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca00adf30 0x7efca0152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca6f06700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca0157b30 0x7efca0152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca6f06700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca009a590 con 0x7efca00abda0 2024-05-08T23:32:56.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca6f06700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca01f2bb0 con 0x7efca00adf30 2024-05-08T23:32:56.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca6f06700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca0141dc0 con 0x7efca0157b30 2024-05-08T23:32:56.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca5f04700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efca00abda0 0x7efca01edf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca5f04700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efca00abda0 0x7efca01edf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57782/0 (socket says 172.21.15.81:57782) 2024-05-08T23:32:56.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca5f04700 1 -- 172.21.15.81:0/4125928238 learned_addr learned my addr 172.21.15.81:0/4125928238 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:56.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca6705700 1 --2- 172.21.15.81:0/4125928238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca0157b30 0x7efca0152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca5703700 1 --2- 172.21.15.81:0/4125928238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca00adf30 0x7efca0152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca5f04700 1 -- 172.21.15.81:0/4125928238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca0157b30 msgr2=0x7efca0152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca5f04700 1 --2- 172.21.15.81:0/4125928238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca0157b30 0x7efca0152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca5f04700 1 -- 172.21.15.81:0/4125928238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca00adf30 msgr2=0x7efca0152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca5f04700 1 --2- 172.21.15.81:0/4125928238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca00adf30 0x7efca0152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca5f04700 1 -- 172.21.15.81:0/4125928238 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9c0094b0 con 0x7efca00abda0 2024-05-08T23:32:56.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca5703700 1 --2- 172.21.15.81:0/4125928238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca00adf30 0x7efca0152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:56.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca5f04700 1 --2- 172.21.15.81:0/4125928238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efca00abda0 0x7efca01edf40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c700 tx=0x7efc9000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:56.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efc967fc700 1 -- 172.21.15.81:0/4125928238 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc90005cd0 con 0x7efca00abda0 2024-05-08T23:32:56.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efc967fc700 1 -- 172.21.15.81:0/4125928238 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc90005e30 con 0x7efca00abda0 2024-05-08T23:32:56.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca6f06700 1 -- 172.21.15.81:0/4125928238 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc9800bd40 con 0x7efca00abda0 2024-05-08T23:32:56.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efc967fc700 1 -- 172.21.15.81:0/4125928238 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc900072d0 con 0x7efca00abda0 2024-05-08T23:32:56.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca6f06700 1 -- 172.21.15.81:0/4125928238 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca0099ed0 con 0x7efca00abda0 2024-05-08T23:32:56.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.206+0000 7efca6f06700 1 -- 172.21.15.81:0/4125928238 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca00684b0 con 0x7efca00abda0 2024-05-08T23:32:56.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.210+0000 7efc967fc700 1 -- 172.21.15.81:0/4125928238 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efc90007430 con 0x7efca00abda0 2024-05-08T23:32:56.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.210+0000 7efc967fc700 1 --2- 172.21.15.81:0/4125928238 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efc84066730 0x7efc84068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.210+0000 7efc967fc700 1 -- 172.21.15.81:0/4125928238 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc90083550 con 0x7efca00abda0 2024-05-08T23:32:56.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.210+0000 7efca5703700 1 --2- 172.21.15.81:0/4125928238 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efc84066730 0x7efc84068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.210+0000 7efca5703700 1 --2- 172.21.15.81:0/4125928238 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efc84066730 0x7efc84068bf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efca01f1b40 tx=0x7efc9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:56.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.218+0000 7efc967fc700 1 -- 172.21.15.81:0/4125928238 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efc900508d0 con 0x7efca00abda0 2024-05-08T23:32:56.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.386+0000 7efca6f06700 1 -- 172.21.15.81:0/4125928238 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efca015e000 con 0x7efca00abda0 2024-05-08T23:32:56.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.386+0000 7efc967fc700 1 -- 172.21.15.81:0/4125928238 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7efc90053ef0 con 0x7efca00abda0 2024-05-08T23:32:56.388 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:32:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.386+0000 7efca6f06700 1 -- 172.21.15.81:0/4125928238 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efc84066730 msgr2=0x7efc84068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.386+0000 7efca6f06700 1 --2- 172.21.15.81:0/4125928238 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efc84066730 0x7efc84068bf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efca01f1b40 tx=0x7efc9c008040 comp rx=0 tx=0).stop 2024-05-08T23:32:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.386+0000 7efca6f06700 1 -- 172.21.15.81:0/4125928238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efca00abda0 msgr2=0x7efca01edf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.386+0000 7efca6f06700 1 --2- 172.21.15.81:0/4125928238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efca00abda0 0x7efca01edf40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c700 tx=0x7efc9000ca10 comp rx=0 tx=0).stop 2024-05-08T23:32:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.386+0000 7efca6f06700 1 -- 172.21.15.81:0/4125928238 shutdown_connections 2024-05-08T23:32:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.386+0000 7efca6f06700 1 --2- 172.21.15.81:0/4125928238 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efc84066730 0x7efc84068bf0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.386+0000 7efca6f06700 1 --2- 172.21.15.81:0/4125928238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efca0157b30 0x7efca0152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.386+0000 7efca6f06700 1 --2- 172.21.15.81:0/4125928238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efca00adf30 0x7efca0152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.386+0000 7efca6f06700 1 --2- 172.21.15.81:0/4125928238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efca00abda0 0x7efca01edf40 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.386+0000 7efca6f06700 1 -- 172.21.15.81:0/4125928238 >> 172.21.15.81:0/4125928238 conn(0x7efca00b0de0 msgr2=0x7efca00ad7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.386+0000 7efca6f06700 1 -- 172.21.15.81:0/4125928238 shutdown_connections 2024-05-08T23:32:56.392 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.386+0000 7efca6f06700 1 -- 172.21.15.81:0/4125928238 wait complete. 2024-05-08T23:32:56.403 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:32:56.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.510+0000 7f717fb09700 1 Processor -- start 2024-05-08T23:32:56.515 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.510+0000 7f717fb09700 1 -- start start 2024-05-08T23:32:56.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717fb09700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f71780d07e0 0x7f717815ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717fb09700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f717815c400 0x7f71780cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717fb09700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71780b4660 con 0x7f71780cfcd0 2024-05-08T23:32:56.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717fb09700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f717815f5b0 con 0x7f71780d07e0 2024-05-08T23:32:56.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717fb09700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f717815f890 con 0x7f717815c400 2024-05-08T23:32:56.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717e0a6700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f71780cfcd0 0x7f71780b3210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:59416/0 (socket says 172.21.15.81:59416) 2024-05-08T23:32:56.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717e0a6700 1 -- 172.21.15.81:0/76051387 learned_addr learned my addr 172.21.15.81:0/76051387 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:56.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717d8a5700 1 --2- 172.21.15.81:0/76051387 >> v2:172.21.15.81:3300/0 conn(0x7f71780d07e0 0x7f717815ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717d0a4700 1 --2- 172.21.15.81:0/76051387 >> v2:172.21.15.117:3300/0 conn(0x7f717815c400 0x7f71780cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717c8a3700 1 -- 172.21.15.81:0/76051387 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585301594 0 0) 0x7f71780b4660 con 0x7f71780cfcd0 2024-05-08T23:32:56.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717c8a3700 1 -- 172.21.15.81:0/76051387 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71640035d0 con 0x7f71780cfcd0 2024-05-08T23:32:56.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717d0a4700 1 -- 172.21.15.81:0/76051387 >> v2:172.21.15.81:3300/0 conn(0x7f71780d07e0 msgr2=0x7f717815ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:56.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717d0a4700 1 --2- 172.21.15.81:0/76051387 >> v2:172.21.15.81:3300/0 conn(0x7f71780d07e0 0x7f717815ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717d0a4700 1 -- 172.21.15.81:0/76051387 >> v1:172.21.15.81:6789/0 conn(0x7f71780cfcd0 legacy=0x7f71780b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717d0a4700 1 -- 172.21.15.81:0/76051387 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71780b48d0 con 0x7f717815c400 2024-05-08T23:32:56.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717d0a4700 1 --2- 172.21.15.81:0/76051387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717815c400 0x7f71780cf790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f71780d1fd0 tx=0x7f716c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffb4a36259d67a66 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:56.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717c8a3700 1 -- 172.21.15.81:0/76051387 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c00e070 con 0x7f717815c400 2024-05-08T23:32:56.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717c8a3700 1 -- 172.21.15.81:0/76051387 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c00bb20 con 0x7f717815c400 2024-05-08T23:32:56.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717c8a3700 1 -- 172.21.15.81:0/76051387 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c00be90 con 0x7f717815c400 2024-05-08T23:32:56.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717fb09700 1 -- 172.21.15.81:0/76051387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717815c400 msgr2=0x7f71780cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717fb09700 1 --2- 172.21.15.81:0/76051387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717815c400 0x7f71780cf790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f71780d1fd0 tx=0x7f716c00b330 comp rx=0 tx=0).stop 2024-05-08T23:32:56.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717fb09700 1 -- 172.21.15.81:0/76051387 shutdown_connections 2024-05-08T23:32:56.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717fb09700 1 --2- 172.21.15.81:0/76051387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717815c400 0x7f71780cf790 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717fb09700 1 --2- 172.21.15.81:0/76051387 >> v2:172.21.15.81:3300/0 conn(0x7f71780d07e0 0x7f717815ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717fb09700 1 -- 172.21.15.81:0/76051387 >> 172.21.15.81:0/76051387 conn(0x7f7178073ab0 msgr2=0x7f7178073ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:56.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717fb09700 1 -- 172.21.15.81:0/76051387 shutdown_connections 2024-05-08T23:32:56.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.514+0000 7f717fb09700 1 -- 172.21.15.81:0/76051387 wait complete. 2024-05-08T23:32:56.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.518+0000 7f717fb09700 1 Processor -- start 2024-05-08T23:32:56.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.518+0000 7f717fb09700 1 -- start start 2024-05-08T23:32:56.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.518+0000 7f717fb09700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71780cfcd0 0x7f7178142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.518+0000 7f717fb09700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71780d07e0 0x7f71781f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.518+0000 7f717fb09700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717815c400 0x7f717815a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.518+0000 7f717fb09700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7178155f30 con 0x7f71780cfcd0 2024-05-08T23:32:56.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.518+0000 7f717fb09700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7178142f50 con 0x7f71780d07e0 2024-05-08T23:32:56.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.518+0000 7f717fb09700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71781f15d0 con 0x7f717815c400 2024-05-08T23:32:56.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.518+0000 7f717d8a5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71780cfcd0 0x7f7178142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.518+0000 7f717d8a5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71780cfcd0 0x7f7178142a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57800/0 (socket says 172.21.15.81:57800) 2024-05-08T23:32:56.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.518+0000 7f717d8a5700 1 -- 172.21.15.81:0/1000417326 learned_addr learned my addr 172.21.15.81:0/1000417326 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:56.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.522+0000 7f717d0a4700 1 --2- 172.21.15.81:0/1000417326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71780d07e0 0x7f71781f0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.522+0000 7f717e0a6700 1 --2- 172.21.15.81:0/1000417326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717815c400 0x7f717815a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.522+0000 7f717d8a5700 1 -- 172.21.15.81:0/1000417326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717815c400 msgr2=0x7f717815a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.522+0000 7f717d8a5700 1 --2- 172.21.15.81:0/1000417326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717815c400 0x7f717815a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.522+0000 7f717d8a5700 1 -- 172.21.15.81:0/1000417326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71780d07e0 msgr2=0x7f71781f0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.522+0000 7f717d8a5700 1 --2- 172.21.15.81:0/1000417326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71780d07e0 0x7f71781f0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.522+0000 7f717d8a5700 1 -- 172.21.15.81:0/1000417326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c009a70 con 0x7f71780cfcd0 2024-05-08T23:32:56.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.522+0000 7f717d8a5700 1 --2- 172.21.15.81:0/1000417326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71780cfcd0 0x7f7178142a10 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f717000e9d0 tx=0x7f717000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:56.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.522+0000 7f716a7fc700 1 -- 172.21.15.81:0/1000417326 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170009720 con 0x7f71780cfcd0 2024-05-08T23:32:56.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.522+0000 7f716a7fc700 1 -- 172.21.15.81:0/1000417326 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717000ee70 con 0x7f71780cfcd0 2024-05-08T23:32:56.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.522+0000 7f717fb09700 1 -- 172.21.15.81:0/1000417326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71781f18b0 con 0x7f71780cfcd0 2024-05-08T23:32:56.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.522+0000 7f716a7fc700 1 -- 172.21.15.81:0/1000417326 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717000ca50 con 0x7f71780cfcd0 2024-05-08T23:32:56.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.522+0000 7f717fb09700 1 -- 172.21.15.81:0/1000417326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7178156140 con 0x7f71780cfcd0 2024-05-08T23:32:56.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.522+0000 7f717fb09700 1 -- 172.21.15.81:0/1000417326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71780684b0 con 0x7f71780cfcd0 2024-05-08T23:32:56.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.526+0000 7f716a7fc700 1 -- 172.21.15.81:0/1000417326 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f717000cbb0 con 0x7f71780cfcd0 2024-05-08T23:32:56.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.526+0000 7f716a7fc700 1 --2- 172.21.15.81:0/1000417326 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f715c066730 0x7f715c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.526+0000 7f716a7fc700 1 -- 172.21.15.81:0/1000417326 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7170084190 con 0x7f71780cfcd0 2024-05-08T23:32:56.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.526+0000 7f717d0a4700 1 --2- 172.21.15.81:0/1000417326 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f715c066730 0x7f715c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.526+0000 7f717d0a4700 1 --2- 172.21.15.81:0/1000417326 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f715c066730 0x7f715c068bf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f717815c850 tx=0x7f716c011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:56.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.530+0000 7f716a7fc700 1 -- 172.21.15.81:0/1000417326 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7170051500 con 0x7f71780cfcd0 2024-05-08T23:32:56.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.698+0000 7f717fb09700 1 -- 172.21.15.81:0/1000417326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7178070b00 con 0x7f71780cfcd0 2024-05-08T23:32:56.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.698+0000 7f716a7fc700 1 -- 172.21.15.81:0/1000417326 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7f717000ce60 con 0x7f71780cfcd0 2024-05-08T23:32:56.700 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:32:56.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.698+0000 7f717fb09700 1 -- 172.21.15.81:0/1000417326 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f715c066730 msgr2=0x7f715c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.698+0000 7f717fb09700 1 --2- 172.21.15.81:0/1000417326 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f715c066730 0x7f715c068bf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f717815c850 tx=0x7f716c011000 comp rx=0 tx=0).stop 2024-05-08T23:32:56.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.698+0000 7f717fb09700 1 -- 172.21.15.81:0/1000417326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71780cfcd0 msgr2=0x7f7178142a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.698+0000 7f717fb09700 1 --2- 172.21.15.81:0/1000417326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71780cfcd0 0x7f7178142a10 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f717000e9d0 tx=0x7f717000ece0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.698+0000 7f717fb09700 1 -- 172.21.15.81:0/1000417326 shutdown_connections 2024-05-08T23:32:56.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.698+0000 7f717fb09700 1 --2- 172.21.15.81:0/1000417326 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f715c066730 0x7f715c068bf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.698+0000 7f717fb09700 1 --2- 172.21.15.81:0/1000417326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717815c400 0x7f717815a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.698+0000 7f717fb09700 1 --2- 172.21.15.81:0/1000417326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71780d07e0 0x7f71781f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.698+0000 7f717fb09700 1 --2- 172.21.15.81:0/1000417326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71780cfcd0 0x7f7178142a10 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.698+0000 7f717fb09700 1 -- 172.21.15.81:0/1000417326 >> 172.21.15.81:0/1000417326 conn(0x7f7178073ab0 msgr2=0x7f717806e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:56.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.698+0000 7f717fb09700 1 -- 172.21.15.81:0/1000417326 shutdown_connections 2024-05-08T23:32:56.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.698+0000 7f717fb09700 1 -- 172.21.15.81:0/1000417326 wait complete. 2024-05-08T23:32:56.715 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:32:56.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6fabfff700 1 Processor -- start 2024-05-08T23:32:56.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6fabfff700 1 -- start start 2024-05-08T23:32:56.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6fabfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fac0d00d0 0x7f6fac0b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6fabfff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac0d3e80 con 0x7f6fac160ab0 2024-05-08T23:32:56.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6fabfff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac0ce780 con 0x7f6fac0b4610 2024-05-08T23:32:56.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6fabfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac160e90 con 0x7f6fac0d00d0 2024-05-08T23:32:56.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faaffd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f6fac160ab0 0x7f6fac0cf9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:59430/0 (socket says 172.21.15.81:59430) 2024-05-08T23:32:56.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faaffd700 1 -- 172.21.15.81:0/1421956791 learned_addr learned my addr 172.21.15.81:0/1421956791 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:32:56.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6fa3fff700 1 --2- 172.21.15.81:0/1421956791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fac0d00d0 0x7f6fac0b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 -- 172.21.15.81:0/1421956791 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 85245249 0 0) 0x7f6fac0d3e80 con 0x7f6fac160ab0 2024-05-08T23:32:56.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 -- 172.21.15.81:0/1421956791 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f980035d0 con 0x7f6fac160ab0 2024-05-08T23:32:56.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 -- 172.21.15.81:0/1421956791 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2217976052 0 0) 0x7f6f980035d0 con 0x7f6fac160ab0 2024-05-08T23:32:56.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 -- 172.21.15.81:0/1421956791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fac0d00d0 msgr2=0x7f6fac0b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:56.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 --2- 172.21.15.81:0/1421956791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fac0d00d0 0x7f6fac0b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 -- 172.21.15.81:0/1421956791 >> v1:172.21.15.117:6789/0 conn(0x7f6fac0b4610 legacy=0x7f6fac0b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 -- 172.21.15.81:0/1421956791 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac0cea60 con 0x7f6fac160ab0 2024-05-08T23:32:56.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 -- 172.21.15.81:0/1421956791 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f6f9c001670 con 0x7f6fac160ab0 2024-05-08T23:32:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 -- 172.21.15.81:0/1421956791 >> v1:172.21.15.81:6789/0 conn(0x7f6fac160ab0 legacy=0x7f6fac0cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 --2- 172.21.15.81:0/1421956791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f98003a50 0x7f6f98003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 --2- 172.21.15.81:0/1421956791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f98004870 0x7f6f98004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 --2- 172.21.15.81:0/1421956791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fac0d00d0 0x7f6f98005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 -- 172.21.15.81:0/1421956791 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac160e90 con 0x7f6f98003a50 2024-05-08T23:32:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 -- 172.21.15.81:0/1421956791 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9800bae0 con 0x7f6f98004870 2024-05-08T23:32:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 -- 172.21.15.81:0/1421956791 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9800bc50 con 0x7f6fac0d00d0 2024-05-08T23:32:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faaffd700 1 --2- 172.21.15.81:0/1421956791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f98003a50 0x7f6f98003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6fa3fff700 1 --2- 172.21.15.81:0/1421956791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f98004870 0x7f6f98004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6fab7fe700 1 --2- 172.21.15.81:0/1421956791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fac0d00d0 0x7f6f98005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faaffd700 1 -- 172.21.15.81:0/1421956791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fac0d00d0 msgr2=0x7f6f98005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faaffd700 1 --2- 172.21.15.81:0/1421956791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fac0d00d0 0x7f6f98005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faaffd700 1 -- 172.21.15.81:0/1421956791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f98004870 msgr2=0x7f6f98004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faaffd700 1 --2- 172.21.15.81:0/1421956791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f98004870 0x7f6f98004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faaffd700 1 -- 172.21.15.81:0/1421956791 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f980035d0 con 0x7f6f98003a50 2024-05-08T23:32:56.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6fa3fff700 1 --2- 172.21.15.81:0/1421956791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f98004870 0x7f6f98004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:56.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6fab7fe700 1 --2- 172.21.15.81:0/1421956791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fac0d00d0 0x7f6f98005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:56.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faaffd700 1 --2- 172.21.15.81:0/1421956791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f98003a50 0x7f6f98003e30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00c290 tx=0x7f6f980058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3031465aa53f5a64 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:56.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 -- 172.21.15.81:0/1421956791 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c007b90 con 0x7f6f98003a50 2024-05-08T23:32:56.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 -- 172.21.15.81:0/1421956791 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c00cd70 con 0x7f6f98003a50 2024-05-08T23:32:56.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.830+0000 7f6faa7fc700 1 -- 172.21.15.81:0/1421956791 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c005bc0 con 0x7f6f98003a50 2024-05-08T23:32:56.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fabfff700 1 -- 172.21.15.81:0/1421956791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f98003a50 msgr2=0x7f6f98003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fabfff700 1 --2- 172.21.15.81:0/1421956791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f98003a50 0x7f6f98003e30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00c290 tx=0x7f6f980058a0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fab7fe700 1 -- 172.21.15.81:0/1421956791 reap_dead start 2024-05-08T23:32:56.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fabfff700 1 -- 172.21.15.81:0/1421956791 shutdown_connections 2024-05-08T23:32:56.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fabfff700 1 -- 172.21.15.81:0/1421956791 >> 172.21.15.81:0/1421956791 conn(0x7f6fac09c390 msgr2=0x7f6fac0d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:56.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fabfff700 1 -- 172.21.15.81:0/1421956791 shutdown_connections 2024-05-08T23:32:56.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fabfff700 1 -- 172.21.15.81:0/1421956791 wait complete. 2024-05-08T23:32:56.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fabfff700 1 Processor -- start 2024-05-08T23:32:56.841 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fabfff700 1 -- start start 2024-05-08T23:32:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fabfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f98003a50 0x7f6fac1f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fabfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fac1e5d10 0x7f6fac0afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fabfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fac0b1590 0x7f6fac0b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fabfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0b0520 con 0x7f6fac1e5d10 2024-05-08T23:32:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fabfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0b0800 con 0x7f6fac0b1590 2024-05-08T23:32:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fabfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0b0ae0 con 0x7f6f98003a50 2024-05-08T23:32:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fa3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fac1e5d10 0x7f6fac0afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fa3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fac1e5d10 0x7f6fac0afcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57830/0 (socket says 172.21.15.81:57830) 2024-05-08T23:32:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fa3fff700 1 -- 172.21.15.81:0/1502722901 learned_addr learned my addr 172.21.15.81:0/1502722901 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:32:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fab7fe700 1 --2- 172.21.15.81:0/1502722901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fac0b1590 0x7f6fac0b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6faaffd700 1 --2- 172.21.15.81:0/1502722901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f98003a50 0x7f6fac1f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fa3fff700 1 -- 172.21.15.81:0/1502722901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f98003a50 msgr2=0x7f6fac1f28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fa3fff700 1 --2- 172.21.15.81:0/1502722901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f98003a50 0x7f6fac1f28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fa3fff700 1 -- 172.21.15.81:0/1502722901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fac0b1590 msgr2=0x7f6fac0b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:56.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fa3fff700 1 --2- 172.21.15.81:0/1502722901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fac0b1590 0x7f6fac0b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:56.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fa3fff700 1 -- 172.21.15.81:0/1502722901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac0cbf70 con 0x7f6fac1e5d10 2024-05-08T23:32:56.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.834+0000 7f6fa3fff700 1 --2- 172.21.15.81:0/1502722901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fac1e5d10 0x7f6fac0afcf0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6f940118d0 tx=0x7f6f94011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:56.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.838+0000 7f6fa37fe700 1 -- 172.21.15.81:0/1502722901 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f94004220 con 0x7f6fac1e5d10 2024-05-08T23:32:56.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.838+0000 7f6fa37fe700 1 -- 172.21.15.81:0/1502722901 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f94004380 con 0x7f6fac1e5d10 2024-05-08T23:32:56.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.838+0000 7f6fabfff700 1 -- 172.21.15.81:0/1502722901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fac0b0230 con 0x7f6fac1e5d10 2024-05-08T23:32:56.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.838+0000 7f6fa37fe700 1 -- 172.21.15.81:0/1502722901 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f940036c0 con 0x7f6fac1e5d10 2024-05-08T23:32:56.843 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.838+0000 7f6fabfff700 1 -- 172.21.15.81:0/1502722901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fac0b2f70 con 0x7f6fac1e5d10 2024-05-08T23:32:56.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.838+0000 7f6fa37fe700 1 -- 172.21.15.81:0/1502722901 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6f94003820 con 0x7f6fac1e5d10 2024-05-08T23:32:56.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.838+0000 7f6fabfff700 1 -- 172.21.15.81:0/1502722901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fac0cbd90 con 0x7f6fac1e5d10 2024-05-08T23:32:56.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.838+0000 7f6fa37fe700 1 --2- 172.21.15.81:0/1502722901 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6f8c066730 0x7f6f8c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:56.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.838+0000 7f6fa37fe700 1 -- 172.21.15.81:0/1502722901 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f94084310 con 0x7f6fac1e5d10 2024-05-08T23:32:56.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.838+0000 7f6faaffd700 1 --2- 172.21.15.81:0/1502722901 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6f8c066730 0x7f6f8c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:56.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.838+0000 7f6faaffd700 1 --2- 172.21.15.81:0/1502722901 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6f8c066730 0x7f6f8c068bf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c015040 tx=0x7f6f9c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:56.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:56.846+0000 7f6fa37fe700 1 -- 172.21.15.81:0/1502722901 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6f94051710 con 0x7f6fac1e5d10 2024-05-08T23:32:57.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:57.034+0000 7f6fabfff700 1 -- 172.21.15.81:0/1502722901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6fac0cc100 con 0x7f6fac1e5d10 2024-05-08T23:32:57.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:57.034+0000 7f6fa37fe700 1 -- 172.21.15.81:0/1502722901 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6f94054d30 con 0x7f6fac1e5d10 2024-05-08T23:32:57.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:57.038+0000 7f6fabfff700 1 -- 172.21.15.81:0/1502722901 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6f8c066730 msgr2=0x7f6f8c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:57.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:57.038+0000 7f6fabfff700 1 --2- 172.21.15.81:0/1502722901 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6f8c066730 0x7f6f8c068bf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c015040 tx=0x7f6f9c007000 comp rx=0 tx=0).stop 2024-05-08T23:32:57.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:57.038+0000 7f6fabfff700 1 -- 172.21.15.81:0/1502722901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fac1e5d10 msgr2=0x7f6fac0afcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:57.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:57.038+0000 7f6fabfff700 1 --2- 172.21.15.81:0/1502722901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fac1e5d10 0x7f6fac0afcf0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6f940118d0 tx=0x7f6f94011be0 comp rx=0 tx=0).stop 2024-05-08T23:32:57.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:57.038+0000 7f6fabfff700 1 -- 172.21.15.81:0/1502722901 shutdown_connections 2024-05-08T23:32:57.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:57.038+0000 7f6fabfff700 1 --2- 172.21.15.81:0/1502722901 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6f8c066730 0x7f6f8c068bf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:57.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:57.038+0000 7f6fabfff700 1 --2- 172.21.15.81:0/1502722901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fac0b1590 0x7f6fac0b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:57.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:57.038+0000 7f6fabfff700 1 --2- 172.21.15.81:0/1502722901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6fac1e5d10 0x7f6fac0afcf0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:57.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:57.038+0000 7f6fabfff700 1 --2- 172.21.15.81:0/1502722901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f98003a50 0x7f6fac1f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:57.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:57.038+0000 7f6fabfff700 1 -- 172.21.15.81:0/1502722901 >> 172.21.15.81:0/1502722901 conn(0x7f6fac09c390 msgr2=0x7f6fac1566c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:57.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:57.038+0000 7f6fabfff700 1 -- 172.21.15.81:0/1502722901 shutdown_connections 2024-05-08T23:32:57.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:32:57.038+0000 7f6fabfff700 1 -- 172.21.15.81:0/1502722901 wait complete. 2024-05-08T23:32:57.043 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:32:57.054 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2024-05-08T23:32:57.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:57.054 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:57.067 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[39193]: fuse finished with error 0 and tester_r 0 2024-05-08T23:32:57.069 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:33:03.170 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:33:03.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:03.170 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:33:03.179 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:33:03.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.298+0000 7f5d0a1d5700 1 Processor -- start 2024-05-08T23:33:03.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.298+0000 7f5d0a1d5700 1 -- start start 2024-05-08T23:33:03.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.298+0000 7f5d0a1d5700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5d040ce990 0x7f5d0415f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:03.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.298+0000 7f5d0a1d5700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d0415f770 0x7f5d040b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:03.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.298+0000 7f5d0a1d5700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0415eda0 con 0x7f5d0415afa0 2024-05-08T23:33:03.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.298+0000 7f5d0a1d5700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0415ea70 con 0x7f5d040ce990 2024-05-08T23:33:03.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.298+0000 7f5d0a1d5700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0415ba90 con 0x7f5d0415f770 2024-05-08T23:33:03.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.298+0000 7f5d091d3700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5d0415afa0 0x7f5d0415b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:59432/0 (socket says 172.21.15.81:59432) 2024-05-08T23:33:03.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.298+0000 7f5d091d3700 1 -- 172.21.15.81:0/726949813 learned_addr learned my addr 172.21.15.81:0/726949813 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:03.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d089d2700 1 --2- 172.21.15.81:0/726949813 >> v2:172.21.15.117:3300/0 conn(0x7f5d040ce990 0x7f5d0415f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:03.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d099d4700 1 --2- 172.21.15.81:0/726949813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d0415f770 0x7f5d040b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:03.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d03fff700 1 -- 172.21.15.81:0/726949813 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035138272 0 0) 0x7f5d0415eda0 con 0x7f5d0415afa0 2024-05-08T23:33:03.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d03fff700 1 -- 172.21.15.81:0/726949813 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cf00035d0 con 0x7f5d0415afa0 2024-05-08T23:33:03.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d089d2700 1 -- 172.21.15.81:0/726949813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d0415f770 msgr2=0x7f5d040b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:03.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d089d2700 1 --2- 172.21.15.81:0/726949813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d0415f770 0x7f5d040b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:03.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d089d2700 1 -- 172.21.15.81:0/726949813 >> v1:172.21.15.81:6789/0 conn(0x7f5d0415afa0 legacy=0x7f5d0415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:03.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d089d2700 1 -- 172.21.15.81:0/726949813 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0415bd40 con 0x7f5d040ce990 2024-05-08T23:33:03.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d089d2700 1 --2- 172.21.15.81:0/726949813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d040ce990 0x7f5d0415f230 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009d90 tx=0x7f5cec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feebcd325a2dffb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:03.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d03fff700 1 -- 172.21.15.81:0/726949813 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec00e070 con 0x7f5d040ce990 2024-05-08T23:33:03.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d03fff700 1 -- 172.21.15.81:0/726949813 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec00bb20 con 0x7f5d040ce990 2024-05-08T23:33:03.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d03fff700 1 -- 172.21.15.81:0/726949813 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec00be90 con 0x7f5d040ce990 2024-05-08T23:33:03.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d0a1d5700 1 -- 172.21.15.81:0/726949813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d040ce990 msgr2=0x7f5d0415f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:03.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d0a1d5700 1 --2- 172.21.15.81:0/726949813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d040ce990 0x7f5d0415f230 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009d90 tx=0x7f5cec00b330 comp rx=0 tx=0).stop 2024-05-08T23:33:03.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d0a1d5700 1 -- 172.21.15.81:0/726949813 shutdown_connections 2024-05-08T23:33:03.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d0a1d5700 1 --2- 172.21.15.81:0/726949813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d0415f770 0x7f5d040b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:03.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d0a1d5700 1 --2- 172.21.15.81:0/726949813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d040ce990 0x7f5d0415f230 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:03.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d0a1d5700 1 -- 172.21.15.81:0/726949813 >> 172.21.15.81:0/726949813 conn(0x7f5d0406c330 msgr2=0x7f5d0406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:03.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d0a1d5700 1 -- 172.21.15.81:0/726949813 shutdown_connections 2024-05-08T23:33:03.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.302+0000 7f5d0a1d5700 1 -- 172.21.15.81:0/726949813 wait complete. 2024-05-08T23:33:03.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.306+0000 7f5d0a1d5700 1 Processor -- start 2024-05-08T23:33:03.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.306+0000 7f5d0a1d5700 1 -- start start 2024-05-08T23:33:03.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.306+0000 7f5d0a1d5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d040ce990 0x7f5d04157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:03.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.306+0000 7f5d0a1d5700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d0415afa0 0x7f5d04142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:03.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.306+0000 7f5d0a1d5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d0415f770 0x7f5d04154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:03.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.306+0000 7f5d0a1d5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04142d80 con 0x7f5d0415f770 2024-05-08T23:33:03.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.306+0000 7f5d0a1d5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04156ce0 con 0x7f5d040ce990 2024-05-08T23:33:03.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.306+0000 7f5d0a1d5700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04157860 con 0x7f5d0415afa0 2024-05-08T23:33:03.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d099d4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d0415f770 0x7f5d04154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:03.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d091d3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d040ce990 0x7f5d04157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:03.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d099d4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d0415f770 0x7f5d04154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57840/0 (socket says 172.21.15.81:57840) 2024-05-08T23:33:03.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d099d4700 1 -- 172.21.15.81:0/2443519051 learned_addr learned my addr 172.21.15.81:0/2443519051 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:03.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d089d2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d0415afa0 0x7f5d04142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:03.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d099d4700 1 -- 172.21.15.81:0/2443519051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d0415afa0 msgr2=0x7f5d04142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:03.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d099d4700 1 --2- 172.21.15.81:0/2443519051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d0415afa0 0x7f5d04142840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:03.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d099d4700 1 -- 172.21.15.81:0/2443519051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d040ce990 msgr2=0x7f5d04157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:03.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d099d4700 1 --2- 172.21.15.81:0/2443519051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d040ce990 0x7f5d04157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:03.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d099d4700 1 -- 172.21.15.81:0/2443519051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec009a70 con 0x7f5d0415f770 2024-05-08T23:33:03.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d099d4700 1 --2- 172.21.15.81:0/2443519051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d0415f770 0x7f5d04154260 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00e750 tx=0x7f5cfc00ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:03.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d01ffb700 1 -- 172.21.15.81:0/2443519051 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc009560 con 0x7f5d0415f770 2024-05-08T23:33:03.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d01ffb700 1 -- 172.21.15.81:0/2443519051 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cfc013680 con 0x7f5d0415f770 2024-05-08T23:33:03.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d0a1d5700 1 -- 172.21.15.81:0/2443519051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d04156f30 con 0x7f5d0415f770 2024-05-08T23:33:03.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d01ffb700 1 -- 172.21.15.81:0/2443519051 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc013a10 con 0x7f5d0415f770 2024-05-08T23:33:03.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d0a1d5700 1 -- 172.21.15.81:0/2443519051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d041574f0 con 0x7f5d0415f770 2024-05-08T23:33:03.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d0a1d5700 1 -- 172.21.15.81:0/2443519051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d040684b0 con 0x7f5d0415f770 2024-05-08T23:33:03.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.310+0000 7f5d01ffb700 1 -- 172.21.15.81:0/2443519051 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5cfc013b70 con 0x7f5d0415f770 2024-05-08T23:33:03.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.314+0000 7f5d01ffb700 1 --2- 172.21.15.81:0/2443519051 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5ce4066730 0x7f5ce4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:03.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.314+0000 7f5d01ffb700 1 -- 172.21.15.81:0/2443519051 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cfc083ea0 con 0x7f5d0415f770 2024-05-08T23:33:03.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.314+0000 7f5d091d3700 1 --2- 172.21.15.81:0/2443519051 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5ce4066730 0x7f5ce4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:03.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.314+0000 7f5d091d3700 1 --2- 172.21.15.81:0/2443519051 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5ce4066730 0x7f5ce4068bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4007fd0 tx=0x7f5cf4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:03.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.322+0000 7f5d01ffb700 1 -- 172.21.15.81:0/2443519051 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5cfc051200 con 0x7f5d0415f770 2024-05-08T23:33:03.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.534+0000 7f5d0a1d5700 1 -- 172.21.15.81:0/2443519051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5d041579a0 con 0x7f5d0415f770 2024-05-08T23:33:03.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.534+0000 7f5d01ffb700 1 -- 172.21.15.81:0/2443519051 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5cfc054820 con 0x7f5d0415f770 2024-05-08T23:33:03.538 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:33:03.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.538+0000 7f5d0a1d5700 1 -- 172.21.15.81:0/2443519051 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5ce4066730 msgr2=0x7f5ce4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:03.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.538+0000 7f5d0a1d5700 1 --2- 172.21.15.81:0/2443519051 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5ce4066730 0x7f5ce4068bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4007fd0 tx=0x7f5cf4009040 comp rx=0 tx=0).stop 2024-05-08T23:33:03.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.538+0000 7f5d0a1d5700 1 -- 172.21.15.81:0/2443519051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d0415f770 msgr2=0x7f5d04154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:03.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.538+0000 7f5d0a1d5700 1 --2- 172.21.15.81:0/2443519051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d0415f770 0x7f5d04154260 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00e750 tx=0x7f5cfc00ea60 comp rx=0 tx=0).stop 2024-05-08T23:33:03.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.542+0000 7f5d0a1d5700 1 -- 172.21.15.81:0/2443519051 shutdown_connections 2024-05-08T23:33:03.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.542+0000 7f5d0a1d5700 1 --2- 172.21.15.81:0/2443519051 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5ce4066730 0x7f5ce4068bf0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:03.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.542+0000 7f5d0a1d5700 1 --2- 172.21.15.81:0/2443519051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d0415f770 0x7f5d04154260 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:03.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.542+0000 7f5d0a1d5700 1 --2- 172.21.15.81:0/2443519051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d0415afa0 0x7f5d04142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:03.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.542+0000 7f5d0a1d5700 1 --2- 172.21.15.81:0/2443519051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d040ce990 0x7f5d04157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:03.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.542+0000 7f5d0a1d5700 1 -- 172.21.15.81:0/2443519051 >> 172.21.15.81:0/2443519051 conn(0x7f5d0406c330 msgr2=0x7f5d040cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:03.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.542+0000 7f5d0a1d5700 1 -- 172.21.15.81:0/2443519051 shutdown_connections 2024-05-08T23:33:03.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:03.542+0000 7f5d0a1d5700 1 -- 172.21.15.81:0/2443519051 wait complete. 2024-05-08T23:33:03.558 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:33:03.558 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:33:03.559 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2024-05-08T23:33:03.559 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:33:03.559 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:33:03.559 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:33:03.559 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:33:03.559 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:33:03.559 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:33:03.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:03.559 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2024-05-08T23:33:03.566 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:33:03.566 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:33:03.566 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:33:03.566 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:03.566 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-05-08T23:33:03.566 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:03.566 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:33:03.566 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:33:03.566 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:33:03.566 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:33:03.566 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:33:03.566 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:33:03.567 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:33:03.567 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:33:03.567 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T23:33:03.567 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42432sec preferred_lft 42432sec 2024-05-08T23:33:03.567 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2024-05-08T23:33:03.567 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:03.567 INFO:teuthology.orchestra.run.smithi081.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T23:33:03.567 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 02:42:22:62:7a:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T23:33:03.567 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T23:33:03.567 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:03.567 INFO:teuthology.orchestra.run.smithi081.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:33:03.567 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:2b:10:6b:08:e3 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:33:03.567 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:33:03.567 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:03.567 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::48fc:c7ff:fe41:6d94/64 scope link 2024-05-08T23:33:03.568 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:03.568 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:33:03.568 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:2b:10:6b:08:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:33:03.568 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::702b:10ff:fe6b:8e3/64 scope link 2024-05-08T23:33:03.568 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:03.568 INFO:teuthology.orchestra.run.smithi081.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:33:03.568 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:9d:1b:4e:6b:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:33:03.568 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::709d:1bff:fe4e:6bd5/64 scope link 2024-05-08T23:33:03.568 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:03.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:03.568 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:33:03.622 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-05-08T23:33:03.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:03.623 DEBUG:teuthology.orchestra.run.smithi081:> route 2024-05-08T23:33:03.632 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2024-05-08T23:33:03.632 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:33:03.633 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T23:33:03.633 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T23:33:03.633 INFO:teuthology.orchestra.run.smithi081.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T23:33:03.633 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T23:33:03.633 INFO:teuthology.orchestra.run.smithi081.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T23:33:03.633 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:33:03.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:03.633 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-05-08T23:33:03.633 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-05-08T23:33:03.633 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T23:33:03.633 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T23:33:03.634 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T23:33:03.634 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-05-08T23:33:03.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:03.671 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-05-08T23:33:03.722 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T23:33:03.722 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:33:03.722 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:33:03.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:03.723 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2024-05-08T23:33:03.735 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T23:33:03.735 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:33:03.735 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:33:03.742 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:33:03.743 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:33:03.743 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:33:03.789 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:33:03.807 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:33:03.807 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:33:03.818 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:33:03.819 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:33:03.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:03.820 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:33:03.826 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T23:33:03.826 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T23:33:03.829 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:33:03.842 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:33:03.843 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:33:03.900 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:33:03.900 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:33:03.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:03.901 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:33:03.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2024-05-08T23:33:03.918+0000 7f1360a0c640 -1 init, newargv = 0x55782d312c30 newargc=15 2024-05-08T23:33:03.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[39609]: starting ceph client 2024-05-08T23:33:03.928 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[39609]: starting fuse 2024-05-08T23:33:05.010 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:33:05.029 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:33:05.029 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:33:05.041 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:33:05.042 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:33:05.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:05.043 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:33:05.050 INFO:teuthology.orchestra.run.smithi081.stdout:52 2024-05-08T23:33:05.050 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T23:33:05.050 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:33:05.050 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> import re 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> import os 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T23:33:05.051 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T23:33:05.052 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T23:33:05.052 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2024-05-08T23:33:05.052 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2024-05-08T23:33:05.052 DEBUG:teuthology.orchestra.run.smithi081:> return f 2024-05-08T23:33:05.052 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T23:33:05.052 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:33:05.052 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2024-05-08T23:33:05.052 DEBUG:teuthology.orchestra.run.smithi081:> ' 2024-05-08T23:33:05.174 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.39609.asok 2024-05-08T23:33:05.360 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39609.asok 2024-05-08T23:33:05.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:05.360 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39609.asok status 2024-05-08T23:33:05.463 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:33:05.463 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2024-05-08T23:33:05.463 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T23:33:05.463 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:33:05.463 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2024-05-08T23:33:05.463 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2024-05-08T23:33:05.463 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T23:33:05.463 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "39609", 2024-05-08T23:33:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2024-05-08T23:33:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:33:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2024-05-08T23:33:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2024-05-08T23:33:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 6100, 2024-05-08T23:33:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2024-05-08T23:33:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2024-05-08T23:33:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2024-05-08T23:33:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 6100 2024-05-08T23:33:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:33:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:33:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:33:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:33:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2375653654 2024-05-08T23:33:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:33:05.465 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:33:05.465 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:33:05.465 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:33:05.465 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:33:05.465 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2375653654 2024-05-08T23:33:05.465 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:33:05.465 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.6100 192.168.0.1:0/2375653654", 2024-05-08T23:33:05.465 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/2375653654", 2024-05-08T23:33:05.465 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2024-05-08T23:33:05.465 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 23, 2024-05-08T23:33:05.465 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 52, 2024-05-08T23:33:05.465 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2024-05-08T23:33:05.465 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2024-05-08T23:33:05.465 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2024-05-08T23:33:05.465 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:33:05.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:05.473 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2024-05-08T23:33:05.473 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T23:33:05.481 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2024-05-08T23:33:05.481 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T23:33:05.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:05.481 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:33:05.492 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:33:05.492 DEBUG:teuthology.orchestra.run.smithi081:> import os 2024-05-08T23:33:05.493 DEBUG:teuthology.orchestra.run.smithi081:> import errno 2024-05-08T23:33:05.493 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:33:05.493 DEBUG:teuthology.orchestra.run.smithi081:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2024-05-08T23:33:05.493 DEBUG:teuthology.orchestra.run.smithi081:> try: 2024-05-08T23:33:05.493 DEBUG:teuthology.orchestra.run.smithi081:> if False: 2024-05-08T23:33:05.493 DEBUG:teuthology.orchestra.run.smithi081:> ret = os.read(fd, 1024) 2024-05-08T23:33:05.493 DEBUG:teuthology.orchestra.run.smithi081:> else: 2024-05-08T23:33:05.493 DEBUG:teuthology.orchestra.run.smithi081:> os.write(fd, b'"'"'content'"'"') 2024-05-08T23:33:05.493 DEBUG:teuthology.orchestra.run.smithi081:> except OSError as e: 2024-05-08T23:33:05.493 DEBUG:teuthology.orchestra.run.smithi081:> if e.errno != errno.EPERM: 2024-05-08T23:33:05.493 DEBUG:teuthology.orchestra.run.smithi081:> raise 2024-05-08T23:33:05.493 DEBUG:teuthology.orchestra.run.smithi081:> else: 2024-05-08T23:33:05.493 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("client does not check permission of data pool") 2024-05-08T23:33:05.493 DEBUG:teuthology.orchestra.run.smithi081:> ' 2024-05-08T23:33:05.749 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow w pool=cephfs_data' 2024-05-08T23:33:05.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.882+0000 7fad765e7700 1 Processor -- start 2024-05-08T23:33:05.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.882+0000 7fad765e7700 1 -- start start 2024-05-08T23:33:05.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.882+0000 7fad765e7700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fad70160920 0x7fad700ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:05.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.882+0000 7fad765e7700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad700d0050 0x7fad700b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:05.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.882+0000 7fad765e7700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad700d3d40 con 0x7fad700d1da0 2024-05-08T23:33:05.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.882+0000 7fad765e7700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad700d0d00 con 0x7fad70160920 2024-05-08T23:33:05.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.882+0000 7fad765e7700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad700d0fe0 con 0x7fad700d0050 2024-05-08T23:33:05.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.882+0000 7fad755e5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fad70160920 0x7fad700ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:05.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.882+0000 7fad74de4700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fad700d1da0 0x7fad700cf940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34246/0 (socket says 172.21.15.81:34246) 2024-05-08T23:33:05.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.882+0000 7fad74de4700 1 -- 172.21.15.81:0/2650279145 learned_addr learned my addr 172.21.15.81:0/2650279145 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:05.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad75de6700 1 --2- 172.21.15.81:0/2650279145 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad700d0050 0x7fad700b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:05.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad755e5700 1 -- 172.21.15.81:0/2650279145 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad700d0050 msgr2=0x7fad700b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:05.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad755e5700 1 --2- 172.21.15.81:0/2650279145 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad700d0050 0x7fad700b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:05.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad755e5700 1 -- 172.21.15.81:0/2650279145 >> v1:172.21.15.81:6789/0 conn(0x7fad700d1da0 legacy=0x7fad700cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:05.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad67fff700 1 -- 172.21.15.81:0/2650279145 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875082100 0 0) 0x7fad700d3d40 con 0x7fad700d1da0 2024-05-08T23:33:05.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad755e5700 1 -- 172.21.15.81:0/2650279145 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad700d12c0 con 0x7fad70160920 2024-05-08T23:33:05.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad755e5700 1 --2- 172.21.15.81:0/2650279145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad70160920 0x7fad700ce6f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fad60009850 tx=0x7fad60009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49502201b291eabb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:05.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad67fff700 1 -- 172.21.15.81:0/2650279145 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60011830 con 0x7fad70160920 2024-05-08T23:33:05.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad67fff700 1 -- 172.21.15.81:0/2650279145 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad60011990 con 0x7fad70160920 2024-05-08T23:33:05.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad67fff700 1 -- 172.21.15.81:0/2650279145 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60011d40 con 0x7fad70160920 2024-05-08T23:33:05.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad765e7700 1 -- 172.21.15.81:0/2650279145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad70160920 msgr2=0x7fad700ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:05.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad765e7700 1 --2- 172.21.15.81:0/2650279145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad70160920 0x7fad700ce6f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fad60009850 tx=0x7fad60009b60 comp rx=0 tx=0).stop 2024-05-08T23:33:05.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad765e7700 1 -- 172.21.15.81:0/2650279145 shutdown_connections 2024-05-08T23:33:05.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad765e7700 1 --2- 172.21.15.81:0/2650279145 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad700d0050 0x7fad700b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:05.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad765e7700 1 --2- 172.21.15.81:0/2650279145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad70160920 0x7fad700ce6f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:05.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad765e7700 1 -- 172.21.15.81:0/2650279145 >> 172.21.15.81:0/2650279145 conn(0x7fad7009c340 msgr2=0x7fad700d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:05.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad765e7700 1 -- 172.21.15.81:0/2650279145 shutdown_connections 2024-05-08T23:33:05.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.886+0000 7fad765e7700 1 -- 172.21.15.81:0/2650279145 wait complete. 2024-05-08T23:33:05.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.890+0000 7fad765e7700 1 Processor -- start 2024-05-08T23:33:05.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.890+0000 7fad765e7700 1 -- start start 2024-05-08T23:33:05.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.890+0000 7fad765e7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad700d0050 0x7fad7015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:05.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.890+0000 7fad765e7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad700d1da0 0x7fad7015e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:05.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.890+0000 7fad765e7700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad70160920 0x7fad7015ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:05.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.890+0000 7fad765e7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7015dce0 con 0x7fad700d1da0 2024-05-08T23:33:05.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.890+0000 7fad765e7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7015af30 con 0x7fad700d0050 2024-05-08T23:33:05.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.890+0000 7fad765e7700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7015d0d0 con 0x7fad70160920 2024-05-08T23:33:05.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.890+0000 7fad74de4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad700d1da0 0x7fad7015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:05.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.890+0000 7fad755e5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad700d0050 0x7fad7015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:05.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.890+0000 7fad74de4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad700d1da0 0x7fad7015e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36604/0 (socket says 172.21.15.81:36604) 2024-05-08T23:33:05.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.890+0000 7fad74de4700 1 -- 172.21.15.81:0/3765164927 learned_addr learned my addr 172.21.15.81:0/3765164927 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:05.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.890+0000 7fad75de6700 1 --2- 172.21.15.81:0/3765164927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad70160920 0x7fad7015ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:05.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.894+0000 7fad74de4700 1 -- 172.21.15.81:0/3765164927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad70160920 msgr2=0x7fad7015ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:05.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.894+0000 7fad74de4700 1 --2- 172.21.15.81:0/3765164927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad70160920 0x7fad7015ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:05.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.894+0000 7fad74de4700 1 -- 172.21.15.81:0/3765164927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad700d0050 msgr2=0x7fad7015a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:05.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.894+0000 7fad74de4700 1 --2- 172.21.15.81:0/3765164927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad700d0050 0x7fad7015a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:05.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.894+0000 7fad74de4700 1 -- 172.21.15.81:0/3765164927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad600094b0 con 0x7fad700d1da0 2024-05-08T23:33:05.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.894+0000 7fad74de4700 1 --2- 172.21.15.81:0/3765164927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad700d1da0 0x7fad7015e170 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c4e0 tx=0x7fad6c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:05.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.894+0000 7fad65ffb700 1 -- 172.21.15.81:0/3765164927 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c004ca0 con 0x7fad700d1da0 2024-05-08T23:33:05.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.894+0000 7fad65ffb700 1 -- 172.21.15.81:0/3765164927 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c004e20 con 0x7fad700d1da0 2024-05-08T23:33:05.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.894+0000 7fad765e7700 1 -- 172.21.15.81:0/3765164927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad70160e80 con 0x7fad700d1da0 2024-05-08T23:33:05.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.894+0000 7fad65ffb700 1 -- 172.21.15.81:0/3765164927 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c007310 con 0x7fad700d1da0 2024-05-08T23:33:05.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.894+0000 7fad765e7700 1 -- 172.21.15.81:0/3765164927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7015d690 con 0x7fad700d1da0 2024-05-08T23:33:05.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.894+0000 7fad765e7700 1 -- 172.21.15.81:0/3765164927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad7015de90 con 0x7fad700d1da0 2024-05-08T23:33:05.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.898+0000 7fad65ffb700 1 -- 172.21.15.81:0/3765164927 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fad6c007470 con 0x7fad700d1da0 2024-05-08T23:33:05.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.898+0000 7fad65ffb700 1 --2- 172.21.15.81:0/3765164927 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fad54066730 0x7fad54068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:05.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.898+0000 7fad65ffb700 1 -- 172.21.15.81:0/3765164927 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad6c083dc0 con 0x7fad700d1da0 2024-05-08T23:33:05.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.898+0000 7fad755e5700 1 --2- 172.21.15.81:0/3765164927 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fad54066730 0x7fad54068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:05.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.898+0000 7fad755e5700 1 --2- 172.21.15.81:0/3765164927 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fad54066730 0x7fad54068bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fad600090b0 tx=0x7fad6001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:05.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:05.906+0000 7fad65ffb700 1 -- 172.21.15.81:0/3765164927 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fad7015de90 con 0x7fad700d1da0 2024-05-08T23:33:06.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.134+0000 7fad765e7700 1 -- 172.21.15.81:0/3765164927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]} v 0) v1 -- 0x7fad700684b0 con 0x7fad700d1da0 2024-05-08T23:33:06.142 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.138+0000 7fad65ffb700 1 -- 172.21.15.81:0/3765164927 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]}]=0 updated caps for client.0 v16) v1 ==== 183+0+0 (secure 0 0 0) 0x7fad6c0510b0 con 0x7fad700d1da0 2024-05-08T23:33:06.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.142+0000 7fad765e7700 1 -- 172.21.15.81:0/3765164927 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fad54066730 msgr2=0x7fad54068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.142+0000 7fad765e7700 1 --2- 172.21.15.81:0/3765164927 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fad54066730 0x7fad54068bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fad600090b0 tx=0x7fad6001c040 comp rx=0 tx=0).stop 2024-05-08T23:33:06.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.142+0000 7fad765e7700 1 -- 172.21.15.81:0/3765164927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad700d1da0 msgr2=0x7fad7015e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.142+0000 7fad765e7700 1 --2- 172.21.15.81:0/3765164927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad700d1da0 0x7fad7015e170 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c4e0 tx=0x7fad6c00c7f0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.142+0000 7fad765e7700 1 -- 172.21.15.81:0/3765164927 shutdown_connections 2024-05-08T23:33:06.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.142+0000 7fad765e7700 1 --2- 172.21.15.81:0/3765164927 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fad54066730 0x7fad54068bf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.142+0000 7fad765e7700 1 --2- 172.21.15.81:0/3765164927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad70160920 0x7fad7015ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.142+0000 7fad765e7700 1 --2- 172.21.15.81:0/3765164927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad700d1da0 0x7fad7015e170 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.142+0000 7fad765e7700 1 --2- 172.21.15.81:0/3765164927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad700d0050 0x7fad7015a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.142+0000 7fad765e7700 1 -- 172.21.15.81:0/3765164927 >> 172.21.15.81:0/3765164927 conn(0x7fad7009c340 msgr2=0x7fad700cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:06.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.142+0000 7fad765e7700 1 -- 172.21.15.81:0/3765164927 shutdown_connections 2024-05-08T23:33:06.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.142+0000 7fad765e7700 1 -- 172.21.15.81:0/3765164927 wait complete. 2024-05-08T23:33:06.146 INFO:teuthology.orchestra.run.smithi081.stderr:updated caps for client.0 2024-05-08T23:33:06.157 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:33:06.157 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:33:06.163 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:33:06.251 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:33:06.254 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:33:06.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11359e700 1 Processor -- start 2024-05-08T23:33:06.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11359e700 1 -- start start 2024-05-08T23:33:06.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11359e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe114162700 0x7fe1140d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11359e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe114162ae0 0x7fe1140d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11359e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1140cf780 con 0x7fe1140d15e0 2024-05-08T23:33:06.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11359e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1140d0f80 con 0x7fe114162700 2024-05-08T23:33:06.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11359e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1140d1260 con 0x7fe114162ae0 2024-05-08T23:33:06.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11259c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe114162700 0x7fe1140d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe112d9d700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe1140d15e0 0x7fe1140cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34262/0 (socket says 172.21.15.81:34262) 2024-05-08T23:33:06.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe112d9d700 1 -- 172.21.15.81:0/4247257964 learned_addr learned my addr 172.21.15.81:0/4247257964 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:06.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe111d9b700 1 --2- 172.21.15.81:0/4247257964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe114162ae0 0x7fe1140d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11259c700 1 -- 172.21.15.81:0/4247257964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe114162ae0 msgr2=0x7fe1140d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:06.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11259c700 1 --2- 172.21.15.81:0/4247257964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe114162ae0 0x7fe1140d3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11259c700 1 -- 172.21.15.81:0/4247257964 >> v1:172.21.15.81:6789/0 conn(0x7fe1140d15e0 legacy=0x7fe1140cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11259c700 1 -- 172.21.15.81:0/4247257964 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1140cf9d0 con 0x7fe114162700 2024-05-08T23:33:06.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11159a700 1 -- 172.21.15.81:0/4247257964 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141131266 0 0) 0x7fe1140cf780 con 0x7fe1140d15e0 2024-05-08T23:33:06.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11259c700 1 --2- 172.21.15.81:0/4247257964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe114162700 0x7fe1140d01d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe104009d90 tx=0x7fe10400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7137366bfb08e4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:06.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11159a700 1 -- 172.21.15.81:0/4247257964 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10400e070 con 0x7fe114162700 2024-05-08T23:33:06.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11159a700 1 -- 172.21.15.81:0/4247257964 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10400bbd0 con 0x7fe114162700 2024-05-08T23:33:06.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11159a700 1 -- 172.21.15.81:0/4247257964 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104012430 con 0x7fe114162700 2024-05-08T23:33:06.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11359e700 1 -- 172.21.15.81:0/4247257964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe114162700 msgr2=0x7fe1140d01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11359e700 1 --2- 172.21.15.81:0/4247257964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe114162700 0x7fe1140d01d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe104009d90 tx=0x7fe10400b330 comp rx=0 tx=0).stop 2024-05-08T23:33:06.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11359e700 1 -- 172.21.15.81:0/4247257964 shutdown_connections 2024-05-08T23:33:06.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11359e700 1 --2- 172.21.15.81:0/4247257964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe114162ae0 0x7fe1140d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11359e700 1 --2- 172.21.15.81:0/4247257964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe114162700 0x7fe1140d01d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11359e700 1 -- 172.21.15.81:0/4247257964 >> 172.21.15.81:0/4247257964 conn(0x7fe11409c370 msgr2=0x7fe1140d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:06.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11359e700 1 -- 172.21.15.81:0/4247257964 shutdown_connections 2024-05-08T23:33:06.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.274+0000 7fe11359e700 1 -- 172.21.15.81:0/4247257964 wait complete. 2024-05-08T23:33:06.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11359e700 1 Processor -- start 2024-05-08T23:33:06.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11359e700 1 -- start start 2024-05-08T23:33:06.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11359e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1140d15e0 0x7fe114156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11359e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe114162700 0x7fe1140c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11359e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe114162ae0 0x7fe1140b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11359e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1140c9960 con 0x7fe1140d15e0 2024-05-08T23:33:06.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11359e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1140c74a0 con 0x7fe114162700 2024-05-08T23:33:06.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11359e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1140c8e80 con 0x7fe114162ae0 2024-05-08T23:33:06.283 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1140d15e0 0x7fe114156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1140d15e0 0x7fe114156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36618/0 (socket says 172.21.15.81:36618) 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11259c700 1 -- 172.21.15.81:0/2719523843 learned_addr learned my addr 172.21.15.81:0/2719523843 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe111d9b700 1 --2- 172.21.15.81:0/2719523843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe114162700 0x7fe1140c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe112d9d700 1 --2- 172.21.15.81:0/2719523843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe114162ae0 0x7fe1140b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11259c700 1 -- 172.21.15.81:0/2719523843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe114162ae0 msgr2=0x7fe1140b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11259c700 1 --2- 172.21.15.81:0/2719523843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe114162ae0 0x7fe1140b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11259c700 1 -- 172.21.15.81:0/2719523843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe114162700 msgr2=0x7fe1140c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11259c700 1 --2- 172.21.15.81:0/2719523843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe114162700 0x7fe1140c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11259c700 1 -- 172.21.15.81:0/2719523843 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe104009a70 con 0x7fe1140d15e0 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe111d9b700 1 --2- 172.21.15.81:0/2719523843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe114162700 0x7fe1140c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11259c700 1 --2- 172.21.15.81:0/2719523843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1140d15e0 0x7fe114156280 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe104014040 tx=0x7fe104004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe10affd700 1 -- 172.21.15.81:0/2719523843 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104012430 con 0x7fe1140d15e0 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe10affd700 1 -- 172.21.15.81:0/2719523843 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe104020070 con 0x7fe1140d15e0 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe10affd700 1 -- 172.21.15.81:0/2719523843 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104004260 con 0x7fe1140d15e0 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11359e700 1 -- 172.21.15.81:0/2719523843 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1140c9160 con 0x7fe1140d15e0 2024-05-08T23:33:06.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.278+0000 7fe11359e700 1 -- 172.21.15.81:0/2719523843 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1141567c0 con 0x7fe1140d15e0 2024-05-08T23:33:06.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.282+0000 7fe11359e700 1 -- 172.21.15.81:0/2719523843 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1140c9680 con 0x7fe1140d15e0 2024-05-08T23:33:06.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.282+0000 7fe10affd700 1 -- 172.21.15.81:0/2719523843 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe1040043c0 con 0x7fe1140d15e0 2024-05-08T23:33:06.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.282+0000 7fe10affd700 1 --2- 172.21.15.81:0/2719523843 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe0f4066730 0x7fe0f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.282+0000 7fe10affd700 1 -- 172.21.15.81:0/2719523843 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe104022070 con 0x7fe1140d15e0 2024-05-08T23:33:06.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.282+0000 7fe111d9b700 1 --2- 172.21.15.81:0/2719523843 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe0f4066730 0x7fe0f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.282+0000 7fe111d9b700 1 --2- 172.21.15.81:0/2719523843 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe0f4066730 0x7fe0f4068bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe1140cd620 tx=0x7fe0fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:06.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.290+0000 7fe10affd700 1 -- 172.21.15.81:0/2719523843 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe1140c9680 con 0x7fe1140d15e0 2024-05-08T23:33:06.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.466+0000 7fe11359e700 1 -- 172.21.15.81:0/2719523843 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1140d3c00 con 0x7fe1140d15e0 2024-05-08T23:33:06.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.466+0000 7fe10affd700 1 -- 172.21.15.81:0/2719523843 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16101 (secure 0 0 0) 0x7fe104053380 con 0x7fe1140d15e0 2024-05-08T23:33:06.471 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:06.471 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":52,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:32:41.855853+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:39.161796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:39.853963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:33:06.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.470+0000 7fe11359e700 1 -- 172.21.15.81:0/2719523843 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe0f4066730 msgr2=0x7fe0f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.470+0000 7fe11359e700 1 --2- 172.21.15.81:0/2719523843 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe0f4066730 0x7fe0f4068bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe1140cd620 tx=0x7fe0fc008040 comp rx=0 tx=0).stop 2024-05-08T23:33:06.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.470+0000 7fe11359e700 1 -- 172.21.15.81:0/2719523843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1140d15e0 msgr2=0x7fe114156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.470+0000 7fe11359e700 1 --2- 172.21.15.81:0/2719523843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1140d15e0 0x7fe114156280 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe104014040 tx=0x7fe104004af0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.470+0000 7fe11359e700 1 -- 172.21.15.81:0/2719523843 shutdown_connections 2024-05-08T23:33:06.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.470+0000 7fe11359e700 1 --2- 172.21.15.81:0/2719523843 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe0f4066730 0x7fe0f4068bf0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.470+0000 7fe11359e700 1 --2- 172.21.15.81:0/2719523843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe114162ae0 0x7fe1140b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.470+0000 7fe11359e700 1 --2- 172.21.15.81:0/2719523843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe114162700 0x7fe1140c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.470+0000 7fe11359e700 1 --2- 172.21.15.81:0/2719523843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1140d15e0 0x7fe114156280 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.470+0000 7fe11359e700 1 -- 172.21.15.81:0/2719523843 >> 172.21.15.81:0/2719523843 conn(0x7fe11409c370 msgr2=0x7fe114160020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:06.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.470+0000 7fe11359e700 1 -- 172.21.15.81:0/2719523843 shutdown_connections 2024-05-08T23:33:06.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.470+0000 7fe11359e700 1 -- 172.21.15.81:0/2719523843 wait complete. 2024-05-08T23:33:06.486 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:32:39.161796+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:32:39.853963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:33:06.487 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:33:06.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.602+0000 7f1adda34700 1 Processor -- start 2024-05-08T23:33:06.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.602+0000 7f1adda34700 1 -- start start 2024-05-08T23:33:06.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.602+0000 7f1adda34700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad8160ce0 0x7f1ad80b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.602+0000 7f1adda34700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1ad80d01c0 0x7f1ad80b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.602+0000 7f1adda34700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad80cf480 con 0x7f1ad80d3d40 2024-05-08T23:33:06.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.602+0000 7f1adda34700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad80ce890 con 0x7f1ad80d01c0 2024-05-08T23:33:06.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.602+0000 7f1adda34700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad80ceb50 con 0x7f1ad8160ce0 2024-05-08T23:33:06.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.602+0000 7f1ad77fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1ad80d01c0 0x7f1ad80b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.602+0000 7f1ad6ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad8160ce0 0x7f1ad80b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.602+0000 7f1ad6ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad8160ce0 0x7f1ad80b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:39878/0 (socket says 172.21.15.81:39878) 2024-05-08T23:33:06.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.602+0000 7f1ad67fc700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f1ad80d3d40 0x7f1ad80cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.81:60310/0 (socket says 172.21.15.81:60310) 2024-05-08T23:33:06.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.602+0000 7f1ad77fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1ad80d01c0 0x7f1ad80b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:57682/0 (socket says 172.21.15.81:57682) 2024-05-08T23:33:06.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.602+0000 7f1ad6ffd700 1 -- 172.21.15.81:0/3967985599 learned_addr learned my addr 172.21.15.81:0/3967985599 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:06.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.602+0000 7f1ad5ffb700 1 -- 172.21.15.81:0/3967985599 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161642582 0 0) 0x7f1ad80cf480 con 0x7f1ad80d3d40 2024-05-08T23:33:06.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1ad5ffb700 1 -- 172.21.15.81:0/3967985599 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ac00035d0 con 0x7f1ad80d3d40 2024-05-08T23:33:06.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1ad6ffd700 1 -- 172.21.15.81:0/3967985599 >> v2:172.21.15.117:3300/0 conn(0x7f1ad80d01c0 msgr2=0x7f1ad80b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:06.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1ad6ffd700 1 --2- 172.21.15.81:0/3967985599 >> v2:172.21.15.117:3300/0 conn(0x7f1ad80d01c0 0x7f1ad80b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1ad6ffd700 1 -- 172.21.15.81:0/3967985599 >> v1:172.21.15.117:6789/0 conn(0x7f1ad80d3d40 legacy=0x7f1ad80cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1ad6ffd700 1 -- 172.21.15.81:0/3967985599 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad80cee30 con 0x7f1ad8160ce0 2024-05-08T23:33:06.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1ad77fe700 1 --2- 172.21.15.81:0/3967985599 >> v2:172.21.15.117:3300/0 conn(0x7f1ad80d01c0 0x7f1ad80b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:06.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1ad6ffd700 1 --2- 172.21.15.81:0/3967985599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad8160ce0 0x7f1ad80b2040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009910 tx=0x7f1acc009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb36f07dbaf5d23e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:06.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1ad5ffb700 1 -- 172.21.15.81:0/3967985599 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc011830 con 0x7f1ad8160ce0 2024-05-08T23:33:06.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1ad5ffb700 1 -- 172.21.15.81:0/3967985599 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc011990 con 0x7f1ad8160ce0 2024-05-08T23:33:06.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1ad5ffb700 1 -- 172.21.15.81:0/3967985599 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc011d00 con 0x7f1ad8160ce0 2024-05-08T23:33:06.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1adda34700 1 -- 172.21.15.81:0/3967985599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad8160ce0 msgr2=0x7f1ad80b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1adda34700 1 --2- 172.21.15.81:0/3967985599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad8160ce0 0x7f1ad80b2040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009910 tx=0x7f1acc009c20 comp rx=0 tx=0).stop 2024-05-08T23:33:06.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1adda34700 1 -- 172.21.15.81:0/3967985599 shutdown_connections 2024-05-08T23:33:06.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1adda34700 1 --2- 172.21.15.81:0/3967985599 >> v2:172.21.15.117:3300/0 conn(0x7f1ad80d01c0 0x7f1ad80b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1adda34700 1 --2- 172.21.15.81:0/3967985599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad8160ce0 0x7f1ad80b2040 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1adda34700 1 -- 172.21.15.81:0/3967985599 >> 172.21.15.81:0/3967985599 conn(0x7f1ad809c4c0 msgr2=0x7f1ad80d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:06.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1adda34700 1 -- 172.21.15.81:0/3967985599 shutdown_connections 2024-05-08T23:33:06.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1adda34700 1 -- 172.21.15.81:0/3967985599 wait complete. 2024-05-08T23:33:06.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.606+0000 7f1adda34700 1 Processor -- start 2024-05-08T23:33:06.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1adda34700 1 -- start start 2024-05-08T23:33:06.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1adda34700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad80d01c0 0x7f1ad81f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1adda34700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad80d3d40 0x7f1ad80c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1adda34700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ad8160ce0 0x7f1ad80b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1adda34700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad80b79a0 con 0x7f1ad8160ce0 2024-05-08T23:33:06.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1adda34700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad81f84f0 con 0x7f1ad80d01c0 2024-05-08T23:33:06.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1adda34700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad80c7fc0 con 0x7f1ad80d3d40 2024-05-08T23:33:06.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1ad77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ad8160ce0 0x7f1ad80b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1ad77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ad8160ce0 0x7f1ad80b7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36632/0 (socket says 172.21.15.81:36632) 2024-05-08T23:33:06.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1ad77fe700 1 -- 172.21.15.81:0/1158422571 learned_addr learned my addr 172.21.15.81:0/1158422571 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:06.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1ad67fc700 1 --2- 172.21.15.81:0/1158422571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad80d3d40 0x7f1ad80c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1ad6ffd700 1 --2- 172.21.15.81:0/1158422571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad80d01c0 0x7f1ad81f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1ad77fe700 1 -- 172.21.15.81:0/1158422571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad80d3d40 msgr2=0x7f1ad80c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1ad77fe700 1 --2- 172.21.15.81:0/1158422571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad80d3d40 0x7f1ad80c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1ad77fe700 1 -- 172.21.15.81:0/1158422571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad80d01c0 msgr2=0x7f1ad81f7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1ad77fe700 1 --2- 172.21.15.81:0/1158422571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad80d01c0 0x7f1ad81f7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1ad77fe700 1 -- 172.21.15.81:0/1158422571 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad0009a70 con 0x7f1ad8160ce0 2024-05-08T23:33:06.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1ad77fe700 1 --2- 172.21.15.81:0/1158422571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ad8160ce0 0x7f1ad80b7dc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000efc0 tx=0x7f1ad000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:06.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1abffff700 1 -- 172.21.15.81:0/1158422571 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000ca90 con 0x7f1ad8160ce0 2024-05-08T23:33:06.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1abffff700 1 -- 172.21.15.81:0/1158422571 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad000f040 con 0x7f1ad8160ce0 2024-05-08T23:33:06.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1adda34700 1 -- 172.21.15.81:0/1158422571 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1acc009570 con 0x7f1ad8160ce0 2024-05-08T23:33:06.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1abffff700 1 -- 172.21.15.81:0/1158422571 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000ce10 con 0x7f1ad8160ce0 2024-05-08T23:33:06.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.610+0000 7f1adda34700 1 -- 172.21.15.81:0/1158422571 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad80b7000 con 0x7f1ad8160ce0 2024-05-08T23:33:06.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.614+0000 7f1adda34700 1 -- 172.21.15.81:0/1158422571 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad80b7ae0 con 0x7f1ad8160ce0 2024-05-08T23:33:06.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.614+0000 7f1abffff700 1 -- 172.21.15.81:0/1158422571 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1ad0003a40 con 0x7f1ad8160ce0 2024-05-08T23:33:06.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.614+0000 7f1abffff700 1 --2- 172.21.15.81:0/1158422571 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1ac4066730 0x7f1ac4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.614+0000 7f1ad6ffd700 1 --2- 172.21.15.81:0/1158422571 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1ac4066730 0x7f1ac4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.614+0000 7f1abffff700 1 -- 172.21.15.81:0/1158422571 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ad0085000 con 0x7f1ad8160ce0 2024-05-08T23:33:06.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.614+0000 7f1ad6ffd700 1 --2- 172.21.15.81:0/1158422571 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1ac4066730 0x7f1ac4068bf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1acc014040 tx=0x7f1acc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:06.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.622+0000 7f1abffff700 1 -- 172.21.15.81:0/1158422571 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1ad0052350 con 0x7f1ad8160ce0 2024-05-08T23:33:06.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.790+0000 7f1adda34700 1 -- 172.21.15.81:0/1158422571 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ad80b7ae0 con 0x7f1ad8160ce0 2024-05-08T23:33:06.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.790+0000 7f1abffff700 1 -- 172.21.15.81:0/1158422571 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ad80b7ae0 con 0x7f1ad8160ce0 2024-05-08T23:33:06.792 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:33:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.790+0000 7f1adda34700 1 -- 172.21.15.81:0/1158422571 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1ac4066730 msgr2=0x7f1ac4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.790+0000 7f1adda34700 1 --2- 172.21.15.81:0/1158422571 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1ac4066730 0x7f1ac4068bf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1acc014040 tx=0x7f1acc008030 comp rx=0 tx=0).stop 2024-05-08T23:33:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.790+0000 7f1adda34700 1 -- 172.21.15.81:0/1158422571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ad8160ce0 msgr2=0x7f1ad80b7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.790+0000 7f1adda34700 1 --2- 172.21.15.81:0/1158422571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ad8160ce0 0x7f1ad80b7dc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000efc0 tx=0x7f1ad000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.790+0000 7f1adda34700 1 -- 172.21.15.81:0/1158422571 shutdown_connections 2024-05-08T23:33:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.790+0000 7f1adda34700 1 --2- 172.21.15.81:0/1158422571 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1ac4066730 0x7f1ac4068bf0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.790+0000 7f1adda34700 1 --2- 172.21.15.81:0/1158422571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ad8160ce0 0x7f1ad80b7dc0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.790+0000 7f1adda34700 1 --2- 172.21.15.81:0/1158422571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad80d3d40 0x7f1ad80c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.790+0000 7f1adda34700 1 --2- 172.21.15.81:0/1158422571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad80d01c0 0x7f1ad81f7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.790+0000 7f1adda34700 1 -- 172.21.15.81:0/1158422571 >> 172.21.15.81:0/1158422571 conn(0x7f1ad809c4c0 msgr2=0x7f1ad815e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.790+0000 7f1adda34700 1 -- 172.21.15.81:0/1158422571 shutdown_connections 2024-05-08T23:33:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.790+0000 7f1adda34700 1 -- 172.21.15.81:0/1158422571 wait complete. 2024-05-08T23:33:06.806 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:33:06.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.918+0000 7f620f59e700 1 Processor -- start 2024-05-08T23:33:06.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.918+0000 7f620f59e700 1 -- start start 2024-05-08T23:33:06.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620f59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f621015d010 0x7f621009a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620f59e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f621009a9c0 con 0x7f621006bfb0 2024-05-08T23:33:06.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620f59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62100b4c50 con 0x7f621006b890 2024-05-08T23:33:06.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620f59e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62100b63c0 con 0x7f621015d010 2024-05-08T23:33:06.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620ed9d700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f621006bfb0 0x7f62100addc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34264/0 (socket says 172.21.15.81:34264) 2024-05-08T23:33:06.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620ed9d700 1 -- 172.21.15.81:0/2336821935 learned_addr learned my addr 172.21.15.81:0/2336821935 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:06.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620e59c700 1 --2- 172.21.15.81:0/2336821935 >> v2:172.21.15.81:3300/0 conn(0x7f621015d010 0x7f621009a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620d59a700 1 -- 172.21.15.81:0/2336821935 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611234941 0 0) 0x7f6208001490 con 0x7f621006bfb0 2024-05-08T23:33:06.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620d59a700 1 -- 172.21.15.81:0/2336821935 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61f80035d0 con 0x7f621006bfb0 2024-05-08T23:33:06.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620e59c700 1 -- 172.21.15.81:0/2336821935 >> v1:172.21.15.117:6789/0 conn(0x7f621006b890 legacy=0x7f62100b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620e59c700 1 -- 172.21.15.81:0/2336821935 >> v1:172.21.15.81:6789/0 conn(0x7f621006bfb0 legacy=0x7f62100addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620e59c700 1 -- 172.21.15.81:0/2336821935 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62100b4ef0 con 0x7f621015d010 2024-05-08T23:33:06.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620e59c700 1 --2- 172.21.15.81:0/2336821935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f621015d010 0x7f621009a480 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6204009850 tx=0x7f6204009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4064caee30fbbb72 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:06.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620d59a700 1 -- 172.21.15.81:0/2336821935 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6204011830 con 0x7f621015d010 2024-05-08T23:33:06.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620d59a700 1 -- 172.21.15.81:0/2336821935 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6204011990 con 0x7f621015d010 2024-05-08T23:33:06.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620d59a700 1 -- 172.21.15.81:0/2336821935 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6204011d40 con 0x7f621015d010 2024-05-08T23:33:06.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620f59e700 1 -- 172.21.15.81:0/2336821935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f621015d010 msgr2=0x7f621009a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620f59e700 1 --2- 172.21.15.81:0/2336821935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f621015d010 0x7f621009a480 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6204009850 tx=0x7f6204009b60 comp rx=0 tx=0).stop 2024-05-08T23:33:06.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620f59e700 1 -- 172.21.15.81:0/2336821935 shutdown_connections 2024-05-08T23:33:06.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620f59e700 1 --2- 172.21.15.81:0/2336821935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f621015d010 0x7f621009a480 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620f59e700 1 -- 172.21.15.81:0/2336821935 >> 172.21.15.81:0/2336821935 conn(0x7f62100b2ae0 msgr2=0x7f62100b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:06.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620f59e700 1 -- 172.21.15.81:0/2336821935 shutdown_connections 2024-05-08T23:33:06.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.922+0000 7f620f59e700 1 -- 172.21.15.81:0/2336821935 wait complete. 2024-05-08T23:33:06.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620f59e700 1 Processor -- start 2024-05-08T23:33:06.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620f59e700 1 -- start start 2024-05-08T23:33:06.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620f59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f621006b890 0x7f6210143040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620f59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f621006bfb0 0x7f62101489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620f59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f621015d010 0x7f6210145ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620f59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100ccc90 con 0x7f621006bfb0 2024-05-08T23:33:06.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620f59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6210148f30 con 0x7f621006b890 2024-05-08T23:33:06.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620f59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6210146020 con 0x7f621015d010 2024-05-08T23:33:06.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620dd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f621006bfb0 0x7f62101489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620dd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f621006bfb0 0x7f62101489f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36652/0 (socket says 172.21.15.81:36652) 2024-05-08T23:33:06.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620dd9b700 1 -- 172.21.15.81:0/1281801529 learned_addr learned my addr 172.21.15.81:0/1281801529 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:06.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620e59c700 1 --2- 172.21.15.81:0/1281801529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f621006b890 0x7f6210143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620ed9d700 1 --2- 172.21.15.81:0/1281801529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f621015d010 0x7f6210145ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620dd9b700 1 -- 172.21.15.81:0/1281801529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f621015d010 msgr2=0x7f6210145ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620dd9b700 1 --2- 172.21.15.81:0/1281801529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f621015d010 0x7f6210145ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620dd9b700 1 -- 172.21.15.81:0/1281801529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f621006b890 msgr2=0x7f6210143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:06.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620dd9b700 1 --2- 172.21.15.81:0/1281801529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f621006b890 0x7f6210143040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:06.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620dd9b700 1 -- 172.21.15.81:0/1281801529 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62040094b0 con 0x7f621006bfb0 2024-05-08T23:33:06.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.926+0000 7f620dd9b700 1 --2- 172.21.15.81:0/1281801529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f621006bfb0 0x7f62101489f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f620000c500 tx=0x7f620000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:06.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.930+0000 7f61feffd700 1 -- 172.21.15.81:0/1281801529 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62000052d0 con 0x7f621006bfb0 2024-05-08T23:33:06.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.930+0000 7f61feffd700 1 -- 172.21.15.81:0/1281801529 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6200005430 con 0x7f621006bfb0 2024-05-08T23:33:06.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.930+0000 7f61feffd700 1 -- 172.21.15.81:0/1281801529 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62000055a0 con 0x7f621006bfb0 2024-05-08T23:33:06.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.930+0000 7f620f59e700 1 -- 172.21.15.81:0/1281801529 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62100d0300 con 0x7f621006bfb0 2024-05-08T23:33:06.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.930+0000 7f620f59e700 1 -- 172.21.15.81:0/1281801529 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621015ef30 con 0x7f621006bfb0 2024-05-08T23:33:06.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.930+0000 7f620f59e700 1 -- 172.21.15.81:0/1281801529 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621015eab0 con 0x7f621006bfb0 2024-05-08T23:33:06.934 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.930+0000 7f61feffd700 1 -- 172.21.15.81:0/1281801529 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f62000038e0 con 0x7f621006bfb0 2024-05-08T23:33:06.934 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.930+0000 7f61feffd700 1 --2- 172.21.15.81:0/1281801529 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f61f0066730 0x7f61f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:06.934 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.930+0000 7f61feffd700 1 -- 172.21.15.81:0/1281801529 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6200083490 con 0x7f621006bfb0 2024-05-08T23:33:06.934 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.930+0000 7f620e59c700 1 --2- 172.21.15.81:0/1281801529 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f61f0066730 0x7f61f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:06.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.934+0000 7f620e59c700 1 --2- 172.21.15.81:0/1281801529 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f61f0066730 0x7f61f0068bf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f62040090b0 tx=0x7f620401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:06.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:06.938+0000 7f61feffd700 1 -- 172.21.15.81:0/1281801529 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f621015eab0 con 0x7f621006bfb0 2024-05-08T23:33:07.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.110+0000 7f620f59e700 1 -- 172.21.15.81:0/1281801529 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f621015eab0 con 0x7f621006bfb0 2024-05-08T23:33:07.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.110+0000 7f61feffd700 1 -- 172.21.15.81:0/1281801529 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7f621015eab0 con 0x7f621006bfb0 2024-05-08T23:33:07.112 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:33:07.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.110+0000 7f620f59e700 1 -- 172.21.15.81:0/1281801529 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f61f0066730 msgr2=0x7f61f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.110+0000 7f620f59e700 1 --2- 172.21.15.81:0/1281801529 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f61f0066730 0x7f61f0068bf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f62040090b0 tx=0x7f620401c040 comp rx=0 tx=0).stop 2024-05-08T23:33:07.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.110+0000 7f620f59e700 1 -- 172.21.15.81:0/1281801529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f621006bfb0 msgr2=0x7f62101489f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.110+0000 7f620f59e700 1 --2- 172.21.15.81:0/1281801529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f621006bfb0 0x7f62101489f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f620000c500 tx=0x7f620000c810 comp rx=0 tx=0).stop 2024-05-08T23:33:07.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.110+0000 7f620f59e700 1 -- 172.21.15.81:0/1281801529 shutdown_connections 2024-05-08T23:33:07.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.110+0000 7f620f59e700 1 --2- 172.21.15.81:0/1281801529 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f61f0066730 0x7f61f0068bf0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.110+0000 7f620f59e700 1 --2- 172.21.15.81:0/1281801529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f621015d010 0x7f6210145ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.110+0000 7f620f59e700 1 --2- 172.21.15.81:0/1281801529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f621006bfb0 0x7f62101489f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.110+0000 7f620f59e700 1 --2- 172.21.15.81:0/1281801529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f621006b890 0x7f6210143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.110+0000 7f620f59e700 1 -- 172.21.15.81:0/1281801529 >> 172.21.15.81:0/1281801529 conn(0x7f62100b2ae0 msgr2=0x7f62100af450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:07.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.110+0000 7f620f59e700 1 -- 172.21.15.81:0/1281801529 shutdown_connections 2024-05-08T23:33:07.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.110+0000 7f620f59e700 1 -- 172.21.15.81:0/1281801529 wait complete. 2024-05-08T23:33:07.128 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:33:07.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.246+0000 7f3f864e0700 1 Processor -- start 2024-05-08T23:33:07.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.246+0000 7f3f864e0700 1 -- start start 2024-05-08T23:33:07.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f864e0700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f80160920 0x7f3f800ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f864e0700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f800d1da0 0x7f3f800cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f864e0700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f800d3d40 con 0x7f3f800cec30 2024-05-08T23:33:07.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f864e0700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f800d0d00 con 0x7f3f80160920 2024-05-08T23:33:07.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f864e0700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800d0fe0 con 0x7f3f800d1da0 2024-05-08T23:33:07.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f854de700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f80160920 0x7f3f800ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f854de700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f80160920 0x7f3f800ce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.81:57728/0 (socket says 172.21.15.81:57728) 2024-05-08T23:33:07.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f854de700 1 -- 172.21.15.81:0/180871098 learned_addr learned my addr 172.21.15.81:0/180871098 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:07.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f84cdd700 1 --2- 172.21.15.81:0/180871098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f800d1da0 0x7f3f800cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f73fff700 1 -- 172.21.15.81:0/180871098 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705525718 0 0) 0x7f3f800d3d40 con 0x7f3f800cec30 2024-05-08T23:33:07.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f73fff700 1 -- 172.21.15.81:0/180871098 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f6c0035d0 con 0x7f3f800cec30 2024-05-08T23:33:07.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f854de700 1 -- 172.21.15.81:0/180871098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f800d1da0 msgr2=0x7f3f800cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:07.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f854de700 1 --2- 172.21.15.81:0/180871098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f800d1da0 0x7f3f800cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f854de700 1 -- 172.21.15.81:0/180871098 >> v1:172.21.15.117:6789/0 conn(0x7f3f800cec30 legacy=0x7f3f80161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f854de700 1 -- 172.21.15.81:0/180871098 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f800d12c0 con 0x7f3f80160920 2024-05-08T23:33:07.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f84cdd700 1 --2- 172.21.15.81:0/180871098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f800d1da0 0x7f3f800cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:07.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f854de700 1 --2- 172.21.15.81:0/180871098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f80160920 0x7f3f800ce6f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009910 tx=0x7f3f7c009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d5e73c9328ea90f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:07.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f73fff700 1 -- 172.21.15.81:0/180871098 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c011830 con 0x7f3f80160920 2024-05-08T23:33:07.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f73fff700 1 -- 172.21.15.81:0/180871098 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c011990 con 0x7f3f80160920 2024-05-08T23:33:07.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f73fff700 1 -- 172.21.15.81:0/180871098 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c011d00 con 0x7f3f80160920 2024-05-08T23:33:07.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f864e0700 1 -- 172.21.15.81:0/180871098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f80160920 msgr2=0x7f3f800ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f864e0700 1 --2- 172.21.15.81:0/180871098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f80160920 0x7f3f800ce6f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009910 tx=0x7f3f7c009c20 comp rx=0 tx=0).stop 2024-05-08T23:33:07.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f864e0700 1 -- 172.21.15.81:0/180871098 shutdown_connections 2024-05-08T23:33:07.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f864e0700 1 --2- 172.21.15.81:0/180871098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f800d1da0 0x7f3f800cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f864e0700 1 --2- 172.21.15.81:0/180871098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f80160920 0x7f3f800ce6f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f864e0700 1 -- 172.21.15.81:0/180871098 >> 172.21.15.81:0/180871098 conn(0x7f3f8009c340 msgr2=0x7f3f800d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:07.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f864e0700 1 -- 172.21.15.81:0/180871098 shutdown_connections 2024-05-08T23:33:07.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.250+0000 7f3f864e0700 1 -- 172.21.15.81:0/180871098 wait complete. 2024-05-08T23:33:07.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.254+0000 7f3f864e0700 1 Processor -- start 2024-05-08T23:33:07.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.254+0000 7f3f864e0700 1 -- start start 2024-05-08T23:33:07.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.254+0000 7f3f864e0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f800cec30 0x7f3f8009ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.254+0000 7f3f864e0700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f800d1da0 0x7f3f8009eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.254+0000 7f3f864e0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f80160920 0x7f3f8014df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.254+0000 7f3f864e0700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8009d500 con 0x7f3f80160920 2024-05-08T23:33:07.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.254+0000 7f3f864e0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8009d7e0 con 0x7f3f800cec30 2024-05-08T23:33:07.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.254+0000 7f3f864e0700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8009dac0 con 0x7f3f800d1da0 2024-05-08T23:33:07.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.254+0000 7f3f85cdf700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f80160920 0x7f3f8014df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.254+0000 7f3f854de700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f800cec30 0x7f3f8009ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.254+0000 7f3f85cdf700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f80160920 0x7f3f8014df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36658/0 (socket says 172.21.15.81:36658) 2024-05-08T23:33:07.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.254+0000 7f3f84cdd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f800d1da0 0x7f3f8009eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.254+0000 7f3f85cdf700 1 -- 172.21.15.81:0/4178959378 learned_addr learned my addr 172.21.15.81:0/4178959378 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:07.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.258+0000 7f3f85cdf700 1 -- 172.21.15.81:0/4178959378 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f800d1da0 msgr2=0x7f3f8009eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.258+0000 7f3f85cdf700 1 --2- 172.21.15.81:0/4178959378 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f800d1da0 0x7f3f8009eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.258+0000 7f3f85cdf700 1 -- 172.21.15.81:0/4178959378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f800cec30 msgr2=0x7f3f8009ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.258+0000 7f3f85cdf700 1 --2- 172.21.15.81:0/4178959378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f800cec30 0x7f3f8009ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.258+0000 7f3f85cdf700 1 -- 172.21.15.81:0/4178959378 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f740094b0 con 0x7f3f80160920 2024-05-08T23:33:07.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.258+0000 7f3f85cdf700 1 --2- 172.21.15.81:0/4178959378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f80160920 0x7f3f8014df20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800cbc0 tx=0x7f3f7800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:07.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.258+0000 7f3f71ffb700 1 -- 172.21.15.81:0/4178959378 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78003c20 con 0x7f3f80160920 2024-05-08T23:33:07.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.258+0000 7f3f71ffb700 1 -- 172.21.15.81:0/4178959378 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f78003d80 con 0x7f3f80160920 2024-05-08T23:33:07.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.258+0000 7f3f864e0700 1 -- 172.21.15.81:0/4178959378 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7c009570 con 0x7f3f80160920 2024-05-08T23:33:07.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.258+0000 7f3f71ffb700 1 -- 172.21.15.81:0/4178959378 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7800aaa0 con 0x7f3f80160920 2024-05-08T23:33:07.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.258+0000 7f3f864e0700 1 -- 172.21.15.81:0/4178959378 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f80160e80 con 0x7f3f80160920 2024-05-08T23:33:07.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.258+0000 7f3f864e0700 1 -- 172.21.15.81:0/4178959378 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f800684b0 con 0x7f3f80160920 2024-05-08T23:33:07.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.262+0000 7f3f71ffb700 1 -- 172.21.15.81:0/4178959378 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3f7800ac00 con 0x7f3f80160920 2024-05-08T23:33:07.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.262+0000 7f3f71ffb700 1 --2- 172.21.15.81:0/4178959378 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f64066730 0x7f3f64068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.262+0000 7f3f71ffb700 1 -- 172.21.15.81:0/4178959378 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f78084080 con 0x7f3f80160920 2024-05-08T23:33:07.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.262+0000 7f3f854de700 1 --2- 172.21.15.81:0/4178959378 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f64066730 0x7f3f64068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.262+0000 7f3f854de700 1 --2- 172.21.15.81:0/4178959378 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f64066730 0x7f3f64068bf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009c60 tx=0x7f3f7c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:07.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.266+0000 7f3f71ffb700 1 -- 172.21.15.81:0/4178959378 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3f78051480 con 0x7f3f80160920 2024-05-08T23:33:07.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.434+0000 7f3f864e0700 1 -- 172.21.15.81:0/4178959378 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f800d34f0 con 0x7f3f80160920 2024-05-08T23:33:07.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.434+0000 7f3f71ffb700 1 -- 172.21.15.81:0/4178959378 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f78054aa0 con 0x7f3f80160920 2024-05-08T23:33:07.439 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:33:07.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.438+0000 7f3f864e0700 1 -- 172.21.15.81:0/4178959378 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f64066730 msgr2=0x7f3f64068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.438+0000 7f3f864e0700 1 --2- 172.21.15.81:0/4178959378 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f64066730 0x7f3f64068bf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009c60 tx=0x7f3f7c008030 comp rx=0 tx=0).stop 2024-05-08T23:33:07.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.438+0000 7f3f864e0700 1 -- 172.21.15.81:0/4178959378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f80160920 msgr2=0x7f3f8014df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.438+0000 7f3f864e0700 1 --2- 172.21.15.81:0/4178959378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f80160920 0x7f3f8014df20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800cbc0 tx=0x7f3f7800ced0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.438+0000 7f3f864e0700 1 -- 172.21.15.81:0/4178959378 shutdown_connections 2024-05-08T23:33:07.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.438+0000 7f3f864e0700 1 --2- 172.21.15.81:0/4178959378 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3f64066730 0x7f3f64068bf0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.438+0000 7f3f864e0700 1 --2- 172.21.15.81:0/4178959378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f80160920 0x7f3f8014df20 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.438+0000 7f3f864e0700 1 --2- 172.21.15.81:0/4178959378 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f800d1da0 0x7f3f8009eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.438+0000 7f3f864e0700 1 --2- 172.21.15.81:0/4178959378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f800cec30 0x7f3f8009ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.438+0000 7f3f864e0700 1 -- 172.21.15.81:0/4178959378 >> 172.21.15.81:0/4178959378 conn(0x7f3f8009c340 msgr2=0x7f3f800b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:07.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.438+0000 7f3f864e0700 1 -- 172.21.15.81:0/4178959378 shutdown_connections 2024-05-08T23:33:07.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.438+0000 7f3f864e0700 1 -- 172.21.15.81:0/4178959378 wait complete. 2024-05-08T23:33:07.454 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:33:07.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.570+0000 7f8a1a1be700 1 Processor -- start 2024-05-08T23:33:07.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.570+0000 7f8a1a1be700 1 -- start start 2024-05-08T23:33:07.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.570+0000 7f8a1a1be700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8a14071e70 0x7f8a141624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.570+0000 7f8a1a1be700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a14158960 con 0x7f8a140b3c20 2024-05-08T23:33:07.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.570+0000 7f8a1a1be700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a140ae990 con 0x7f8a14070110 2024-05-08T23:33:07.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.570+0000 7f8a1a1be700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a140b1f60 con 0x7f8a14071e70 2024-05-08T23:33:07.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.570+0000 7f8a189bb700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8a14071e70 0x7f8a141624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.570+0000 7f8a189bb700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8a14071e70 0x7f8a141624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36662/0 (socket says 172.21.15.81:36662) 2024-05-08T23:33:07.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.570+0000 7f8a189bb700 1 -- 172.21.15.81:0/3829646919 learned_addr learned my addr 172.21.15.81:0/3829646919 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:07.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.570+0000 7f8a189bb700 1 -- 172.21.15.81:0/3829646919 >> v1:172.21.15.117:6789/0 conn(0x7f8a14070110 legacy=0x7f8a140705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.570+0000 7f8a189bb700 1 -- 172.21.15.81:0/3829646919 >> v1:172.21.15.81:6789/0 conn(0x7f8a140b3c20 legacy=0x7f8a140b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.570+0000 7f8a189bb700 1 -- 172.21.15.81:0/3829646919 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a140b2170 con 0x7f8a14071e70 2024-05-08T23:33:07.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a189bb700 1 --2- 172.21.15.81:0/3829646919 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a14071e70 0x7f8a141624b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009910 tx=0x7f8a00009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f592d671213d16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:07.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a13fff700 1 -- 172.21.15.81:0/3829646919 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0000ba60 con 0x7f8a14071e70 2024-05-08T23:33:07.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a13fff700 1 -- 172.21.15.81:0/3829646919 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0000bbc0 con 0x7f8a14071e70 2024-05-08T23:33:07.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a13fff700 1 -- 172.21.15.81:0/3829646919 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a00009e60 con 0x7f8a14071e70 2024-05-08T23:33:07.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a1a1be700 1 -- 172.21.15.81:0/3829646919 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a14071e70 msgr2=0x7f8a141624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a1a1be700 1 --2- 172.21.15.81:0/3829646919 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a14071e70 0x7f8a141624b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009910 tx=0x7f8a00009c20 comp rx=0 tx=0).stop 2024-05-08T23:33:07.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a1a1be700 1 -- 172.21.15.81:0/3829646919 shutdown_connections 2024-05-08T23:33:07.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a1a1be700 1 --2- 172.21.15.81:0/3829646919 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a14071e70 0x7f8a141624b0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a1a1be700 1 -- 172.21.15.81:0/3829646919 >> 172.21.15.81:0/3829646919 conn(0x7f8a1406c330 msgr2=0x7f8a1406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:07.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a1a1be700 1 -- 172.21.15.81:0/3829646919 shutdown_connections 2024-05-08T23:33:07.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a1a1be700 1 -- 172.21.15.81:0/3829646919 wait complete. 2024-05-08T23:33:07.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a1a1be700 1 Processor -- start 2024-05-08T23:33:07.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a1a1be700 1 -- start start 2024-05-08T23:33:07.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a1a1be700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a14070110 0x7f8a14151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a1a1be700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a14071e70 0x7f8a141535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a1a1be700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a140b3c20 0x7f8a1413e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a1a1be700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a14152360 con 0x7f8a14070110 2024-05-08T23:33:07.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a1a1be700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a14152540 con 0x7f8a14071e70 2024-05-08T23:33:07.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a1a1be700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a14152820 con 0x7f8a140b3c20 2024-05-08T23:33:07.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a191bc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a14070110 0x7f8a14151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a191bc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a14070110 0x7f8a14151310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36678/0 (socket says 172.21.15.81:36678) 2024-05-08T23:33:07.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a191bc700 1 -- 172.21.15.81:0/2198708567 learned_addr learned my addr 172.21.15.81:0/2198708567 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:07.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a189bb700 1 --2- 172.21.15.81:0/2198708567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a14071e70 0x7f8a141535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a199bd700 1 --2- 172.21.15.81:0/2198708567 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a140b3c20 0x7f8a1413e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a191bc700 1 -- 172.21.15.81:0/2198708567 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a140b3c20 msgr2=0x7f8a1413e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a191bc700 1 --2- 172.21.15.81:0/2198708567 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a140b3c20 0x7f8a1413e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a191bc700 1 -- 172.21.15.81:0/2198708567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a14071e70 msgr2=0x7f8a141535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a191bc700 1 --2- 172.21.15.81:0/2198708567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a14071e70 0x7f8a141535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.574+0000 7f8a191bc700 1 -- 172.21.15.81:0/2198708567 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a00009570 con 0x7f8a14070110 2024-05-08T23:33:07.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.578+0000 7f8a191bc700 1 --2- 172.21.15.81:0/2198708567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a14070110 0x7f8a14151310 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00c500 tx=0x7f8a0c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:07.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.578+0000 7f8a11ffb700 1 -- 172.21.15.81:0/2198708567 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0052d0 con 0x7f8a14070110 2024-05-08T23:33:07.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.578+0000 7f8a11ffb700 1 -- 172.21.15.81:0/2198708567 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0c005430 con 0x7f8a14070110 2024-05-08T23:33:07.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.578+0000 7f8a1a1be700 1 -- 172.21.15.81:0/2198708567 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a14152b00 con 0x7f8a14070110 2024-05-08T23:33:07.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.578+0000 7f8a11ffb700 1 -- 172.21.15.81:0/2198708567 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0055a0 con 0x7f8a14070110 2024-05-08T23:33:07.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.578+0000 7f8a1a1be700 1 -- 172.21.15.81:0/2198708567 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a141530c0 con 0x7f8a14070110 2024-05-08T23:33:07.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.578+0000 7f8a1a1be700 1 -- 172.21.15.81:0/2198708567 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a140ce2a0 con 0x7f8a14070110 2024-05-08T23:33:07.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.578+0000 7f8a11ffb700 1 -- 172.21.15.81:0/2198708567 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8a0c0038e0 con 0x7f8a14070110 2024-05-08T23:33:07.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.578+0000 7f8a11ffb700 1 --2- 172.21.15.81:0/2198708567 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f89fc066730 0x7f89fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.578+0000 7f8a11ffb700 1 -- 172.21.15.81:0/2198708567 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a0c083510 con 0x7f8a14070110 2024-05-08T23:33:07.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.578+0000 7f8a189bb700 1 --2- 172.21.15.81:0/2198708567 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f89fc066730 0x7f89fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.578+0000 7f8a189bb700 1 --2- 172.21.15.81:0/2198708567 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f89fc066730 0x7f89fc068bf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a00006f80 tx=0x7f8a00008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:07.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.582+0000 7f8a11ffb700 1 -- 172.21.15.81:0/2198708567 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8a0c0507f0 con 0x7f8a14070110 2024-05-08T23:33:07.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.762+0000 7f8a1a1be700 1 -- 172.21.15.81:0/2198708567 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8a141532b0 con 0x7f8a14070110 2024-05-08T23:33:07.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.762+0000 7f8a11ffb700 1 -- 172.21.15.81:0/2198708567 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8a0c081e60 con 0x7f8a14070110 2024-05-08T23:33:07.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.766+0000 7f8a1a1be700 1 -- 172.21.15.81:0/2198708567 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f89fc066730 msgr2=0x7f89fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.766+0000 7f8a1a1be700 1 --2- 172.21.15.81:0/2198708567 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f89fc066730 0x7f89fc068bf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a00006f80 tx=0x7f8a00008030 comp rx=0 tx=0).stop 2024-05-08T23:33:07.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.766+0000 7f8a1a1be700 1 -- 172.21.15.81:0/2198708567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a14070110 msgr2=0x7f8a14151310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.766+0000 7f8a1a1be700 1 --2- 172.21.15.81:0/2198708567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a14070110 0x7f8a14151310 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00c500 tx=0x7f8a0c00c810 comp rx=0 tx=0).stop 2024-05-08T23:33:07.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.766+0000 7f8a1a1be700 1 -- 172.21.15.81:0/2198708567 shutdown_connections 2024-05-08T23:33:07.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.766+0000 7f8a1a1be700 1 --2- 172.21.15.81:0/2198708567 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f89fc066730 0x7f89fc068bf0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.766+0000 7f8a1a1be700 1 --2- 172.21.15.81:0/2198708567 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a140b3c20 0x7f8a1413e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.766+0000 7f8a1a1be700 1 --2- 172.21.15.81:0/2198708567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a14071e70 0x7f8a141535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.766+0000 7f8a1a1be700 1 --2- 172.21.15.81:0/2198708567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a14070110 0x7f8a14151310 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.766+0000 7f8a1a1be700 1 -- 172.21.15.81:0/2198708567 >> 172.21.15.81:0/2198708567 conn(0x7f8a1406c330 msgr2=0x7f8a140cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:07.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.766+0000 7f8a1a1be700 1 -- 172.21.15.81:0/2198708567 shutdown_connections 2024-05-08T23:33:07.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.766+0000 7f8a1a1be700 1 -- 172.21.15.81:0/2198708567 wait complete. 2024-05-08T23:33:07.770 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:33:07.781 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:33:07.781 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:33:07.789 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:33:07.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea5da0700 1 Processor -- start 2024-05-08T23:33:07.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea5da0700 1 -- start start 2024-05-08T23:33:07.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea5da0700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efea015afa0 0x7efea015b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea5da0700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea015eda0 con 0x7efea015b8c0 2024-05-08T23:33:07.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea5da0700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea015ea70 con 0x7efea00ce990 2024-05-08T23:33:07.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea5da0700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea00ced70 con 0x7efea015afa0 2024-05-08T23:33:07.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe97fff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7efea015b8c0 0x7efea015f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34288/0 (socket says 172.21.15.81:34288) 2024-05-08T23:33:07.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe97fff700 1 -- 172.21.15.81:0/4162137211 learned_addr learned my addr 172.21.15.81:0/4162137211 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:07.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe977fe700 1 -- 172.21.15.81:0/4162137211 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340422382 0 0) 0x7efea015eda0 con 0x7efea015b8c0 2024-05-08T23:33:07.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea4d9e700 1 --2- 172.21.15.81:0/4162137211 >> v2:172.21.15.117:3300/0 conn(0x7efea015afa0 0x7efea015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe977fe700 1 -- 172.21.15.81:0/4162137211 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe880035d0 con 0x7efea015b8c0 2024-05-08T23:33:07.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe977fe700 1 -- 172.21.15.81:0/4162137211 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1687341040 0 0) 0x7efe880035d0 con 0x7efea015b8c0 2024-05-08T23:33:07.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe977fe700 1 -- 172.21.15.81:0/4162137211 >> v2:172.21.15.117:3300/0 conn(0x7efea015afa0 msgr2=0x7efea015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:07.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe977fe700 1 --2- 172.21.15.81:0/4162137211 >> v2:172.21.15.117:3300/0 conn(0x7efea015afa0 0x7efea015b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe977fe700 1 -- 172.21.15.81:0/4162137211 >> v1:172.21.15.117:6789/0 conn(0x7efea00ce990 legacy=0x7efea00b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe977fe700 1 -- 172.21.15.81:0/4162137211 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea015bd40 con 0x7efea015b8c0 2024-05-08T23:33:07.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe977fe700 1 -- 172.21.15.81:0/4162137211 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7efe90001670 con 0x7efea015b8c0 2024-05-08T23:33:07.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe977fe700 1 -- 172.21.15.81:0/4162137211 >> v1:172.21.15.81:6789/0 conn(0x7efea015b8c0 legacy=0x7efea015f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe977fe700 1 --2- 172.21.15.81:0/4162137211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe88003a20 0x7efe88003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe977fe700 1 --2- 172.21.15.81:0/4162137211 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe88004870 0x7efe88004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe977fe700 1 --2- 172.21.15.81:0/4162137211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe88005230 0x7efe8800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe977fe700 1 -- 172.21.15.81:0/4162137211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea00ced70 con 0x7efe88003a20 2024-05-08T23:33:07.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe977fe700 1 -- 172.21.15.81:0/4162137211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8800c180 con 0x7efe88005230 2024-05-08T23:33:07.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe977fe700 1 -- 172.21.15.81:0/4162137211 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8800c340 con 0x7efe88004870 2024-05-08T23:33:07.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea4d9e700 1 --2- 172.21.15.81:0/4162137211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe88003a20 0x7efe88003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea559f700 1 --2- 172.21.15.81:0/4162137211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe88005230 0x7efe8800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe97fff700 1 --2- 172.21.15.81:0/4162137211 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe88004870 0x7efe88004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea4d9e700 1 -- 172.21.15.81:0/4162137211 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe88004870 msgr2=0x7efe88004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:07.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea4d9e700 1 --2- 172.21.15.81:0/4162137211 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe88004870 0x7efe88004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea4d9e700 1 -- 172.21.15.81:0/4162137211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe88005230 msgr2=0x7efe8800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:07.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea4d9e700 1 --2- 172.21.15.81:0/4162137211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe88005230 0x7efe8800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea559f700 1 -- 172.21.15.81:0/4162137211 reap_dead start 2024-05-08T23:33:07.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea4d9e700 1 -- 172.21.15.81:0/4162137211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe880035d0 con 0x7efe88003a20 2024-05-08T23:33:07.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efe97fff700 1 --2- 172.21.15.81:0/4162137211 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe88004870 0x7efe88004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:07.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.902+0000 7efea4d9e700 1 --2- 172.21.15.81:0/4162137211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe88003a20 0x7efe88003e40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0098b0 tx=0x7efe880058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2667eadc47392249 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:07.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.906+0000 7efe977fe700 1 -- 172.21.15.81:0/4162137211 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c00c070 con 0x7efe88003a20 2024-05-08T23:33:07.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.906+0000 7efe977fe700 1 -- 172.21.15.81:0/4162137211 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c013610 con 0x7efe88003a20 2024-05-08T23:33:07.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.906+0000 7efe977fe700 1 -- 172.21.15.81:0/4162137211 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c0139c0 con 0x7efe88003a20 2024-05-08T23:33:07.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.906+0000 7efea5da0700 1 -- 172.21.15.81:0/4162137211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe88003a20 msgr2=0x7efe88003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.906+0000 7efea5da0700 1 --2- 172.21.15.81:0/4162137211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe88003a20 0x7efe88003e40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0098b0 tx=0x7efe880058a0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.906+0000 7efea5da0700 1 -- 172.21.15.81:0/4162137211 shutdown_connections 2024-05-08T23:33:07.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.906+0000 7efea5da0700 1 --2- 172.21.15.81:0/4162137211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe88003a20 0x7efe88003e40 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.906+0000 7efea5da0700 1 -- 172.21.15.81:0/4162137211 >> 172.21.15.81:0/4162137211 conn(0x7efea006c330 msgr2=0x7efea006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:07.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.906+0000 7efea5da0700 1 -- 172.21.15.81:0/4162137211 shutdown_connections 2024-05-08T23:33:07.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.906+0000 7efea5da0700 1 -- 172.21.15.81:0/4162137211 wait complete. 2024-05-08T23:33:07.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.906+0000 7efea5da0700 1 Processor -- start 2024-05-08T23:33:07.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.906+0000 7efea5da0700 1 -- start start 2024-05-08T23:33:07.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea5da0700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe88003a20 0x7efea013f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea5da0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea01f4d10 0x7efea0151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea5da0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efea01e6730 0x7efea01e6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea5da0700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea01e70f0 con 0x7efea01e6730 2024-05-08T23:33:07.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea5da0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea0151300 con 0x7efea01f4d10 2024-05-08T23:33:07.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea5da0700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea0151cd0 con 0x7efe88003a20 2024-05-08T23:33:07.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea559f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efea01e6730 0x7efea01e6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea559f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efea01e6730 0x7efea01e6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36700/0 (socket says 172.21.15.81:36700) 2024-05-08T23:33:07.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea559f700 1 -- 172.21.15.81:0/580525240 learned_addr learned my addr 172.21.15.81:0/580525240 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:07.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efe97fff700 1 --2- 172.21.15.81:0/580525240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea01f4d10 0x7efea0151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea4d9e700 1 --2- 172.21.15.81:0/580525240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe88003a20 0x7efea013f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea559f700 1 -- 172.21.15.81:0/580525240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe88003a20 msgr2=0x7efea013f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea559f700 1 --2- 172.21.15.81:0/580525240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe88003a20 0x7efea013f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea559f700 1 -- 172.21.15.81:0/580525240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea01f4d10 msgr2=0x7efea0151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:07.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea559f700 1 --2- 172.21.15.81:0/580525240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea01f4d10 0x7efea0151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:07.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea559f700 1 -- 172.21.15.81:0/580525240 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea01506f0 con 0x7efea01e6730 2024-05-08T23:33:07.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea559f700 1 --2- 172.21.15.81:0/580525240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efea01e6730 0x7efea01e6bb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7efea00712a0 tx=0x7efe980021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:07.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efe957fa700 1 -- 172.21.15.81:0/580525240 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98003cf0 con 0x7efea01e6730 2024-05-08T23:33:07.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efe957fa700 1 -- 172.21.15.81:0/580525240 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe98003e50 con 0x7efea01e6730 2024-05-08T23:33:07.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea5da0700 1 -- 172.21.15.81:0/580525240 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea01509d0 con 0x7efea01e6730 2024-05-08T23:33:07.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efe957fa700 1 -- 172.21.15.81:0/580525240 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98005310 con 0x7efea01e6730 2024-05-08T23:33:07.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea5da0700 1 -- 172.21.15.81:0/580525240 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea0150f90 con 0x7efea01e6730 2024-05-08T23:33:07.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efea5da0700 1 -- 172.21.15.81:0/580525240 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea0151120 con 0x7efea01e6730 2024-05-08T23:33:07.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efe957fa700 1 -- 172.21.15.81:0/580525240 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efe98005470 con 0x7efea01e6730 2024-05-08T23:33:07.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efe957fa700 1 --2- 172.21.15.81:0/580525240 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efe8c066730 0x7efe8c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:07.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.910+0000 7efe957fa700 1 -- 172.21.15.81:0/580525240 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe98084240 con 0x7efea01e6730 2024-05-08T23:33:07.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.914+0000 7efea4d9e700 1 --2- 172.21.15.81:0/580525240 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efe8c066730 0x7efe8c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:07.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.914+0000 7efea4d9e700 1 --2- 172.21.15.81:0/580525240 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efe8c066730 0x7efe8c068bf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efe9c016040 tx=0x7efe9c00c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:07.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:07.918+0000 7efe957fa700 1 -- 172.21.15.81:0/580525240 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efe980515b0 con 0x7efea01e6730 2024-05-08T23:33:08.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.086+0000 7efea5da0700 1 -- 172.21.15.81:0/580525240 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efea00684b0 con 0x7efea01e6730 2024-05-08T23:33:08.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.090+0000 7efe957fa700 1 -- 172.21.15.81:0/580525240 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16101 (secure 0 0 0) 0x7efe98054bd0 con 0x7efea01e6730 2024-05-08T23:33:08.092 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:08.092 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":52,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:32:41.855853+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:39.161796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:39.853963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:33:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.090+0000 7efea5da0700 1 -- 172.21.15.81:0/580525240 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efe8c066730 msgr2=0x7efe8c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.090+0000 7efea5da0700 1 --2- 172.21.15.81:0/580525240 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efe8c066730 0x7efe8c068bf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efe9c016040 tx=0x7efe9c00c000 comp rx=0 tx=0).stop 2024-05-08T23:33:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.090+0000 7efea5da0700 1 -- 172.21.15.81:0/580525240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efea01e6730 msgr2=0x7efea01e6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.090+0000 7efea5da0700 1 --2- 172.21.15.81:0/580525240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efea01e6730 0x7efea01e6bb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7efea00712a0 tx=0x7efe980021e0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.090+0000 7efea5da0700 1 -- 172.21.15.81:0/580525240 shutdown_connections 2024-05-08T23:33:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.090+0000 7efea5da0700 1 --2- 172.21.15.81:0/580525240 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7efe8c066730 0x7efe8c068bf0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.090+0000 7efea5da0700 1 --2- 172.21.15.81:0/580525240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efea01e6730 0x7efea01e6bb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.090+0000 7efea5da0700 1 --2- 172.21.15.81:0/580525240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea01f4d10 0x7efea0151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.090+0000 7efea5da0700 1 --2- 172.21.15.81:0/580525240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe88003a20 0x7efea013f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.090+0000 7efea5da0700 1 -- 172.21.15.81:0/580525240 >> 172.21.15.81:0/580525240 conn(0x7efea006c330 msgr2=0x7efea00cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.090+0000 7efea5da0700 1 -- 172.21.15.81:0/580525240 shutdown_connections 2024-05-08T23:33:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.090+0000 7efea5da0700 1 -- 172.21.15.81:0/580525240 wait complete. 2024-05-08T23:33:08.107 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:32:39.161796+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:32:39.853963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:33:08.108 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:33:08.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.226+0000 7f9846dbc700 1 Processor -- start 2024-05-08T23:33:08.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.226+0000 7f9846dbc700 1 -- start start 2024-05-08T23:33:08.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f9846dbc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006ca60 0x7f98400cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:08.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f9846dbc700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98400cd470 con 0x7f98400cb970 2024-05-08T23:33:08.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f9846dbc700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98400ced50 con 0x7f984015aed0 2024-05-08T23:33:08.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f9846dbc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98400cceb0 con 0x7f984006ca60 2024-05-08T23:33:08.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f983ffff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f98400cb970 0x7f984006c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34290/0 (socket says 172.21.15.81:34290) 2024-05-08T23:33:08.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f983ffff700 1 -- 172.21.15.81:0/3260587070 learned_addr learned my addr 172.21.15.81:0/3260587070 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:08.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f9845359700 1 --2- 172.21.15.81:0/3260587070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006ca60 0x7f98400cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:08.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f983f7fe700 1 -- 172.21.15.81:0/3260587070 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675258862 0 0) 0x7f98400cd470 con 0x7f98400cb970 2024-05-08T23:33:08.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f983f7fe700 1 -- 172.21.15.81:0/3260587070 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98280035d0 con 0x7f98400cb970 2024-05-08T23:33:08.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f983f7fe700 1 -- 172.21.15.81:0/3260587070 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 272905229 0 0) 0x7f98280035d0 con 0x7f98400cb970 2024-05-08T23:33:08.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f9845359700 1 -- 172.21.15.81:0/3260587070 >> v1:172.21.15.117:6789/0 conn(0x7f984015aed0 legacy=0x7f98400b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f9845359700 1 -- 172.21.15.81:0/3260587070 >> v1:172.21.15.81:6789/0 conn(0x7f98400cb970 legacy=0x7f984006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f9845359700 1 -- 172.21.15.81:0/3260587070 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98400cd190 con 0x7f984006ca60 2024-05-08T23:33:08.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f9845359700 1 --2- 172.21.15.81:0/3260587070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006ca60 0x7f98400cc7c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9838009880 tx=0x7f9838009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a6959080b03310e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:08.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f983f7fe700 1 -- 172.21.15.81:0/3260587070 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98300019f0 con 0x7f984006ca60 2024-05-08T23:33:08.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f983f7fe700 1 -- 172.21.15.81:0/3260587070 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9838010590 con 0x7f984006ca60 2024-05-08T23:33:08.234 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f983f7fe700 1 -- 172.21.15.81:0/3260587070 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838010900 con 0x7f984006ca60 2024-05-08T23:33:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f9846dbc700 1 -- 172.21.15.81:0/3260587070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006ca60 msgr2=0x7f98400cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.230+0000 7f9846dbc700 1 --2- 172.21.15.81:0/3260587070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006ca60 0x7f98400cc7c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9838009880 tx=0x7f9838009b90 comp rx=0 tx=0).stop 2024-05-08T23:33:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.234+0000 7f9846dbc700 1 -- 172.21.15.81:0/3260587070 shutdown_connections 2024-05-08T23:33:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.234+0000 7f9846dbc700 1 --2- 172.21.15.81:0/3260587070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006ca60 0x7f98400cc7c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.234+0000 7f9846dbc700 1 -- 172.21.15.81:0/3260587070 >> 172.21.15.81:0/3260587070 conn(0x7f984006a360 msgr2=0x7f984006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:08.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.234+0000 7f9846dbc700 1 -- 172.21.15.81:0/3260587070 shutdown_connections 2024-05-08T23:33:08.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.234+0000 7f9846dbc700 1 -- 172.21.15.81:0/3260587070 wait complete. 2024-05-08T23:33:08.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.234+0000 7f9846dbc700 1 Processor -- start 2024-05-08T23:33:08.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.234+0000 7f9846dbc700 1 -- start start 2024-05-08T23:33:08.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f9846dbc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006ca60 0x7f9840143720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:08.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f9846dbc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98400cb970 0x7f98401f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:08.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f9846dbc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984015aed0 0x7f9840153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:08.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f9846dbc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98400add00 con 0x7f98400cb970 2024-05-08T23:33:08.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f9846dbc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98401f2540 con 0x7f984015aed0 2024-05-08T23:33:08.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f9846dbc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98401f2820 con 0x7f984006ca60 2024-05-08T23:33:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f983ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98400cb970 0x7f98401f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f983ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98400cb970 0x7f98401f35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36710/0 (socket says 172.21.15.81:36710) 2024-05-08T23:33:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f983ffff700 1 -- 172.21.15.81:0/3280209949 learned_addr learned my addr 172.21.15.81:0/3280209949 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f9844b58700 1 --2- 172.21.15.81:0/3280209949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006ca60 0x7f9840143720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f9845359700 1 --2- 172.21.15.81:0/3280209949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984015aed0 0x7f9840153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f983ffff700 1 -- 172.21.15.81:0/3280209949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006ca60 msgr2=0x7f9840143720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f983ffff700 1 --2- 172.21.15.81:0/3280209949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006ca60 0x7f9840143720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f983ffff700 1 -- 172.21.15.81:0/3280209949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984015aed0 msgr2=0x7f9840153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f983ffff700 1 --2- 172.21.15.81:0/3280209949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984015aed0 0x7f9840153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f983ffff700 1 -- 172.21.15.81:0/3280209949 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98380094b0 con 0x7f98400cb970 2024-05-08T23:33:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f983ffff700 1 --2- 172.21.15.81:0/3280209949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98400cb970 0x7f98401f35e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9830001700 tx=0x7f983000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f983d7fa700 1 -- 172.21.15.81:0/3280209949 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983000ada0 con 0x7f98400cb970 2024-05-08T23:33:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f983d7fa700 1 -- 172.21.15.81:0/3280209949 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98300070d0 con 0x7f98400cb970 2024-05-08T23:33:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f983d7fa700 1 -- 172.21.15.81:0/3280209949 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9830007480 con 0x7f98400cb970 2024-05-08T23:33:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f9846dbc700 1 -- 172.21.15.81:0/3280209949 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98401f2b00 con 0x7f98400cb970 2024-05-08T23:33:08.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.238+0000 7f9846dbc700 1 -- 172.21.15.81:0/3280209949 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98401f30c0 con 0x7f98400cb970 2024-05-08T23:33:08.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.242+0000 7f9846dbc700 1 -- 172.21.15.81:0/3280209949 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98400684b0 con 0x7f98400cb970 2024-05-08T23:33:08.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.242+0000 7f983d7fa700 1 -- 172.21.15.81:0/3280209949 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f98300075e0 con 0x7f98400cb970 2024-05-08T23:33:08.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.242+0000 7f983d7fa700 1 --2- 172.21.15.81:0/3280209949 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f982c066730 0x7f982c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:08.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.242+0000 7f983d7fa700 1 -- 172.21.15.81:0/3280209949 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9830083ea0 con 0x7f98400cb970 2024-05-08T23:33:08.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.242+0000 7f9844b58700 1 --2- 172.21.15.81:0/3280209949 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f982c066730 0x7f982c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:08.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.242+0000 7f9844b58700 1 --2- 172.21.15.81:0/3280209949 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f982c066730 0x7f982c068bf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9834000f60 tx=0x7f9834009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:08.251 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.250+0000 7f983d7fa700 1 -- 172.21.15.81:0/3280209949 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9830051200 con 0x7f98400cb970 2024-05-08T23:33:08.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.422+0000 7f9846dbc700 1 -- 172.21.15.81:0/3280209949 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98400af050 con 0x7f98400cb970 2024-05-08T23:33:08.424 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.422+0000 7f983d7fa700 1 -- 172.21.15.81:0/3280209949 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98300180a0 con 0x7f98400cb970 2024-05-08T23:33:08.424 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:33:08.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.426+0000 7f9846dbc700 1 -- 172.21.15.81:0/3280209949 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f982c066730 msgr2=0x7f982c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.426+0000 7f9846dbc700 1 --2- 172.21.15.81:0/3280209949 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f982c066730 0x7f982c068bf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9834000f60 tx=0x7f9834009040 comp rx=0 tx=0).stop 2024-05-08T23:33:08.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.426+0000 7f9846dbc700 1 -- 172.21.15.81:0/3280209949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98400cb970 msgr2=0x7f98401f35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.426+0000 7f9846dbc700 1 --2- 172.21.15.81:0/3280209949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98400cb970 0x7f98401f35e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9830001700 tx=0x7f983000a340 comp rx=0 tx=0).stop 2024-05-08T23:33:08.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.426+0000 7f9846dbc700 1 -- 172.21.15.81:0/3280209949 shutdown_connections 2024-05-08T23:33:08.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.426+0000 7f9846dbc700 1 --2- 172.21.15.81:0/3280209949 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f982c066730 0x7f982c068bf0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.426+0000 7f9846dbc700 1 --2- 172.21.15.81:0/3280209949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984015aed0 0x7f9840153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.426+0000 7f9846dbc700 1 --2- 172.21.15.81:0/3280209949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98400cb970 0x7f98401f35e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.426+0000 7f9846dbc700 1 --2- 172.21.15.81:0/3280209949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006ca60 0x7f9840143720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.426+0000 7f9846dbc700 1 -- 172.21.15.81:0/3280209949 >> 172.21.15.81:0/3280209949 conn(0x7f984006a360 msgr2=0x7f98400b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:08.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.426+0000 7f9846dbc700 1 -- 172.21.15.81:0/3280209949 shutdown_connections 2024-05-08T23:33:08.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.426+0000 7f9846dbc700 1 -- 172.21.15.81:0/3280209949 wait complete. 2024-05-08T23:33:08.445 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:33:08.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.562+0000 7f3586d77700 1 Processor -- start 2024-05-08T23:33:08.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.562+0000 7f3586d77700 1 -- start start 2024-05-08T23:33:08.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3586d77700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f358015aee0 0x7f35800cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:08.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3586d77700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358015b350 con 0x7f358006b370 2024-05-08T23:33:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3586d77700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358015a440 con 0x7f358015a060 2024-05-08T23:33:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3586d77700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35800cd840 con 0x7f358015aee0 2024-05-08T23:33:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3586576700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f358006b370 0x7f35800b2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34306/0 (socket says 172.21.15.81:34306) 2024-05-08T23:33:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3586576700 1 -- 172.21.15.81:0/3117640022 learned_addr learned my addr 172.21.15.81:0/3117640022 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3585574700 1 --2- 172.21.15.81:0/3117640022 >> v2:172.21.15.117:3300/0 conn(0x7f358015aee0 0x7f35800cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3584d73700 1 -- 172.21.15.81:0/3117640022 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343866499 0 0) 0x7f358015b350 con 0x7f358006b370 2024-05-08T23:33:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3584d73700 1 -- 172.21.15.81:0/3117640022 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f356c0035d0 con 0x7f358006b370 2024-05-08T23:33:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3585574700 1 -- 172.21.15.81:0/3117640022 >> v1:172.21.15.117:6789/0 conn(0x7f358015a060 legacy=0x7f358006da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3585574700 1 -- 172.21.15.81:0/3117640022 >> v1:172.21.15.81:6789/0 conn(0x7f358006b370 legacy=0x7f35800b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3584d73700 1 -- 172.21.15.81:0/3117640022 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2802938321 0 0) 0x7f356c0035d0 con 0x7f358006b370 2024-05-08T23:33:08.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3585574700 1 -- 172.21.15.81:0/3117640022 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358006e220 con 0x7f358015aee0 2024-05-08T23:33:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3585574700 1 --2- 172.21.15.81:0/3117640022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f358015aee0 0x7f35800cc400 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3570009850 tx=0x7f3570009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72d33aaf48188342 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3584d73700 1 -- 172.21.15.81:0/3117640022 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570011830 con 0x7f358015aee0 2024-05-08T23:33:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3584d73700 1 -- 172.21.15.81:0/3117640022 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3570011990 con 0x7f358015aee0 2024-05-08T23:33:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3584d73700 1 -- 172.21.15.81:0/3117640022 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570011d40 con 0x7f358015aee0 2024-05-08T23:33:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3586d77700 1 -- 172.21.15.81:0/3117640022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f358015aee0 msgr2=0x7f35800cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3586d77700 1 --2- 172.21.15.81:0/3117640022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f358015aee0 0x7f35800cc400 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3570009850 tx=0x7f3570009b60 comp rx=0 tx=0).stop 2024-05-08T23:33:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3586d77700 1 -- 172.21.15.81:0/3117640022 shutdown_connections 2024-05-08T23:33:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3586d77700 1 --2- 172.21.15.81:0/3117640022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f358015aee0 0x7f35800cc400 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3586d77700 1 -- 172.21.15.81:0/3117640022 >> 172.21.15.81:0/3117640022 conn(0x7f35800708f0 msgr2=0x7f3580070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:08.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3586d77700 1 -- 172.21.15.81:0/3117640022 shutdown_connections 2024-05-08T23:33:08.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.566+0000 7f3586d77700 1 -- 172.21.15.81:0/3117640022 wait complete. 2024-05-08T23:33:08.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3586d77700 1 Processor -- start 2024-05-08T23:33:08.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3586d77700 1 -- start start 2024-05-08T23:33:08.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3586d77700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f358006b370 0x7f3580150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:08.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3586d77700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f358015a060 0x7f35800ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:08.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3586d77700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f358015aee0 0x7f35800ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:08.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3586d77700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35801f0d70 con 0x7f358006b370 2024-05-08T23:33:08.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3586d77700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35801f0a20 con 0x7f358015aee0 2024-05-08T23:33:08.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3586d77700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3580151730 con 0x7f358015a060 2024-05-08T23:33:08.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3585d75700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f358006b370 0x7f3580150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:08.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3585d75700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f358006b370 0x7f3580150dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36718/0 (socket says 172.21.15.81:36718) 2024-05-08T23:33:08.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3585d75700 1 -- 172.21.15.81:0/2671848932 learned_addr learned my addr 172.21.15.81:0/2671848932 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:08.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3586576700 1 --2- 172.21.15.81:0/2671848932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f358015aee0 0x7f35800ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:08.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3585574700 1 --2- 172.21.15.81:0/2671848932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f358015a060 0x7f35800ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:08.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3585d75700 1 -- 172.21.15.81:0/2671848932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f358015a060 msgr2=0x7f35800ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3585d75700 1 --2- 172.21.15.81:0/2671848932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f358015a060 0x7f35800ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3585d75700 1 -- 172.21.15.81:0/2671848932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f358015aee0 msgr2=0x7f35800ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3585d75700 1 --2- 172.21.15.81:0/2671848932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f358015aee0 0x7f35800ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.570+0000 7f3585d75700 1 -- 172.21.15.81:0/2671848932 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35700094b0 con 0x7f358006b370 2024-05-08T23:33:08.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.574+0000 7f3585d75700 1 --2- 172.21.15.81:0/2671848932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f358006b370 0x7f3580150dd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f357c00c500 tx=0x7f357c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:08.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.574+0000 7f35767fc700 1 -- 172.21.15.81:0/2671848932 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c0052d0 con 0x7f358006b370 2024-05-08T23:33:08.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.574+0000 7f35767fc700 1 -- 172.21.15.81:0/2671848932 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357c005430 con 0x7f358006b370 2024-05-08T23:33:08.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.574+0000 7f3586d77700 1 -- 172.21.15.81:0/2671848932 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3580151a10 con 0x7f358006b370 2024-05-08T23:33:08.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.574+0000 7f35767fc700 1 -- 172.21.15.81:0/2671848932 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c0055a0 con 0x7f358006b370 2024-05-08T23:33:08.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.574+0000 7f3586d77700 1 -- 172.21.15.81:0/2671848932 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3580151310 con 0x7f358006b370 2024-05-08T23:33:08.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.574+0000 7f3586d77700 1 -- 172.21.15.81:0/2671848932 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3580151500 con 0x7f358006b370 2024-05-08T23:33:08.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.574+0000 7f35767fc700 1 -- 172.21.15.81:0/2671848932 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f357c0038e0 con 0x7f358006b370 2024-05-08T23:33:08.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.578+0000 7f35767fc700 1 --2- 172.21.15.81:0/2671848932 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3564066730 0x7f3564068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:08.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.578+0000 7f35767fc700 1 -- 172.21.15.81:0/2671848932 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f357c083510 con 0x7f358006b370 2024-05-08T23:33:08.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.578+0000 7f3585574700 1 --2- 172.21.15.81:0/2671848932 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3564066730 0x7f3564068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:08.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.578+0000 7f3585574700 1 --2- 172.21.15.81:0/2671848932 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3564066730 0x7f3564068bf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f35700090b0 tx=0x7f3570008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:08.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.586+0000 7f35767fc700 1 -- 172.21.15.81:0/2671848932 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f357c050870 con 0x7f358006b370 2024-05-08T23:33:08.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.754+0000 7f3586d77700 1 -- 172.21.15.81:0/2671848932 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f35801f0b90 con 0x7f358006b370 2024-05-08T23:33:08.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.754+0000 7f35767fc700 1 -- 172.21.15.81:0/2671848932 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7f357c053e90 con 0x7f358006b370 2024-05-08T23:33:08.758 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:33:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.758+0000 7f3586d77700 1 -- 172.21.15.81:0/2671848932 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3564066730 msgr2=0x7f3564068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.758+0000 7f3586d77700 1 --2- 172.21.15.81:0/2671848932 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3564066730 0x7f3564068bf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f35700090b0 tx=0x7f3570008030 comp rx=0 tx=0).stop 2024-05-08T23:33:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.758+0000 7f3586d77700 1 -- 172.21.15.81:0/2671848932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f358006b370 msgr2=0x7f3580150dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.758+0000 7f3586d77700 1 --2- 172.21.15.81:0/2671848932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f358006b370 0x7f3580150dd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f357c00c500 tx=0x7f357c00c810 comp rx=0 tx=0).stop 2024-05-08T23:33:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.758+0000 7f3586d77700 1 -- 172.21.15.81:0/2671848932 shutdown_connections 2024-05-08T23:33:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.758+0000 7f3586d77700 1 --2- 172.21.15.81:0/2671848932 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f3564066730 0x7f3564068bf0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.758+0000 7f3586d77700 1 --2- 172.21.15.81:0/2671848932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f358015aee0 0x7f35800ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.758+0000 7f3586d77700 1 --2- 172.21.15.81:0/2671848932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f358015a060 0x7f35800ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.758+0000 7f3586d77700 1 --2- 172.21.15.81:0/2671848932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f358006b370 0x7f3580150dd0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.758+0000 7f3586d77700 1 -- 172.21.15.81:0/2671848932 >> 172.21.15.81:0/2671848932 conn(0x7f35800708f0 msgr2=0x7f35800cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.758+0000 7f3586d77700 1 -- 172.21.15.81:0/2671848932 shutdown_connections 2024-05-08T23:33:08.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.758+0000 7f3586d77700 1 -- 172.21.15.81:0/2671848932 wait complete. 2024-05-08T23:33:08.773 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:33:08.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.886+0000 7f9e4b59e700 1 Processor -- start 2024-05-08T23:33:08.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.886+0000 7f9e4b59e700 1 -- start start 2024-05-08T23:33:08.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.886+0000 7f9e4b59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9e4c0cf000 0x7f9e4c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:08.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4b59e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4c0cf460 con 0x7f9e4c15d660 2024-05-08T23:33:08.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4b59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4c15dad0 con 0x7f9e4c099ce0 2024-05-08T23:33:08.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4b59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4c0d03a0 con 0x7f9e4c0cf000 2024-05-08T23:33:08.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4ad9d700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f9e4c15d660 0x7f9e4c0cbf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34322/0 (socket says 172.21.15.81:34322) 2024-05-08T23:33:08.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4ad9d700 1 -- 172.21.15.81:0/861002644 learned_addr learned my addr 172.21.15.81:0/861002644 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:08.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e49d9b700 1 --2- 172.21.15.81:0/861002644 >> v2:172.21.15.117:3300/0 conn(0x7f9e4c0cf000 0x7f9e4c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:08.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4959a700 1 -- 172.21.15.81:0/861002644 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621590684 0 0) 0x7f9e4c0cf460 con 0x7f9e4c15d660 2024-05-08T23:33:08.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4959a700 1 -- 172.21.15.81:0/861002644 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e380035d0 con 0x7f9e4c15d660 2024-05-08T23:33:08.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e49d9b700 1 -- 172.21.15.81:0/861002644 >> v1:172.21.15.117:6789/0 conn(0x7f9e4c099ce0 legacy=0x7f9e4c09a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4959a700 1 -- 172.21.15.81:0/861002644 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2913352203 0 0) 0x7f9e380035d0 con 0x7f9e4c15d660 2024-05-08T23:33:08.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e49d9b700 1 -- 172.21.15.81:0/861002644 >> v1:172.21.15.81:6789/0 conn(0x7f9e4c15d660 legacy=0x7f9e4c0cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e49d9b700 1 -- 172.21.15.81:0/861002644 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c0d00c0 con 0x7f9e4c0cf000 2024-05-08T23:33:08.894 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e49d9b700 1 --2- 172.21.15.81:0/861002644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c0cf000 0x7f9e4c15d120 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009850 tx=0x7f9e3c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63cb29704eeff8fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:08.894 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4959a700 1 -- 172.21.15.81:0/861002644 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e440019f0 con 0x7f9e4c0cf000 2024-05-08T23:33:08.894 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4959a700 1 -- 172.21.15.81:0/861002644 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c004030 con 0x7f9e4c0cf000 2024-05-08T23:33:08.894 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4959a700 1 -- 172.21.15.81:0/861002644 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0043c0 con 0x7f9e4c0cf000 2024-05-08T23:33:08.894 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4b59e700 1 -- 172.21.15.81:0/861002644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c0cf000 msgr2=0x7f9e4c15d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.894 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4b59e700 1 --2- 172.21.15.81:0/861002644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c0cf000 0x7f9e4c15d120 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009850 tx=0x7f9e3c009b60 comp rx=0 tx=0).stop 2024-05-08T23:33:08.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4b59e700 1 -- 172.21.15.81:0/861002644 shutdown_connections 2024-05-08T23:33:08.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4b59e700 1 --2- 172.21.15.81:0/861002644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c0cf000 0x7f9e4c15d120 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4b59e700 1 -- 172.21.15.81:0/861002644 >> 172.21.15.81:0/861002644 conn(0x7f9e4c0d0dc0 msgr2=0x7f9e4c0d11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:08.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4b59e700 1 -- 172.21.15.81:0/861002644 shutdown_connections 2024-05-08T23:33:08.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.890+0000 7f9e4b59e700 1 -- 172.21.15.81:0/861002644 wait complete. 2024-05-08T23:33:08.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e4b59e700 1 Processor -- start 2024-05-08T23:33:08.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e4b59e700 1 -- start start 2024-05-08T23:33:08.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e4b59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e4c099ce0 0x7f9e4c1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:08.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e4b59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e4c0cf000 0x7f9e4c0cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:08.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e4b59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c15d660 0x7f9e4c0cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:08.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e4b59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c0b4450 con 0x7f9e4c0cf000 2024-05-08T23:33:08.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e4b59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c0c9260 con 0x7f9e4c15d660 2024-05-08T23:33:08.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e4b59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c15f8f0 con 0x7f9e4c099ce0 2024-05-08T23:33:08.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e49d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e4c0cf000 0x7f9e4c0cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:08.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e49d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e4c0cf000 0x7f9e4c0cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36722/0 (socket says 172.21.15.81:36722) 2024-05-08T23:33:08.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e49d9b700 1 -- 172.21.15.81:0/3643772007 learned_addr learned my addr 172.21.15.81:0/3643772007 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:08.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e4ad9d700 1 --2- 172.21.15.81:0/3643772007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c15d660 0x7f9e4c0cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:08.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e4a59c700 1 --2- 172.21.15.81:0/3643772007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e4c099ce0 0x7f9e4c1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:08.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e49d9b700 1 -- 172.21.15.81:0/3643772007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e4c099ce0 msgr2=0x7f9e4c1594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e49d9b700 1 --2- 172.21.15.81:0/3643772007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e4c099ce0 0x7f9e4c1594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e49d9b700 1 -- 172.21.15.81:0/3643772007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c15d660 msgr2=0x7f9e4c0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:08.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e49d9b700 1 --2- 172.21.15.81:0/3643772007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c15d660 0x7f9e4c0cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:08.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e49d9b700 1 -- 172.21.15.81:0/3643772007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0094b0 con 0x7f9e4c0cf000 2024-05-08T23:33:08.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e4ad9d700 1 --2- 172.21.15.81:0/3643772007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c15d660 0x7f9e4c0cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:08.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e49d9b700 1 --2- 172.21.15.81:0/3643772007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e4c0cf000 0x7f9e4c0cd640 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c071e00 tx=0x7f9e3c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:08.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e42ffd700 1 -- 172.21.15.81:0/3643772007 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c003e10 con 0x7f9e4c0cf000 2024-05-08T23:33:08.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e42ffd700 1 -- 172.21.15.81:0/3643772007 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c01f070 con 0x7f9e4c0cf000 2024-05-08T23:33:08.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e4b59e700 1 -- 172.21.15.81:0/3643772007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4c15fbd0 con 0x7f9e4c0cf000 2024-05-08T23:33:08.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e42ffd700 1 -- 172.21.15.81:0/3643772007 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c011870 con 0x7f9e4c0cf000 2024-05-08T23:33:08.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.894+0000 7f9e4b59e700 1 -- 172.21.15.81:0/3643772007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4c15ed10 con 0x7f9e4c0cf000 2024-05-08T23:33:08.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.898+0000 7f9e4b59e700 1 -- 172.21.15.81:0/3643772007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4c0b41d0 con 0x7f9e4c0cf000 2024-05-08T23:33:08.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.898+0000 7f9e42ffd700 1 -- 172.21.15.81:0/3643772007 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e3c021030 con 0x7f9e4c0cf000 2024-05-08T23:33:08.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.898+0000 7f9e42ffd700 1 --2- 172.21.15.81:0/3643772007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9e2c066730 0x7f9e2c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:08.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.898+0000 7f9e4a59c700 1 --2- 172.21.15.81:0/3643772007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9e2c066730 0x7f9e2c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:08.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.898+0000 7f9e42ffd700 1 -- 172.21.15.81:0/3643772007 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e3c0854b0 con 0x7f9e4c0cf000 2024-05-08T23:33:08.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.898+0000 7f9e4a59c700 1 --2- 172.21.15.81:0/3643772007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9e2c066730 0x7f9e2c068bf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c160d30 tx=0x7f9e34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:08.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:08.906+0000 7f9e42ffd700 1 -- 172.21.15.81:0/3643772007 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9e4c0b41d0 con 0x7f9e4c0cf000 2024-05-08T23:33:09.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.074+0000 7f9e4b59e700 1 -- 172.21.15.81:0/3643772007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e4c0b4590 con 0x7f9e4c0cf000 2024-05-08T23:33:09.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.078+0000 7f9e42ffd700 1 -- 172.21.15.81:0/3643772007 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e4c0b4590 con 0x7f9e4c0cf000 2024-05-08T23:33:09.079 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:33:09.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.078+0000 7f9e4b59e700 1 -- 172.21.15.81:0/3643772007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9e2c066730 msgr2=0x7f9e2c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:09.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.078+0000 7f9e4b59e700 1 --2- 172.21.15.81:0/3643772007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9e2c066730 0x7f9e2c068bf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c160d30 tx=0x7f9e34009040 comp rx=0 tx=0).stop 2024-05-08T23:33:09.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.078+0000 7f9e4b59e700 1 -- 172.21.15.81:0/3643772007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e4c0cf000 msgr2=0x7f9e4c0cd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:09.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.078+0000 7f9e4b59e700 1 --2- 172.21.15.81:0/3643772007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e4c0cf000 0x7f9e4c0cd640 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c071e00 tx=0x7f9e3c004770 comp rx=0 tx=0).stop 2024-05-08T23:33:09.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.078+0000 7f9e4b59e700 1 -- 172.21.15.81:0/3643772007 shutdown_connections 2024-05-08T23:33:09.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.078+0000 7f9e4b59e700 1 --2- 172.21.15.81:0/3643772007 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9e2c066730 0x7f9e2c068bf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.078+0000 7f9e4b59e700 1 --2- 172.21.15.81:0/3643772007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c15d660 0x7f9e4c0cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.078+0000 7f9e4b59e700 1 --2- 172.21.15.81:0/3643772007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e4c0cf000 0x7f9e4c0cd640 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.078+0000 7f9e4b59e700 1 --2- 172.21.15.81:0/3643772007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e4c099ce0 0x7f9e4c1594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.078+0000 7f9e4b59e700 1 -- 172.21.15.81:0/3643772007 >> 172.21.15.81:0/3643772007 conn(0x7f9e4c0d0dc0 msgr2=0x7f9e4c15b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:09.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.078+0000 7f9e4b59e700 1 -- 172.21.15.81:0/3643772007 shutdown_connections 2024-05-08T23:33:09.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.078+0000 7f9e4b59e700 1 -- 172.21.15.81:0/3643772007 wait complete. 2024-05-08T23:33:09.094 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:33:09.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.202+0000 7fddc3ce6700 1 Processor -- start 2024-05-08T23:33:09.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.202+0000 7fddc3ce6700 1 -- start start 2024-05-08T23:33:09.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc3ce6700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fddbc0b3300 0x7fddbc0b3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:09.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc3ce6700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fddbc071110 0x7fddbc0715a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:09.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc3ce6700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddbc0b2e10 con 0x7fddbc146b30 2024-05-08T23:33:09.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc3ce6700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddbc146f10 con 0x7fddbc0b3300 2024-05-08T23:33:09.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc3ce6700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddbc0b3cb0 con 0x7fddbc071110 2024-05-08T23:33:09.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc1281700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fddbc0b3300 0x7fddbc0b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:09.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc1a82700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fddbc146b30 0x7fddbc0b2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34326/0 (socket says 172.21.15.81:34326) 2024-05-08T23:33:09.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc1a82700 1 -- 172.21.15.81:0/2939494808 learned_addr learned my addr 172.21.15.81:0/2939494808 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:09.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc2283700 1 --2- 172.21.15.81:0/2939494808 >> v2:172.21.15.117:3300/0 conn(0x7fddbc071110 0x7fddbc0715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:09.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc1281700 1 -- 172.21.15.81:0/2939494808 >> v2:172.21.15.117:3300/0 conn(0x7fddbc071110 msgr2=0x7fddbc0715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:09.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc1281700 1 --2- 172.21.15.81:0/2939494808 >> v2:172.21.15.117:3300/0 conn(0x7fddbc071110 0x7fddbc0715a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc0a80700 1 -- 172.21.15.81:0/2939494808 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49698402 0 0) 0x7fddbc0b2e10 con 0x7fddbc146b30 2024-05-08T23:33:09.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc1281700 1 -- 172.21.15.81:0/2939494808 >> v1:172.21.15.81:6789/0 conn(0x7fddbc146b30 legacy=0x7fddbc0b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:09.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc1281700 1 -- 172.21.15.81:0/2939494808 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddbc160ea0 con 0x7fddbc0b3300 2024-05-08T23:33:09.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc1281700 1 --2- 172.21.15.81:0/2939494808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc0b3300 0x7fddbc0b3770 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fddbc070140 tx=0x7fddb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5875af73cb8718 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:09.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc0a80700 1 -- 172.21.15.81:0/2939494808 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb000e070 con 0x7fddbc0b3300 2024-05-08T23:33:09.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc0a80700 1 -- 172.21.15.81:0/2939494808 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb000bb20 con 0x7fddbc0b3300 2024-05-08T23:33:09.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc0a80700 1 -- 172.21.15.81:0/2939494808 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0012430 con 0x7fddbc0b3300 2024-05-08T23:33:09.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc3ce6700 1 -- 172.21.15.81:0/2939494808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc0b3300 msgr2=0x7fddbc0b3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:09.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc3ce6700 1 --2- 172.21.15.81:0/2939494808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc0b3300 0x7fddbc0b3770 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fddbc070140 tx=0x7fddb000b330 comp rx=0 tx=0).stop 2024-05-08T23:33:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc3ce6700 1 -- 172.21.15.81:0/2939494808 shutdown_connections 2024-05-08T23:33:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc3ce6700 1 --2- 172.21.15.81:0/2939494808 >> v2:172.21.15.117:3300/0 conn(0x7fddbc071110 0x7fddbc0715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc3ce6700 1 --2- 172.21.15.81:0/2939494808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc0b3300 0x7fddbc0b3770 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.206+0000 7fddc3ce6700 1 -- 172.21.15.81:0/2939494808 >> 172.21.15.81:0/2939494808 conn(0x7fddbc06a360 msgr2=0x7fddbc06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc3ce6700 1 -- 172.21.15.81:0/2939494808 shutdown_connections 2024-05-08T23:33:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc3ce6700 1 -- 172.21.15.81:0/2939494808 wait complete. 2024-05-08T23:33:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc3ce6700 1 Processor -- start 2024-05-08T23:33:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc3ce6700 1 -- start start 2024-05-08T23:33:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc3ce6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc071110 0x7fddbc13b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc3ce6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddbc0b3300 0x7fddbc157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc3ce6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddbc146b30 0x7fddbc1584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc3ce6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc145270 con 0x7fddbc071110 2024-05-08T23:33:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc3ce6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc144680 con 0x7fddbc0b3300 2024-05-08T23:33:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc3ce6700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc144940 con 0x7fddbc146b30 2024-05-08T23:33:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc1a82700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc071110 0x7fddbc13b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc1a82700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc071110 0x7fddbc13b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36748/0 (socket says 172.21.15.81:36748) 2024-05-08T23:33:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc1a82700 1 -- 172.21.15.81:0/381841481 learned_addr learned my addr 172.21.15.81:0/381841481 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc1281700 1 --2- 172.21.15.81:0/381841481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddbc0b3300 0x7fddbc157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc2283700 1 --2- 172.21.15.81:0/381841481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddbc146b30 0x7fddbc1584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc1a82700 1 -- 172.21.15.81:0/381841481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddbc146b30 msgr2=0x7fddbc1584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc1a82700 1 --2- 172.21.15.81:0/381841481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddbc146b30 0x7fddbc1584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc1a82700 1 -- 172.21.15.81:0/381841481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddbc0b3300 msgr2=0x7fddbc157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc1a82700 1 --2- 172.21.15.81:0/381841481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddbc0b3300 0x7fddbc157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc1a82700 1 -- 172.21.15.81:0/381841481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb0009a70 con 0x7fddbc071110 2024-05-08T23:33:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc1281700 1 --2- 172.21.15.81:0/381841481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddbc0b3300 0x7fddbc157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc1a82700 1 --2- 172.21.15.81:0/381841481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc071110 0x7fddbc13b860 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c4e0 tx=0x7fddb800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddae7fc700 1 -- 172.21.15.81:0/381841481 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8004ca0 con 0x7fddbc071110 2024-05-08T23:33:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc3ce6700 1 -- 172.21.15.81:0/381841481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddbc144c20 con 0x7fddbc071110 2024-05-08T23:33:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddae7fc700 1 -- 172.21.15.81:0/381841481 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb8004e20 con 0x7fddbc071110 2024-05-08T23:33:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddc3ce6700 1 -- 172.21.15.81:0/381841481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddbc13bda0 con 0x7fddbc071110 2024-05-08T23:33:09.218 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.210+0000 7fddae7fc700 1 -- 172.21.15.81:0/381841481 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8007310 con 0x7fddbc071110 2024-05-08T23:33:09.218 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.214+0000 7fddc3ce6700 1 -- 172.21.15.81:0/381841481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddbc13bff0 con 0x7fddbc071110 2024-05-08T23:33:09.218 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.214+0000 7fddae7fc700 1 -- 172.21.15.81:0/381841481 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fddb8007470 con 0x7fddbc071110 2024-05-08T23:33:09.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.214+0000 7fddae7fc700 1 --2- 172.21.15.81:0/381841481 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdda0066730 0x7fdda0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:09.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.214+0000 7fddae7fc700 1 -- 172.21.15.81:0/381841481 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddb8083f90 con 0x7fddbc071110 2024-05-08T23:33:09.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.214+0000 7fddc1281700 1 --2- 172.21.15.81:0/381841481 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdda0066730 0x7fdda0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:09.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.218+0000 7fddc1281700 1 --2- 172.21.15.81:0/381841481 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdda0066730 0x7fdda0068bf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fddb0014040 tx=0x7fddb0011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:09.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.218+0000 7fddae7fc700 1 -- 172.21.15.81:0/381841481 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fddb8051150 con 0x7fddbc071110 2024-05-08T23:33:09.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.402+0000 7fddc3ce6700 1 -- 172.21.15.81:0/381841481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fddbc0684b0 con 0x7fddbc071110 2024-05-08T23:33:09.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.402+0000 7fddae7fc700 1 -- 172.21.15.81:0/381841481 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) v1 ==== 80+0+0 (secure 0 0 0) 0x7fddb8007720 con 0x7fddbc071110 2024-05-08T23:33:09.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.406+0000 7fddc3ce6700 1 -- 172.21.15.81:0/381841481 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdda0066730 msgr2=0x7fdda0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:09.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.406+0000 7fddc3ce6700 1 --2- 172.21.15.81:0/381841481 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdda0066730 0x7fdda0068bf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fddb0014040 tx=0x7fddb0011000 comp rx=0 tx=0).stop 2024-05-08T23:33:09.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.406+0000 7fddc3ce6700 1 -- 172.21.15.81:0/381841481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc071110 msgr2=0x7fddbc13b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:09.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.406+0000 7fddc3ce6700 1 --2- 172.21.15.81:0/381841481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc071110 0x7fddbc13b860 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c4e0 tx=0x7fddb800c7f0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.406+0000 7fddc3ce6700 1 -- 172.21.15.81:0/381841481 shutdown_connections 2024-05-08T23:33:09.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.406+0000 7fddc3ce6700 1 --2- 172.21.15.81:0/381841481 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdda0066730 0x7fdda0068bf0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.406+0000 7fddc3ce6700 1 --2- 172.21.15.81:0/381841481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddbc146b30 0x7fddbc1584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.406+0000 7fddc3ce6700 1 --2- 172.21.15.81:0/381841481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddbc0b3300 0x7fddbc157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.406+0000 7fddc3ce6700 1 --2- 172.21.15.81:0/381841481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fddbc071110 0x7fddbc13b860 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.406+0000 7fddc3ce6700 1 -- 172.21.15.81:0/381841481 >> 172.21.15.81:0/381841481 conn(0x7fddbc06a360 msgr2=0x7fddbc141230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:09.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.406+0000 7fddc3ce6700 1 -- 172.21.15.81:0/381841481 shutdown_connections 2024-05-08T23:33:09.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.406+0000 7fddc3ce6700 1 -- 172.21.15.81:0/381841481 wait complete. 2024-05-08T23:33:09.410 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:33:09.421 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2024-05-08T23:33:09.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:09.421 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T23:33:09.434 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[39609]: fuse finished with error 0 and tester_r 0 2024-05-08T23:33:09.436 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:33:09.514 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:33:09.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:09.514 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:33:09.523 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:33:09.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.638+0000 7f10a22d4700 1 Processor -- start 2024-05-08T23:33:09.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.638+0000 7f10a22d4700 1 -- start start 2024-05-08T23:33:09.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a22d4700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f109c071ab0 0x7f109c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:09.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a22d4700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109c0cea50 con 0x7f109c15ac70 2024-05-08T23:33:09.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a22d4700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109c0ce080 con 0x7f109c071e90 2024-05-08T23:33:09.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a22d4700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109c0cd7c0 con 0x7f109c071ab0 2024-05-08T23:33:09.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a12d2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f109c071ab0 0x7f109c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:09.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a1ad3700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f109c15ac70 0x7f109c13d500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34336/0 (socket says 172.21.15.81:34336) 2024-05-08T23:33:09.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a1ad3700 1 -- 172.21.15.81:0/3199793865 learned_addr learned my addr 172.21.15.81:0/3199793865 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:09.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a12d2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f109c071ab0 0x7f109c0ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36756/0 (socket says 172.21.15.81:36756) 2024-05-08T23:33:09.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f108ffff700 1 -- 172.21.15.81:0/3199793865 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3486197827 0 0) 0x7f109c0ce080 con 0x7f109c071e90 2024-05-08T23:33:09.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f108ffff700 1 -- 172.21.15.81:0/3199793865 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10880035d0 con 0x7f109c071e90 2024-05-08T23:33:09.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f108ffff700 1 -- 172.21.15.81:0/3199793865 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838459836 0 0) 0x7f1094001110 con 0x7f109c15ac70 2024-05-08T23:33:09.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a12d2700 1 -- 172.21.15.81:0/3199793865 >> v1:172.21.15.117:6789/0 conn(0x7f109c071e90 legacy=0x7f109c15a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:09.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a12d2700 1 -- 172.21.15.81:0/3199793865 >> v1:172.21.15.81:6789/0 conn(0x7f109c15ac70 legacy=0x7f109c13d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:09.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a12d2700 1 -- 172.21.15.81:0/3199793865 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c0cda30 con 0x7f109c071ab0 2024-05-08T23:33:09.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a12d2700 1 --2- 172.21.15.81:0/3199793865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c071ab0 0x7f109c0ce510 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1090009850 tx=0x7f1090009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d3fa0dd63bc4f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:09.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f108ffff700 1 -- 172.21.15.81:0/3199793865 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090011830 con 0x7f109c071ab0 2024-05-08T23:33:09.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f108ffff700 1 -- 172.21.15.81:0/3199793865 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1090011990 con 0x7f109c071ab0 2024-05-08T23:33:09.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f108ffff700 1 -- 172.21.15.81:0/3199793865 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090011d00 con 0x7f109c071ab0 2024-05-08T23:33:09.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a22d4700 1 -- 172.21.15.81:0/3199793865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c071ab0 msgr2=0x7f109c0ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:09.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a22d4700 1 --2- 172.21.15.81:0/3199793865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c071ab0 0x7f109c0ce510 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1090009850 tx=0x7f1090009b60 comp rx=0 tx=0).stop 2024-05-08T23:33:09.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a22d4700 1 -- 172.21.15.81:0/3199793865 shutdown_connections 2024-05-08T23:33:09.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a22d4700 1 --2- 172.21.15.81:0/3199793865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c071ab0 0x7f109c0ce510 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a22d4700 1 -- 172.21.15.81:0/3199793865 >> 172.21.15.81:0/3199793865 conn(0x7f109c06c330 msgr2=0x7f109c06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:09.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a22d4700 1 -- 172.21.15.81:0/3199793865 shutdown_connections 2024-05-08T23:33:09.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.642+0000 7f10a22d4700 1 -- 172.21.15.81:0/3199793865 wait complete. 2024-05-08T23:33:09.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.646+0000 7f10a22d4700 1 Processor -- start 2024-05-08T23:33:09.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.646+0000 7f10a22d4700 1 -- start start 2024-05-08T23:33:09.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.646+0000 7f10a22d4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c071ab0 0x7f109c153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:09.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.646+0000 7f10a22d4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f109c071e90 0x7f109c0b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:09.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.646+0000 7f10a22d4700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f109c15ac70 0x7f109c0b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:09.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.646+0000 7f10a22d4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0b3ea0 con 0x7f109c071ab0 2024-05-08T23:33:09.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.646+0000 7f10a22d4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0b38e0 con 0x7f109c071e90 2024-05-08T23:33:09.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.646+0000 7f10a22d4700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0b3bc0 con 0x7f109c15ac70 2024-05-08T23:33:09.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.646+0000 7f10a12d2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c071ab0 0x7f109c153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:09.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.646+0000 7f10a12d2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c071ab0 0x7f109c153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36770/0 (socket says 172.21.15.81:36770) 2024-05-08T23:33:09.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.646+0000 7f10a12d2700 1 -- 172.21.15.81:0/496778468 learned_addr learned my addr 172.21.15.81:0/496778468 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:09.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.646+0000 7f10a0ad1700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f109c071e90 0x7f109c0b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:09.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.646+0000 7f10a1ad3700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f109c15ac70 0x7f109c0b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:09.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.650+0000 7f10a12d2700 1 -- 172.21.15.81:0/496778468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f109c15ac70 msgr2=0x7f109c0b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:09.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.650+0000 7f10a12d2700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f109c15ac70 0x7f109c0b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.650+0000 7f10a12d2700 1 -- 172.21.15.81:0/496778468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f109c071e90 msgr2=0x7f109c0b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:09.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.650+0000 7f10a12d2700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f109c071e90 0x7f109c0b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.650+0000 7f10a12d2700 1 -- 172.21.15.81:0/496778468 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10900094b0 con 0x7f109c071ab0 2024-05-08T23:33:09.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.650+0000 7f10a1ad3700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f109c15ac70 0x7f109c0b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:09.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.650+0000 7f10a0ad1700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f109c071e90 0x7f109c0b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:09.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.650+0000 7f10a12d2700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c071ab0 0x7f109c153540 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f10900090b0 tx=0x7f1090004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:09.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.650+0000 7f108dffb700 1 -- 172.21.15.81:0/496778468 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090004570 con 0x7f109c071ab0 2024-05-08T23:33:09.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.650+0000 7f108dffb700 1 -- 172.21.15.81:0/496778468 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109001f070 con 0x7f109c071ab0 2024-05-08T23:33:09.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.650+0000 7f10a22d4700 1 -- 172.21.15.81:0/496778468 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c140800 con 0x7f109c071ab0 2024-05-08T23:33:09.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.650+0000 7f108dffb700 1 -- 172.21.15.81:0/496778468 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090003900 con 0x7f109c071ab0 2024-05-08T23:33:09.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.650+0000 7f10a22d4700 1 -- 172.21.15.81:0/496778468 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c0b34e0 con 0x7f109c071ab0 2024-05-08T23:33:09.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.650+0000 7f10a22d4700 1 -- 172.21.15.81:0/496778468 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109c140660 con 0x7f109c071ab0 2024-05-08T23:33:09.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.650+0000 7f108dffb700 1 -- 172.21.15.81:0/496778468 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1090021030 con 0x7f109c071ab0 2024-05-08T23:33:09.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.654+0000 7f108dffb700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1080066730 0x7f1080068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:09.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.654+0000 7f108dffb700 1 -- 172.21.15.81:0/496778468 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10900856c0 con 0x7f109c071ab0 2024-05-08T23:33:09.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.654+0000 7f10a0ad1700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1080066730 0x7f1080068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:09.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.654+0000 7f10a0ad1700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1080066730 0x7f1080068bf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f109c0b0bc0 tx=0x7f1098009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:09.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.662+0000 7f108dffb700 1 -- 172.21.15.81:0/496778468 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f109c140660 con 0x7f109c071ab0 2024-05-08T23:33:09.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.870+0000 7f10a22d4700 1 -- 172.21.15.81:0/496778468 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f109c0b3a20 con 0x7f109c071ab0 2024-05-08T23:33:09.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.870+0000 7f108dffb700 1 -- 172.21.15.81:0/496778468 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f109c0b3a20 con 0x7f109c071ab0 2024-05-08T23:33:09.873 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:33:09.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.874+0000 7f10a22d4700 1 -- 172.21.15.81:0/496778468 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1080066730 msgr2=0x7f1080068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:09.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.874+0000 7f10a22d4700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1080066730 0x7f1080068bf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f109c0b0bc0 tx=0x7f1098009040 comp rx=0 tx=0).stop 2024-05-08T23:33:09.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.874+0000 7f10a22d4700 1 -- 172.21.15.81:0/496778468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c071ab0 msgr2=0x7f109c153540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:09.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.874+0000 7f10a22d4700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c071ab0 0x7f109c153540 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f10900090b0 tx=0x7f1090004770 comp rx=0 tx=0).stop 2024-05-08T23:33:09.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.874+0000 7f10a22d4700 1 -- 172.21.15.81:0/496778468 shutdown_connections 2024-05-08T23:33:09.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.874+0000 7f10a22d4700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1080066730 0x7f1080068bf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.874+0000 7f10a22d4700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f109c15ac70 0x7f109c0b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.874+0000 7f10a22d4700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f109c071e90 0x7f109c0b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.876 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.874+0000 7f10a22d4700 1 --2- 172.21.15.81:0/496778468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c071ab0 0x7f109c153540 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:09.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.874+0000 7f10a22d4700 1 -- 172.21.15.81:0/496778468 >> 172.21.15.81:0/496778468 conn(0x7f109c06c330 msgr2=0x7f109c06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:09.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.874+0000 7f10a22d4700 1 -- 172.21.15.81:0/496778468 shutdown_connections 2024-05-08T23:33:09.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:09.874+0000 7f10a22d4700 1 -- 172.21.15.81:0/496778468 wait complete. 2024-05-08T23:33:09.888 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:33:09.888 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:33:09.888 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2024-05-08T23:33:09.888 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:33:09.888 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:33:09.888 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:33:09.889 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:33:09.889 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:33:09.889 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:33:09.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:09.889 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2024-05-08T23:33:09.896 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:33:09.897 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:33:09.897 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:33:09.897 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:09.897 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-05-08T23:33:09.897 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:09.897 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:33:09.897 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:33:09.897 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:33:09.897 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:33:09.897 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-08T23:33:09.897 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:33:09.898 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:33:09.898 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:33:09.898 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T23:33:09.898 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42426sec preferred_lft 42426sec 2024-05-08T23:33:09.898 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2024-05-08T23:33:09.898 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:09.898 INFO:teuthology.orchestra.run.smithi081.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-08T23:33:09.899 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 02:42:22:62:7a:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T23:33:09.899 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-08T23:33:09.899 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:09.899 INFO:teuthology.orchestra.run.smithi081.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T23:33:09.899 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:2b:10:6b:08:e3 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:33:09.899 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T23:33:09.899 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:09.899 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::48fc:c7ff:fe41:6d94/64 scope link 2024-05-08T23:33:09.899 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:09.899 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:33:09.899 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:2b:10:6b:08:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:33:09.899 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::702b:10ff:fe6b:8e3/64 scope link 2024-05-08T23:33:09.899 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:09.899 INFO:teuthology.orchestra.run.smithi081.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T23:33:09.900 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 72:9d:1b:4e:6b:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:33:09.900 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::709d:1bff:fe4e:6bd5/64 scope link 2024-05-08T23:33:09.900 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:33:09.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:09.900 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:33:09.912 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-05-08T23:33:09.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:09.914 DEBUG:teuthology.orchestra.run.smithi081:> route 2024-05-08T23:33:09.963 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2024-05-08T23:33:09.964 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:33:09.964 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-08T23:33:09.964 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T23:33:09.964 INFO:teuthology.orchestra.run.smithi081.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-08T23:33:09.964 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-05-08T23:33:09.964 INFO:teuthology.orchestra.run.smithi081.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-08T23:33:09.964 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:33:09.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:09.964 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-05-08T23:33:09.965 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-05-08T23:33:09.965 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T23:33:09.965 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T23:33:09.965 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T23:33:09.965 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-05-08T23:33:10.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:10.003 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-05-08T23:33:10.010 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T23:33:10.010 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:33:10.011 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:33:10.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:10.011 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2024-05-08T23:33:10.025 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-08T23:33:10.025 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:33:10.025 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:33:10.073 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:33:10.074 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:33:10.074 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:33:10.081 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:33:10.094 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:33:10.094 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:33:10.150 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:33:10.151 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:33:10.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:10.152 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:33:10.156 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T23:33:10.157 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-08T23:33:10.159 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:33:10.177 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:33:10.177 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:33:10.232 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:33:10.233 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:33:10.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:10.233 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:33:10.251 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2024-05-08T23:33:10.246+0000 7f4c84f3a640 -1 init, newargv = 0x55bdd465da70 newargc=15 2024-05-08T23:33:10.251 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[40026]: starting ceph client 2024-05-08T23:33:10.259 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[40026]: starting fuse 2024-05-08T23:33:11.339 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2024-05-08T23:33:11.356 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:33:11.356 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:33:11.414 INFO:teuthology.orchestra.run.smithi081.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:33:11.416 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:33:11.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:11.417 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2024-05-08T23:33:11.423 INFO:teuthology.orchestra.run.smithi081.stdout:52 2024-05-08T23:33:11.423 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> import re 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> import os 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2024-05-08T23:33:11.424 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:33:11.425 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2024-05-08T23:33:11.425 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T23:33:11.425 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T23:33:11.425 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T23:33:11.425 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2024-05-08T23:33:11.425 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2024-05-08T23:33:11.425 DEBUG:teuthology.orchestra.run.smithi081:> return f 2024-05-08T23:33:11.425 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T23:33:11.425 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:33:11.425 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2024-05-08T23:33:11.425 DEBUG:teuthology.orchestra.run.smithi081:> ' 2024-05-08T23:33:11.508 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.40026.asok 2024-05-08T23:33:11.693 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40026.asok 2024-05-08T23:33:11.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:11.694 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40026.asok status 2024-05-08T23:33:11.793 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:33:11.793 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2024-05-08T23:33:11.793 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T23:33:11.793 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:33:11.793 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2024-05-08T23:33:11.793 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2024-05-08T23:33:11.793 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T23:33:11.793 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "40026", 2024-05-08T23:33:11.793 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2024-05-08T23:33:11.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:33:11.794 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2024-05-08T23:33:11.794 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2024-05-08T23:33:11.794 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 6273, 2024-05-08T23:33:11.794 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2024-05-08T23:33:11.794 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2024-05-08T23:33:11.794 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2024-05-08T23:33:11.794 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 6273 2024-05-08T23:33:11.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:33:11.794 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:33:11.794 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:33:11.794 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:33:11.794 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 651161206 2024-05-08T23:33:11.794 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-08T23:33:11.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:33:11.795 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2024-05-08T23:33:11.795 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-05-08T23:33:11.795 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:33:11.795 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 651161206 2024-05-08T23:33:11.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-08T23:33:11.795 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.6273 192.168.0.1:0/651161206", 2024-05-08T23:33:11.795 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/651161206", 2024-05-08T23:33:11.795 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2024-05-08T23:33:11.795 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 23, 2024-05-08T23:33:11.795 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 52, 2024-05-08T23:33:11.795 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2024-05-08T23:33:11.795 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2024-05-08T23:33:11.795 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2024-05-08T23:33:11.795 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:33:11.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:11.803 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2024-05-08T23:33:11.803 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T23:33:11.812 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2024-05-08T23:33:11.812 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T23:33:11.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:11.812 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:33:11.827 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:33:11.827 DEBUG:teuthology.orchestra.run.smithi081:> import os 2024-05-08T23:33:11.827 DEBUG:teuthology.orchestra.run.smithi081:> import errno 2024-05-08T23:33:11.827 DEBUG:teuthology.orchestra.run.smithi081:> 2024-05-08T23:33:11.827 DEBUG:teuthology.orchestra.run.smithi081:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2024-05-08T23:33:11.827 DEBUG:teuthology.orchestra.run.smithi081:> try: 2024-05-08T23:33:11.827 DEBUG:teuthology.orchestra.run.smithi081:> if True: 2024-05-08T23:33:11.827 DEBUG:teuthology.orchestra.run.smithi081:> ret = os.read(fd, 1024) 2024-05-08T23:33:11.827 DEBUG:teuthology.orchestra.run.smithi081:> else: 2024-05-08T23:33:11.827 DEBUG:teuthology.orchestra.run.smithi081:> os.write(fd, b'"'"'content'"'"') 2024-05-08T23:33:11.827 DEBUG:teuthology.orchestra.run.smithi081:> except OSError as e: 2024-05-08T23:33:11.827 DEBUG:teuthology.orchestra.run.smithi081:> if e.errno != errno.EPERM: 2024-05-08T23:33:11.827 DEBUG:teuthology.orchestra.run.smithi081:> raise 2024-05-08T23:33:11.827 DEBUG:teuthology.orchestra.run.smithi081:> else: 2024-05-08T23:33:11.828 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("client does not check permission of data pool") 2024-05-08T23:33:11.828 DEBUG:teuthology.orchestra.run.smithi081:> ' 2024-05-08T23:33:12.085 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2024-05-08T23:33:12.218 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.214+0000 7f8a9f0c9700 1 Processor -- start 2024-05-08T23:33:12.218 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.214+0000 7f8a9f0c9700 1 -- start start 2024-05-08T23:33:12.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9f0c9700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900dfe00 0x7f8a90050ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:12.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9f0c9700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a90054a90 con 0x7f8a900e01e0 2024-05-08T23:33:12.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9f0c9700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a90054d70 con 0x7f8a900505c0 2024-05-08T23:33:12.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9f0c9700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90055b30 con 0x7f8a900dfe00 2024-05-08T23:33:12.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9e8c8700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f8a900e01e0 0x7f8a90008ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34340/0 (socket says 172.21.15.81:34340) 2024-05-08T23:33:12.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9e8c8700 1 -- 172.21.15.81:0/1440047824 learned_addr learned my addr 172.21.15.81:0/1440047824 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:12.221 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9e0c7700 1 --2- 172.21.15.81:0/1440047824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900dfe00 0x7f8a90050ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9d0c5700 1 -- 172.21.15.81:0/1440047824 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224387765 0 0) 0x7f8a90054a90 con 0x7f8a900e01e0 2024-05-08T23:33:12.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9d0c5700 1 -- 172.21.15.81:0/1440047824 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a840035d0 con 0x7f8a900e01e0 2024-05-08T23:33:12.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9d0c5700 1 -- 172.21.15.81:0/1440047824 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4274340531 0 0) 0x7f8a90054d70 con 0x7f8a900505c0 2024-05-08T23:33:12.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9d0c5700 1 -- 172.21.15.81:0/1440047824 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a90054a90 con 0x7f8a900505c0 2024-05-08T23:33:12.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9e0c7700 1 -- 172.21.15.81:0/1440047824 >> v1:172.21.15.117:6789/0 conn(0x7f8a900505c0 legacy=0x7f8a90038eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:12.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9e0c7700 1 -- 172.21.15.81:0/1440047824 >> v1:172.21.15.81:6789/0 conn(0x7f8a900e01e0 legacy=0x7f8a90008ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:12.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9d0c5700 1 -- 172.21.15.81:0/1440047824 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 609257460 0 0) 0x7f8a840035d0 con 0x7f8a900e01e0 2024-05-08T23:33:12.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9e0c7700 1 -- 172.21.15.81:0/1440047824 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a90055050 con 0x7f8a900dfe00 2024-05-08T23:33:12.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9e0c7700 1 --2- 172.21.15.81:0/1440047824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900dfe00 0x7f8a90050ca0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009880 tx=0x7f8a8c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f055e6056d95ced server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:12.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9d0c5700 1 -- 172.21.15.81:0/1440047824 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9804ec20 con 0x7f8a900dfe00 2024-05-08T23:33:12.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9d0c5700 1 -- 172.21.15.81:0/1440047824 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c004030 con 0x7f8a900dfe00 2024-05-08T23:33:12.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.218+0000 7f8a9d0c5700 1 -- 172.21.15.81:0/1440047824 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c0043c0 con 0x7f8a900dfe00 2024-05-08T23:33:12.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.222+0000 7f8a9f0c9700 1 -- 172.21.15.81:0/1440047824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900dfe00 msgr2=0x7f8a90050ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:12.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.222+0000 7f8a9f0c9700 1 --2- 172.21.15.81:0/1440047824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900dfe00 0x7f8a90050ca0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009880 tx=0x7f8a8c009b90 comp rx=0 tx=0).stop 2024-05-08T23:33:12.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.222+0000 7f8a9f0c9700 1 -- 172.21.15.81:0/1440047824 shutdown_connections 2024-05-08T23:33:12.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.222+0000 7f8a9f0c9700 1 --2- 172.21.15.81:0/1440047824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900dfe00 0x7f8a90050ca0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.222+0000 7f8a9f0c9700 1 -- 172.21.15.81:0/1440047824 >> 172.21.15.81:0/1440047824 conn(0x7f8a90055c80 msgr2=0x7f8a90056090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:12.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.222+0000 7f8a9f0c9700 1 -- 172.21.15.81:0/1440047824 shutdown_connections 2024-05-08T23:33:12.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.222+0000 7f8a9f0c9700 1 -- 172.21.15.81:0/1440047824 wait complete. 2024-05-08T23:33:12.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.222+0000 7f8a9f0c9700 1 Processor -- start 2024-05-08T23:33:12.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.222+0000 7f8a9f0c9700 1 -- start start 2024-05-08T23:33:12.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9f0c9700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900505c0 0x7f8a9004d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:12.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9f0c9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a900dfe00 0x7f8a9004cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:12.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9f0c9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a900e01e0 0x7f8a900d91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:12.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9f0c9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a901754b0 con 0x7f8a900dfe00 2024-05-08T23:33:12.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9f0c9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a900d9700 con 0x7f8a900e01e0 2024-05-08T23:33:12.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9f0c9700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9004dca0 con 0x7f8a900505c0 2024-05-08T23:33:12.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9d8c6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a900dfe00 0x7f8a9004cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9d8c6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a900dfe00 0x7f8a9004cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36782/0 (socket says 172.21.15.81:36782) 2024-05-08T23:33:12.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9d8c6700 1 -- 172.21.15.81:0/3164796656 learned_addr learned my addr 172.21.15.81:0/3164796656 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:12.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9e8c8700 1 --2- 172.21.15.81:0/3164796656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a900e01e0 0x7f8a900d91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9e0c7700 1 --2- 172.21.15.81:0/3164796656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900505c0 0x7f8a9004d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9d8c6700 1 -- 172.21.15.81:0/3164796656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900505c0 msgr2=0x7f8a9004d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:12.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9d8c6700 1 --2- 172.21.15.81:0/3164796656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900505c0 0x7f8a9004d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9d8c6700 1 -- 172.21.15.81:0/3164796656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a900e01e0 msgr2=0x7f8a900d91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:12.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9d8c6700 1 --2- 172.21.15.81:0/3164796656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a900e01e0 0x7f8a900d91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9d8c6700 1 -- 172.21.15.81:0/3164796656 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8c0094b0 con 0x7f8a900dfe00 2024-05-08T23:33:12.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9d8c6700 1 --2- 172.21.15.81:0/3164796656 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a900dfe00 0x7f8a9004cd50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400c520 tx=0x7f8a9400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:12.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a8affd700 1 -- 172.21.15.81:0/3164796656 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a940054e0 con 0x7f8a900dfe00 2024-05-08T23:33:12.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a8affd700 1 -- 172.21.15.81:0/3164796656 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a940036a0 con 0x7f8a900dfe00 2024-05-08T23:33:12.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9f0c9700 1 -- 172.21.15.81:0/3164796656 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a901756a0 con 0x7f8a900dfe00 2024-05-08T23:33:12.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a8affd700 1 -- 172.21.15.81:0/3164796656 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94003a10 con 0x7f8a900dfe00 2024-05-08T23:33:12.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.226+0000 7f8a9f0c9700 1 -- 172.21.15.81:0/3164796656 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a90175c60 con 0x7f8a900dfe00 2024-05-08T23:33:12.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.230+0000 7f8a9f0c9700 1 -- 172.21.15.81:0/3164796656 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a90009b90 con 0x7f8a900dfe00 2024-05-08T23:33:12.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.230+0000 7f8a8affd700 1 -- 172.21.15.81:0/3164796656 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8a94003b70 con 0x7f8a900dfe00 2024-05-08T23:33:12.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.230+0000 7f8a8affd700 1 --2- 172.21.15.81:0/3164796656 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8a7c066730 0x7f8a7c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:12.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.230+0000 7f8a8affd700 1 -- 172.21.15.81:0/3164796656 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a94083430 con 0x7f8a900dfe00 2024-05-08T23:33:12.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.230+0000 7f8a9e0c7700 1 --2- 172.21.15.81:0/3164796656 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8a7c066730 0x7f8a7c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.233 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.230+0000 7f8a9e0c7700 1 --2- 172.21.15.81:0/3164796656 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8a7c066730 0x7f8a7c068bf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0090b0 tx=0x7f8a8c010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:12.239 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.238+0000 7f8a8affd700 1 -- 172.21.15.81:0/3164796656 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8a94050850 con 0x7f8a900dfe00 2024-05-08T23:33:12.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.466+0000 7f8a9f0c9700 1 -- 172.21.15.81:0/3164796656 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f8a90050d40 con 0x7f8a900dfe00 2024-05-08T23:33:12.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.474+0000 7f8a8affd700 1 -- 172.21.15.81:0/3164796656 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8a9400ce60 con 0x7f8a900dfe00 2024-05-08T23:33:12.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.478+0000 7f8a9f0c9700 1 -- 172.21.15.81:0/3164796656 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8a7c066730 msgr2=0x7f8a7c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:12.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.478+0000 7f8a9f0c9700 1 --2- 172.21.15.81:0/3164796656 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8a7c066730 0x7f8a7c068bf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0090b0 tx=0x7f8a8c010000 comp rx=0 tx=0).stop 2024-05-08T23:33:12.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.478+0000 7f8a9f0c9700 1 -- 172.21.15.81:0/3164796656 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a900dfe00 msgr2=0x7f8a9004cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:12.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.478+0000 7f8a9f0c9700 1 --2- 172.21.15.81:0/3164796656 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a900dfe00 0x7f8a9004cd50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400c520 tx=0x7f8a9400c830 comp rx=0 tx=0).stop 2024-05-08T23:33:12.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.478+0000 7f8a9f0c9700 1 -- 172.21.15.81:0/3164796656 shutdown_connections 2024-05-08T23:33:12.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.478+0000 7f8a9f0c9700 1 --2- 172.21.15.81:0/3164796656 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8a7c066730 0x7f8a7c068bf0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.478+0000 7f8a9f0c9700 1 --2- 172.21.15.81:0/3164796656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a900e01e0 0x7f8a900d91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.478+0000 7f8a9f0c9700 1 --2- 172.21.15.81:0/3164796656 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a900dfe00 0x7f8a9004cd50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.478+0000 7f8a9f0c9700 1 --2- 172.21.15.81:0/3164796656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900505c0 0x7f8a9004d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.478+0000 7f8a9f0c9700 1 -- 172.21.15.81:0/3164796656 >> 172.21.15.81:0/3164796656 conn(0x7f8a90055c80 msgr2=0x7f8a900df1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:12.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.478+0000 7f8a9f0c9700 1 -- 172.21.15.81:0/3164796656 shutdown_connections 2024-05-08T23:33:12.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.478+0000 7f8a9f0c9700 1 -- 172.21.15.81:0/3164796656 wait complete. 2024-05-08T23:33:12.481 INFO:teuthology.orchestra.run.smithi081.stderr:updated caps for client.0 2024-05-08T23:33:12.494 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T23:33:12.495 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:33:12.511 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T23:33:12.531 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T23:33:12.531 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:33:12.532 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:33:12.562 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:33:12.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.678+0000 7f541db5b700 1 Processor -- start 2024-05-08T23:33:12.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.678+0000 7f541db5b700 1 -- start start 2024-05-08T23:33:12.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f541db5b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54180cf980 0x7f541815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:12.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f541db5b700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541815e830 con 0x7f54180cddf0 2024-05-08T23:33:12.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f541db5b700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54180ce2d0 con 0x7f54180d0bd0 2024-05-08T23:33:12.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f541db5b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54180ce5b0 con 0x7f54180cf980 2024-05-08T23:33:12.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540ffff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f54180cddf0 0x7f54180cfd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34348/0 (socket says 172.21.15.81:34348) 2024-05-08T23:33:12.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540ffff700 1 -- 172.21.15.81:0/1441045168 learned_addr learned my addr 172.21.15.81:0/1441045168 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:12.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f541cb59700 1 --2- 172.21.15.81:0/1441045168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54180cf980 0x7f541815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890675932 0 0) 0x7f5408001110 con 0x7f54180cddf0 2024-05-08T23:33:12.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54000035d0 con 0x7f54180cddf0 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1402862090 0 0) 0x7f54180ce2d0 con 0x7f54180d0bd0 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5408001110 con 0x7f54180d0bd0 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2648025370 0 0) 0x7f54000035d0 con 0x7f54180cddf0 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54180cf980 msgr2=0x7f541815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 --2- 172.21.15.81:0/1441045168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54180cf980 0x7f541815e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 >> v1:172.21.15.117:6789/0 conn(0x7f54180d0bd0 legacy=0x7f54180d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54180ce890 con 0x7f54180cddf0 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f5408001800 con 0x7f54180cddf0 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 >> v1:172.21.15.81:6789/0 conn(0x7f54180cddf0 legacy=0x7f54180cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 --2- 172.21.15.81:0/1441045168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54180cf980 0x7f5400003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 --2- 172.21.15.81:0/1441045168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5400004190 0x7f5400004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 --2- 172.21.15.81:0/1441045168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5400004db0 0x7f5400005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54180ce5b0 con 0x7f5400004db0 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540000bb70 con 0x7f5400004190 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540000bd30 con 0x7f54180cf980 2024-05-08T23:33:12.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540ffff700 1 --2- 172.21.15.81:0/1441045168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5400004190 0x7f5400004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f541d35a700 1 --2- 172.21.15.81:0/1441045168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5400004db0 0x7f5400005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f541cb59700 1 --2- 172.21.15.81:0/1441045168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54180cf980 0x7f5400003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f541d35a700 1 -- 172.21.15.81:0/1441045168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54180cf980 msgr2=0x7f5400003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:12.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f541d35a700 1 --2- 172.21.15.81:0/1441045168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54180cf980 0x7f5400003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f541d35a700 1 -- 172.21.15.81:0/1441045168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5400004190 msgr2=0x7f5400004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:12.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f541d35a700 1 --2- 172.21.15.81:0/1441045168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5400004190 0x7f5400004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f541d35a700 1 -- 172.21.15.81:0/1441045168 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54000035d0 con 0x7f5400004db0 2024-05-08T23:33:12.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f541d35a700 1 --2- 172.21.15.81:0/1441045168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5400004db0 0x7f5400005230 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f541000c2f0 tx=0x7f54000058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4aa851ffb6da85a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:12.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410001f30 con 0x7f5400004db0 2024-05-08T23:33:12.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541000ce90 con 0x7f5400004db0 2024-05-08T23:33:12.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.682+0000 7f540f7fe700 1 -- 172.21.15.81:0/1441045168 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410003c60 con 0x7f5400004db0 2024-05-08T23:33:12.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541db5b700 1 -- 172.21.15.81:0/1441045168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5400004db0 msgr2=0x7f5400005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:12.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541db5b700 1 --2- 172.21.15.81:0/1441045168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5400004db0 0x7f5400005230 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f541000c2f0 tx=0x7f54000058a0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541d35a700 1 -- 172.21.15.81:0/1441045168 reap_dead start 2024-05-08T23:33:12.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541db5b700 1 -- 172.21.15.81:0/1441045168 shutdown_connections 2024-05-08T23:33:12.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541db5b700 1 -- 172.21.15.81:0/1441045168 >> 172.21.15.81:0/1441045168 conn(0x7f541809c370 msgr2=0x7f54180d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:12.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541db5b700 1 -- 172.21.15.81:0/1441045168 shutdown_connections 2024-05-08T23:33:12.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541db5b700 1 -- 172.21.15.81:0/1441045168 wait complete. 2024-05-08T23:33:12.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541db5b700 1 Processor -- start 2024-05-08T23:33:12.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541db5b700 1 -- start start 2024-05-08T23:33:12.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541db5b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5418150f70 0x7f5418146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:12.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541db5b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5418151350 0x7f5418147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:12.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541db5b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5418147570 0x7f54180b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:12.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541db5b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54181559b0 con 0x7f5418150f70 2024-05-08T23:33:12.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541db5b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54180c7ad0 con 0x7f5418147570 2024-05-08T23:33:12.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541db5b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54180c74c0 con 0x7f5418151350 2024-05-08T23:33:12.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541cb59700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5418150f70 0x7f5418146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541cb59700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5418150f70 0x7f5418146af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36794/0 (socket says 172.21.15.81:36794) 2024-05-08T23:33:12.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.686+0000 7f541cb59700 1 -- 172.21.15.81:0/3173291689 learned_addr learned my addr 172.21.15.81:0/3173291689 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:12.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f540ffff700 1 --2- 172.21.15.81:0/3173291689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5418151350 0x7f5418147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f541d35a700 1 --2- 172.21.15.81:0/3173291689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5418147570 0x7f54180b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f541cb59700 1 -- 172.21.15.81:0/3173291689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5418151350 msgr2=0x7f5418147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:12.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f541cb59700 1 --2- 172.21.15.81:0/3173291689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5418151350 0x7f5418147030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f541cb59700 1 -- 172.21.15.81:0/3173291689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5418147570 msgr2=0x7f54180b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:12.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f541cb59700 1 --2- 172.21.15.81:0/3173291689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5418147570 0x7f54180b69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f541cb59700 1 -- 172.21.15.81:0/3173291689 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54181556d0 con 0x7f5418150f70 2024-05-08T23:33:12.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f540ffff700 1 --2- 172.21.15.81:0/3173291689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5418151350 0x7f5418147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:12.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f541cb59700 1 --2- 172.21.15.81:0/3173291689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5418150f70 0x7f5418146af0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5414011a50 tx=0x7f5414011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:12.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f540d7fa700 1 -- 172.21.15.81:0/3173291689 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541400fa00 con 0x7f5418150f70 2024-05-08T23:33:12.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f540d7fa700 1 -- 172.21.15.81:0/3173291689 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541400fb60 con 0x7f5418150f70 2024-05-08T23:33:12.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f541db5b700 1 -- 172.21.15.81:0/3173291689 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54180c7760 con 0x7f5418150f70 2024-05-08T23:33:12.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f540d7fa700 1 -- 172.21.15.81:0/3173291689 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5414014440 con 0x7f5418150f70 2024-05-08T23:33:12.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f541db5b700 1 -- 172.21.15.81:0/3173291689 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54181527b0 con 0x7f5418150f70 2024-05-08T23:33:12.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f541db5b700 1 -- 172.21.15.81:0/3173291689 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54180c7320 con 0x7f5418150f70 2024-05-08T23:33:12.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f540d7fa700 1 -- 172.21.15.81:0/3173291689 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f54140145a0 con 0x7f5418150f70 2024-05-08T23:33:12.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f540d7fa700 1 --2- 172.21.15.81:0/3173291689 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5404066760 0x7f5404068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:12.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f540d7fa700 1 -- 172.21.15.81:0/3173291689 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5414084970 con 0x7f5418150f70 2024-05-08T23:33:12.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.690+0000 7f540ffff700 1 --2- 172.21.15.81:0/3173291689 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5404066760 0x7f5404068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.694+0000 7f540ffff700 1 --2- 172.21.15.81:0/3173291689 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5404066760 0x7f5404068c20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f54180c9ff0 tx=0x7f5408009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:12.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.698+0000 7f540d7fa700 1 -- 172.21.15.81:0/3173291689 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f54180c7320 con 0x7f5418150f70 2024-05-08T23:33:12.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.870+0000 7f541db5b700 1 -- 172.21.15.81:0/3173291689 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54180c7320 con 0x7f5418150f70 2024-05-08T23:33:12.877 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.874+0000 7f540d7fa700 1 -- 172.21.15.81:0/3173291689 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f54180c7320 con 0x7f5418150f70 2024-05-08T23:33:12.878 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:12.878 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":52,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:32:41.855853+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:39.161796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:39.853963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:33:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.878+0000 7f541db5b700 1 -- 172.21.15.81:0/3173291689 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5404066760 msgr2=0x7f5404068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.878+0000 7f541db5b700 1 --2- 172.21.15.81:0/3173291689 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5404066760 0x7f5404068c20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f54180c9ff0 tx=0x7f5408009040 comp rx=0 tx=0).stop 2024-05-08T23:33:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.878+0000 7f541db5b700 1 -- 172.21.15.81:0/3173291689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5418150f70 msgr2=0x7f5418146af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.878+0000 7f541db5b700 1 --2- 172.21.15.81:0/3173291689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5418150f70 0x7f5418146af0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5414011a50 tx=0x7f5414011e10 comp rx=0 tx=0).stop 2024-05-08T23:33:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.882+0000 7f541db5b700 1 -- 172.21.15.81:0/3173291689 shutdown_connections 2024-05-08T23:33:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.882+0000 7f541db5b700 1 --2- 172.21.15.81:0/3173291689 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5404066760 0x7f5404068c20 secure :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f54180c9ff0 tx=0x7f5408009040 comp rx=0 tx=0).stop 2024-05-08T23:33:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.882+0000 7f541db5b700 1 --2- 172.21.15.81:0/3173291689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5418147570 0x7f54180b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.882+0000 7f541db5b700 1 --2- 172.21.15.81:0/3173291689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5418151350 0x7f5418147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.882+0000 7f541db5b700 1 --2- 172.21.15.81:0/3173291689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5418150f70 0x7f5418146af0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.882+0000 7f541db5b700 1 -- 172.21.15.81:0/3173291689 >> 172.21.15.81:0/3173291689 conn(0x7f541809c370 msgr2=0x7f5418160de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.882+0000 7f541db5b700 1 -- 172.21.15.81:0/3173291689 shutdown_connections 2024-05-08T23:33:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:12.882+0000 7f541db5b700 1 -- 172.21.15.81:0/3173291689 wait complete. 2024-05-08T23:33:12.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:32:39.161796+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:32:39.853963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:33:12.896 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:33:13.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.006+0000 7f8cb0024700 1 Processor -- start 2024-05-08T23:33:13.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.006+0000 7f8cb0024700 1 -- start start 2024-05-08T23:33:13.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cb0024700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8ca8160a30 0x7f8ca80d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cb0024700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8ca80d3c50 0x7f8ca80d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cb0024700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ca80d1cc0 0x7f8ca80cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cb0024700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca80b66d0 con 0x7f8ca80d3c50 2024-05-08T23:33:13.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cb0024700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca80d2150 con 0x7f8ca8160a30 2024-05-08T23:33:13.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cb0024700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80d11c0 con 0x7f8ca80d1cc0 2024-05-08T23:33:13.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cad5bf700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8ca80d3c50 0x7f8ca80d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cad5bf700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8ca80d3c50 0x7f8ca80d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36798/0 (socket says 172.21.15.81:36798) 2024-05-08T23:33:13.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cad5bf700 1 -- 172.21.15.81:0/1683947953 learned_addr learned my addr 172.21.15.81:0/1683947953 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:13.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cae5c1700 1 --2- 172.21.15.81:0/1683947953 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ca80d1cc0 0x7f8ca80cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8caddc0700 1 --2- 172.21.15.81:0/1683947953 >> v2:172.21.15.117:3300/0 conn(0x7f8ca8160a30 0x7f8ca80d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cad5bf700 1 -- 172.21.15.81:0/1683947953 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ca80d1cc0 msgr2=0x7f8ca80cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:13.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cad5bf700 1 --2- 172.21.15.81:0/1683947953 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ca80d1cc0 0x7f8ca80cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cad5bf700 1 -- 172.21.15.81:0/1683947953 >> v2:172.21.15.117:3300/0 conn(0x7f8ca8160a30 msgr2=0x7f8ca80d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:13.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cad5bf700 1 --2- 172.21.15.81:0/1683947953 >> v2:172.21.15.117:3300/0 conn(0x7f8ca8160a30 0x7f8ca80d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cad5bf700 1 -- 172.21.15.81:0/1683947953 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca80d14a0 con 0x7f8ca80d3c50 2024-05-08T23:33:13.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cad5bf700 1 --2- 172.21.15.81:0/1683947953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ca80d3c50 0x7f8ca80d1780 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80d0db0 tx=0x7f8c9c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb36856b042c758 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:13.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cacdbe700 1 -- 172.21.15.81:0/1683947953 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c011830 con 0x7f8ca80d3c50 2024-05-08T23:33:13.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cacdbe700 1 -- 172.21.15.81:0/1683947953 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c011990 con 0x7f8ca80d3c50 2024-05-08T23:33:13.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cacdbe700 1 -- 172.21.15.81:0/1683947953 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c011d20 con 0x7f8ca80d3c50 2024-05-08T23:33:13.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cb0024700 1 -- 172.21.15.81:0/1683947953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ca80d3c50 msgr2=0x7f8ca80d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cb0024700 1 --2- 172.21.15.81:0/1683947953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ca80d3c50 0x7f8ca80d1780 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80d0db0 tx=0x7f8c9c009bf0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cb0024700 1 -- 172.21.15.81:0/1683947953 shutdown_connections 2024-05-08T23:33:13.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cb0024700 1 --2- 172.21.15.81:0/1683947953 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ca80d1cc0 0x7f8ca80cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cb0024700 1 --2- 172.21.15.81:0/1683947953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ca80d3c50 0x7f8ca80d1780 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cb0024700 1 --2- 172.21.15.81:0/1683947953 >> v2:172.21.15.117:3300/0 conn(0x7f8ca8160a30 0x7f8ca80d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cb0024700 1 -- 172.21.15.81:0/1683947953 >> 172.21.15.81:0/1683947953 conn(0x7f8ca809c500 msgr2=0x7f8ca80d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:13.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cb0024700 1 -- 172.21.15.81:0/1683947953 shutdown_connections 2024-05-08T23:33:13.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.010+0000 7f8cb0024700 1 -- 172.21.15.81:0/1683947953 wait complete. 2024-05-08T23:33:13.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cb0024700 1 Processor -- start 2024-05-08T23:33:13.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cb0024700 1 -- start start 2024-05-08T23:33:13.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cb0024700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ca80d1cc0 0x7f8ca809ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cb0024700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ca80d3c50 0x7f8ca814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cb0024700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ca8160a30 0x7f8ca814ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cb0024700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca814ca30 con 0x7f8ca8160a30 2024-05-08T23:33:13.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cb0024700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca815aca0 con 0x7f8ca80d1cc0 2024-05-08T23:33:13.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cb0024700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca809d3b0 con 0x7f8ca80d3c50 2024-05-08T23:33:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cae5c1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ca8160a30 0x7f8ca814ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cae5c1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ca8160a30 0x7f8ca814ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36802/0 (socket says 172.21.15.81:36802) 2024-05-08T23:33:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cae5c1700 1 -- 172.21.15.81:0/2260440728 learned_addr learned my addr 172.21.15.81:0/2260440728 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cad5bf700 1 --2- 172.21.15.81:0/2260440728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ca80d3c50 0x7f8ca814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8caddc0700 1 --2- 172.21.15.81:0/2260440728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ca80d1cc0 0x7f8ca809ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cae5c1700 1 -- 172.21.15.81:0/2260440728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ca80d3c50 msgr2=0x7f8ca814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cae5c1700 1 --2- 172.21.15.81:0/2260440728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ca80d3c50 0x7f8ca814afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cae5c1700 1 -- 172.21.15.81:0/2260440728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ca80d1cc0 msgr2=0x7f8ca809ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cae5c1700 1 --2- 172.21.15.81:0/2260440728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ca80d1cc0 0x7f8ca809ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cae5c1700 1 -- 172.21.15.81:0/2260440728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c009570 con 0x7f8ca8160a30 2024-05-08T23:33:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cae5c1700 1 --2- 172.21.15.81:0/2260440728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ca8160a30 0x7f8ca814ce30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800e910 tx=0x7f8c9800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8ca67fc700 1 -- 172.21.15.81:0/2260440728 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c980096c0 con 0x7f8ca8160a30 2024-05-08T23:33:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8ca67fc700 1 -- 172.21.15.81:0/2260440728 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9800c7f0 con 0x7f8ca8160a30 2024-05-08T23:33:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.014+0000 7f8cb0024700 1 -- 172.21.15.81:0/2260440728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca814bdb0 con 0x7f8ca8160a30 2024-05-08T23:33:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.018+0000 7f8ca67fc700 1 -- 172.21.15.81:0/2260440728 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9800cb80 con 0x7f8ca8160a30 2024-05-08T23:33:13.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.018+0000 7f8cb0024700 1 -- 172.21.15.81:0/2260440728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca814c370 con 0x7f8ca8160a30 2024-05-08T23:33:13.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.018+0000 7f8cb0024700 1 -- 172.21.15.81:0/2260440728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca80cf0f0 con 0x7f8ca8160a30 2024-05-08T23:33:13.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.018+0000 7f8ca67fc700 1 -- 172.21.15.81:0/2260440728 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8c9800cce0 con 0x7f8ca8160a30 2024-05-08T23:33:13.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.018+0000 7f8ca67fc700 1 --2- 172.21.15.81:0/2260440728 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8c94066730 0x7f8c94068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.018+0000 7f8ca67fc700 1 -- 172.21.15.81:0/2260440728 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c98084130 con 0x7f8ca8160a30 2024-05-08T23:33:13.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.018+0000 7f8caddc0700 1 --2- 172.21.15.81:0/2260440728 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8c94066730 0x7f8c94068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.022+0000 7f8caddc0700 1 --2- 172.21.15.81:0/2260440728 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8c94066730 0x7f8c94068bf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00073f0 tx=0x7f8ca000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:13.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.026+0000 7f8ca67fc700 1 -- 172.21.15.81:0/2260440728 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8c980514a0 con 0x7f8ca8160a30 2024-05-08T23:33:13.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.190+0000 7f8cb0024700 1 -- 172.21.15.81:0/2260440728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ca80684b0 con 0x7f8ca8160a30 2024-05-08T23:33:13.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.190+0000 7f8ca67fc700 1 -- 172.21.15.81:0/2260440728 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c98054ac0 con 0x7f8ca8160a30 2024-05-08T23:33:13.194 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:33:13.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.194+0000 7f8cb0024700 1 -- 172.21.15.81:0/2260440728 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8c94066730 msgr2=0x7f8c94068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.194+0000 7f8cb0024700 1 --2- 172.21.15.81:0/2260440728 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8c94066730 0x7f8c94068bf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00073f0 tx=0x7f8ca000b040 comp rx=0 tx=0).stop 2024-05-08T23:33:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.194+0000 7f8cb0024700 1 -- 172.21.15.81:0/2260440728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ca8160a30 msgr2=0x7f8ca814ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.194+0000 7f8cb0024700 1 --2- 172.21.15.81:0/2260440728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ca8160a30 0x7f8ca814ce30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800e910 tx=0x7f8c9800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.194+0000 7f8cb0024700 1 -- 172.21.15.81:0/2260440728 shutdown_connections 2024-05-08T23:33:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.194+0000 7f8cb0024700 1 --2- 172.21.15.81:0/2260440728 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8c94066730 0x7f8c94068bf0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.194+0000 7f8cb0024700 1 --2- 172.21.15.81:0/2260440728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ca8160a30 0x7f8ca814ce30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.194+0000 7f8cb0024700 1 --2- 172.21.15.81:0/2260440728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ca80d3c50 0x7f8ca814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.194+0000 7f8cb0024700 1 --2- 172.21.15.81:0/2260440728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ca80d1cc0 0x7f8ca809ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.194+0000 7f8cb0024700 1 -- 172.21.15.81:0/2260440728 >> 172.21.15.81:0/2260440728 conn(0x7f8ca809c500 msgr2=0x7f8ca80d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.194+0000 7f8cb0024700 1 -- 172.21.15.81:0/2260440728 shutdown_connections 2024-05-08T23:33:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.194+0000 7f8cb0024700 1 -- 172.21.15.81:0/2260440728 wait complete. 2024-05-08T23:33:13.208 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:33:13.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.318+0000 7fa2bccb5700 1 Processor -- start 2024-05-08T23:33:13.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.318+0000 7fa2bccb5700 1 -- start start 2024-05-08T23:33:13.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2bccb5700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa2b8160920 0x7fa2b80ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2bccb5700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2b80d0050 0x7fa2b80b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2bccb5700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b80d3d40 con 0x7fa2b80d1da0 2024-05-08T23:33:13.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2bccb5700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b80d0d00 con 0x7fa2b8160920 2024-05-08T23:33:13.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2bccb5700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80d0fe0 con 0x7fa2b80d0050 2024-05-08T23:33:13.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2b77fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa2b8160920 0x7fa2b80ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2aeffd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa2b80d1da0 0x7fa2b80cf940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34364/0 (socket says 172.21.15.81:34364) 2024-05-08T23:33:13.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2aeffd700 1 -- 172.21.15.81:0/1406248710 learned_addr learned my addr 172.21.15.81:0/1406248710 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:13.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2b7fff700 1 --2- 172.21.15.81:0/1406248710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2b80d0050 0x7fa2b80b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2b6ffd700 1 -- 172.21.15.81:0/1406248710 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763358825 0 0) 0x7fa2b80d3d40 con 0x7fa2b80d1da0 2024-05-08T23:33:13.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2b6ffd700 1 -- 172.21.15.81:0/1406248710 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2a40035d0 con 0x7fa2b80d1da0 2024-05-08T23:33:13.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2b77fe700 1 -- 172.21.15.81:0/1406248710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2b80d0050 msgr2=0x7fa2b80b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:13.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2b77fe700 1 --2- 172.21.15.81:0/1406248710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2b80d0050 0x7fa2b80b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2b77fe700 1 -- 172.21.15.81:0/1406248710 >> v1:172.21.15.81:6789/0 conn(0x7fa2b80d1da0 legacy=0x7fa2b80cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2b77fe700 1 -- 172.21.15.81:0/1406248710 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b80d12c0 con 0x7fa2b8160920 2024-05-08T23:33:13.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2b77fe700 1 --2- 172.21.15.81:0/1406248710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2b8160920 0x7fa2b80ce6f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009850 tx=0x7fa2a8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a581f30556f429b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:13.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2b6ffd700 1 -- 172.21.15.81:0/1406248710 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8011830 con 0x7fa2b8160920 2024-05-08T23:33:13.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2b6ffd700 1 -- 172.21.15.81:0/1406248710 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a8011990 con 0x7fa2b8160920 2024-05-08T23:33:13.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2b6ffd700 1 -- 172.21.15.81:0/1406248710 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8011d40 con 0x7fa2b8160920 2024-05-08T23:33:13.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2bccb5700 1 -- 172.21.15.81:0/1406248710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2b8160920 msgr2=0x7fa2b80ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2bccb5700 1 --2- 172.21.15.81:0/1406248710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2b8160920 0x7fa2b80ce6f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009850 tx=0x7fa2a8009b60 comp rx=0 tx=0).stop 2024-05-08T23:33:13.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2bccb5700 1 -- 172.21.15.81:0/1406248710 shutdown_connections 2024-05-08T23:33:13.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2bccb5700 1 --2- 172.21.15.81:0/1406248710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2b80d0050 0x7fa2b80b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2bccb5700 1 --2- 172.21.15.81:0/1406248710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2b8160920 0x7fa2b80ce6f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2bccb5700 1 -- 172.21.15.81:0/1406248710 >> 172.21.15.81:0/1406248710 conn(0x7fa2b809c340 msgr2=0x7fa2b80d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:13.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2bccb5700 1 -- 172.21.15.81:0/1406248710 shutdown_connections 2024-05-08T23:33:13.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.322+0000 7fa2bccb5700 1 -- 172.21.15.81:0/1406248710 wait complete. 2024-05-08T23:33:13.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.326+0000 7fa2bccb5700 1 Processor -- start 2024-05-08T23:33:13.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.326+0000 7fa2bccb5700 1 -- start start 2024-05-08T23:33:13.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.326+0000 7fa2bccb5700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2b80d0050 0x7fa2b8157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.326+0000 7fa2bccb5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2b80d1da0 0x7fa2b815ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.326+0000 7fa2bccb5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2b8160920 0x7fa2b815d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.326+0000 7fa2bccb5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80b3a90 con 0x7fa2b8160920 2024-05-08T23:33:13.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.326+0000 7fa2bccb5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b815dd60 con 0x7fa2b80d1da0 2024-05-08T23:33:13.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.326+0000 7fa2bccb5700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b815e040 con 0x7fa2b80d0050 2024-05-08T23:33:13.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.326+0000 7fa2b7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2b8160920 0x7fa2b815d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.326+0000 7fa2b7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2b8160920 0x7fa2b815d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36818/0 (socket says 172.21.15.81:36818) 2024-05-08T23:33:13.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.326+0000 7fa2b7fff700 1 -- 172.21.15.81:0/680027732 learned_addr learned my addr 172.21.15.81:0/680027732 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:13.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.326+0000 7fa2b77fe700 1 --2- 172.21.15.81:0/680027732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2b80d0050 0x7fa2b8157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.326+0000 7fa2aeffd700 1 --2- 172.21.15.81:0/680027732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2b80d1da0 0x7fa2b815ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2b77fe700 1 -- 172.21.15.81:0/680027732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2b80d1da0 msgr2=0x7fa2b815ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2b77fe700 1 --2- 172.21.15.81:0/680027732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2b80d1da0 0x7fa2b815ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2b77fe700 1 -- 172.21.15.81:0/680027732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2b8160920 msgr2=0x7fa2b815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2b77fe700 1 --2- 172.21.15.81:0/680027732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2b8160920 0x7fa2b815d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2b77fe700 1 -- 172.21.15.81:0/680027732 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a80094b0 con 0x7fa2b80d0050 2024-05-08T23:33:13.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2b7fff700 1 --2- 172.21.15.81:0/680027732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2b8160920 0x7fa2b815d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:13.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2b77fe700 1 --2- 172.21.15.81:0/680027732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2b80d0050 0x7fa2b8157470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8016000 tx=0x7fa2a8004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:13.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2b4ff9700 1 -- 172.21.15.81:0/680027732 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80044d0 con 0x7fa2b80d0050 2024-05-08T23:33:13.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2b4ff9700 1 -- 172.21.15.81:0/680027732 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a801e070 con 0x7fa2b80d0050 2024-05-08T23:33:13.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2bccb5700 1 -- 172.21.15.81:0/680027732 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b8160e80 con 0x7fa2b80d0050 2024-05-08T23:33:13.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2b4ff9700 1 -- 172.21.15.81:0/680027732 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8003900 con 0x7fa2b80d0050 2024-05-08T23:33:13.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2bccb5700 1 -- 172.21.15.81:0/680027732 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b815e600 con 0x7fa2b80d0050 2024-05-08T23:33:13.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2bccb5700 1 -- 172.21.15.81:0/680027732 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b80684b0 con 0x7fa2b80d0050 2024-05-08T23:33:13.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2b4ff9700 1 -- 172.21.15.81:0/680027732 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa2a8003a60 con 0x7fa2b80d0050 2024-05-08T23:33:13.335 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2b4ff9700 1 --2- 172.21.15.81:0/680027732 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa298066730 0x7fa298068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.335 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.330+0000 7fa2b4ff9700 1 -- 172.21.15.81:0/680027732 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2a8085050 con 0x7fa2b80d0050 2024-05-08T23:33:13.335 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.334+0000 7fa2aeffd700 1 --2- 172.21.15.81:0/680027732 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa298066730 0x7fa298068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.334+0000 7fa2aeffd700 1 --2- 172.21.15.81:0/680027732 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa298066730 0x7fa298068bf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80ccef0 tx=0x7fa2a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:13.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.342+0000 7fa2b4ff9700 1 -- 172.21.15.81:0/680027732 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa2a80b6cf0 con 0x7fa2b80d0050 2024-05-08T23:33:13.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.514+0000 7fa2bccb5700 1 -- 172.21.15.81:0/680027732 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa2b80cf510 con 0x7fa2b80d0050 2024-05-08T23:33:13.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.514+0000 7fa2b4ff9700 1 -- 172.21.15.81:0/680027732 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa2a80523b0 con 0x7fa2b80d0050 2024-05-08T23:33:13.518 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:33:13.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.518+0000 7fa2bccb5700 1 -- 172.21.15.81:0/680027732 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa298066730 msgr2=0x7fa298068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.518+0000 7fa2bccb5700 1 --2- 172.21.15.81:0/680027732 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa298066730 0x7fa298068bf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80ccef0 tx=0x7fa2a0009040 comp rx=0 tx=0).stop 2024-05-08T23:33:13.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.518+0000 7fa2bccb5700 1 -- 172.21.15.81:0/680027732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2b80d0050 msgr2=0x7fa2b8157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.518+0000 7fa2bccb5700 1 --2- 172.21.15.81:0/680027732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2b80d0050 0x7fa2b8157470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8016000 tx=0x7fa2a8004770 comp rx=0 tx=0).stop 2024-05-08T23:33:13.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.518+0000 7fa2bccb5700 1 -- 172.21.15.81:0/680027732 shutdown_connections 2024-05-08T23:33:13.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.518+0000 7fa2bccb5700 1 --2- 172.21.15.81:0/680027732 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fa298066730 0x7fa298068bf0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.518+0000 7fa2bccb5700 1 --2- 172.21.15.81:0/680027732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2b8160920 0x7fa2b815d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.518+0000 7fa2bccb5700 1 --2- 172.21.15.81:0/680027732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2b80d1da0 0x7fa2b815ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.518+0000 7fa2bccb5700 1 --2- 172.21.15.81:0/680027732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2b80d0050 0x7fa2b8157470 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.518+0000 7fa2bccb5700 1 -- 172.21.15.81:0/680027732 >> 172.21.15.81:0/680027732 conn(0x7fa2b809c340 msgr2=0x7fa2b806afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:13.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.522+0000 7fa2bccb5700 1 -- 172.21.15.81:0/680027732 shutdown_connections 2024-05-08T23:33:13.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.522+0000 7fa2bccb5700 1 -- 172.21.15.81:0/680027732 wait complete. 2024-05-08T23:33:13.538 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:33:13.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.654+0000 7f5da1874700 1 Processor -- start 2024-05-08T23:33:13.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.654+0000 7f5da1874700 1 -- start start 2024-05-08T23:33:13.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5da1874700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5d9c0d3c50 0x7f5d9c0d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5da1874700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5d9c0d1e90 0x7f5d9c0b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5da1874700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c0b66d0 con 0x7f5d9c160a30 2024-05-08T23:33:13.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5da1874700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c0d11c0 con 0x7f5d9c0d1e90 2024-05-08T23:33:13.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5da1874700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c0d14a0 con 0x7f5d9c0d3c50 2024-05-08T23:33:13.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5d9b7fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5d9c0d1e90 0x7f5d9c0b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5d9affd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5d9c160a30 0x7f5d9c0d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34366/0 (socket says 172.21.15.81:34366) 2024-05-08T23:33:13.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5d9affd700 1 -- 172.21.15.81:0/188312715 learned_addr learned my addr 172.21.15.81:0/188312715 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:13.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5d9a7fc700 1 --2- 172.21.15.81:0/188312715 >> v2:172.21.15.117:3300/0 conn(0x7f5d9c0d3c50 0x7f5d9c0d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5d99ffb700 1 -- 172.21.15.81:0/188312715 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981019498 0 0) 0x7f5d9c0b66d0 con 0x7f5d9c160a30 2024-05-08T23:33:13.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5d9b7fe700 1 -- 172.21.15.81:0/188312715 >> v2:172.21.15.117:3300/0 conn(0x7f5d9c0d3c50 msgr2=0x7f5d9c0d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:13.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5d9b7fe700 1 --2- 172.21.15.81:0/188312715 >> v2:172.21.15.117:3300/0 conn(0x7f5d9c0d3c50 0x7f5d9c0d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5d9b7fe700 1 -- 172.21.15.81:0/188312715 >> v1:172.21.15.81:6789/0 conn(0x7f5d9c160a30 legacy=0x7f5d9c0d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5d9b7fe700 1 -- 172.21.15.81:0/188312715 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c0b6080 con 0x7f5d9c0d1e90 2024-05-08T23:33:13.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5d9b7fe700 1 --2- 172.21.15.81:0/188312715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d9c0d1e90 0x7f5d9c0b6b60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0d0d80 tx=0x7f5d9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4b9140ad2f102d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:13.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5d99ffb700 1 -- 172.21.15.81:0/188312715 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9400e070 con 0x7f5d9c0d1e90 2024-05-08T23:33:13.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5d99ffb700 1 -- 172.21.15.81:0/188312715 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9400bb20 con 0x7f5d9c0d1e90 2024-05-08T23:33:13.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5d99ffb700 1 -- 172.21.15.81:0/188312715 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d94012430 con 0x7f5d9c0d1e90 2024-05-08T23:33:13.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5da1874700 1 -- 172.21.15.81:0/188312715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d9c0d1e90 msgr2=0x7f5d9c0b6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5da1874700 1 --2- 172.21.15.81:0/188312715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d9c0d1e90 0x7f5d9c0b6b60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0d0d80 tx=0x7f5d9400b330 comp rx=0 tx=0).stop 2024-05-08T23:33:13.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5da1874700 1 -- 172.21.15.81:0/188312715 shutdown_connections 2024-05-08T23:33:13.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5da1874700 1 --2- 172.21.15.81:0/188312715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d9c0d1e90 0x7f5d9c0b6b60 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5da1874700 1 --2- 172.21.15.81:0/188312715 >> v2:172.21.15.117:3300/0 conn(0x7f5d9c0d3c50 0x7f5d9c0d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5da1874700 1 -- 172.21.15.81:0/188312715 >> 172.21.15.81:0/188312715 conn(0x7f5d9c09c500 msgr2=0x7f5d9c0d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:13.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5da1874700 1 -- 172.21.15.81:0/188312715 shutdown_connections 2024-05-08T23:33:13.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.658+0000 7f5da1874700 1 -- 172.21.15.81:0/188312715 wait complete. 2024-05-08T23:33:13.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5da1874700 1 Processor -- start 2024-05-08T23:33:13.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5da1874700 1 -- start start 2024-05-08T23:33:13.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5da1874700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d9c0d1e90 0x7f5d9c15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5da1874700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d9c0d3c50 0x7f5d9c0cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5da1874700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d9c160a30 0x7f5d9c09e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5da1874700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c09fcd0 con 0x7f5d9c0d1e90 2024-05-08T23:33:13.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5da1874700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c09d4a0 con 0x7f5d9c0d3c50 2024-05-08T23:33:13.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5da1874700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c09d780 con 0x7f5d9c160a30 2024-05-08T23:33:13.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5d9affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d9c0d1e90 0x7f5d9c15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5d9affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d9c0d1e90 0x7f5d9c15d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36834/0 (socket says 172.21.15.81:36834) 2024-05-08T23:33:13.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5d9affd700 1 -- 172.21.15.81:0/2008737192 learned_addr learned my addr 172.21.15.81:0/2008737192 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:13.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5d9b7fe700 1 --2- 172.21.15.81:0/2008737192 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d9c160a30 0x7f5d9c09e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5d9a7fc700 1 --2- 172.21.15.81:0/2008737192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d9c0d3c50 0x7f5d9c0cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5d9b7fe700 1 -- 172.21.15.81:0/2008737192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d9c0d3c50 msgr2=0x7f5d9c0cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5d9b7fe700 1 --2- 172.21.15.81:0/2008737192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d9c0d3c50 0x7f5d9c0cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5d9b7fe700 1 -- 172.21.15.81:0/2008737192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d9c0d1e90 msgr2=0x7f5d9c15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5d9b7fe700 1 --2- 172.21.15.81:0/2008737192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d9c0d1e90 0x7f5d9c15d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5d9b7fe700 1 -- 172.21.15.81:0/2008737192 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d94009a70 con 0x7f5d9c160a30 2024-05-08T23:33:13.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5d9b7fe700 1 --2- 172.21.15.81:0/2008737192 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d9c160a30 0x7f5d9c09e540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d94016000 tx=0x7f5d94004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:13.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5d83fff700 1 -- 172.21.15.81:0/2008737192 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d94012430 con 0x7f5d9c160a30 2024-05-08T23:33:13.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5d83fff700 1 -- 172.21.15.81:0/2008737192 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d94020070 con 0x7f5d9c160a30 2024-05-08T23:33:13.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5d83fff700 1 -- 172.21.15.81:0/2008737192 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d94004280 con 0x7f5d9c160a30 2024-05-08T23:33:13.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5da1874700 1 -- 172.21.15.81:0/2008737192 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c09da60 con 0x7f5d9c160a30 2024-05-08T23:33:13.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.662+0000 7f5da1874700 1 -- 172.21.15.81:0/2008737192 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c09e020 con 0x7f5d9c160a30 2024-05-08T23:33:13.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.666+0000 7f5da1874700 1 -- 172.21.15.81:0/2008737192 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c0684b0 con 0x7f5d9c160a30 2024-05-08T23:33:13.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.666+0000 7f5d83fff700 1 -- 172.21.15.81:0/2008737192 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5d940043e0 con 0x7f5d9c160a30 2024-05-08T23:33:13.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.666+0000 7f5d83fff700 1 --2- 172.21.15.81:0/2008737192 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5d88066730 0x7f5d88068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.669 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.666+0000 7f5d83fff700 1 -- 172.21.15.81:0/2008737192 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d94022070 con 0x7f5d9c160a30 2024-05-08T23:33:13.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.666+0000 7f5d9affd700 1 --2- 172.21.15.81:0/2008737192 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5d88066730 0x7f5d88068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.666+0000 7f5d9affd700 1 --2- 172.21.15.81:0/2008737192 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5d88066730 0x7f5d88068bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5d90000f60 tx=0x7f5d90007f50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:13.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.678+0000 7f5d83fff700 1 -- 172.21.15.81:0/2008737192 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5d94053060 con 0x7f5d9c160a30 2024-05-08T23:33:13.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.846+0000 7f5da1874700 1 -- 172.21.15.81:0/2008737192 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5d9c09e1b0 con 0x7f5d9c160a30 2024-05-08T23:33:13.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.846+0000 7f5d83fff700 1 -- 172.21.15.81:0/2008737192 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5d9402c020 con 0x7f5d9c160a30 2024-05-08T23:33:13.851 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:33:13.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.850+0000 7f5da1874700 1 -- 172.21.15.81:0/2008737192 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5d88066730 msgr2=0x7f5d88068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.850+0000 7f5da1874700 1 --2- 172.21.15.81:0/2008737192 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5d88066730 0x7f5d88068bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5d90000f60 tx=0x7f5d90007f50 comp rx=0 tx=0).stop 2024-05-08T23:33:13.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.850+0000 7f5da1874700 1 -- 172.21.15.81:0/2008737192 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d9c160a30 msgr2=0x7f5d9c09e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.850+0000 7f5da1874700 1 --2- 172.21.15.81:0/2008737192 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d9c160a30 0x7f5d9c09e540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d94016000 tx=0x7f5d94004af0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.850+0000 7f5da1874700 1 -- 172.21.15.81:0/2008737192 shutdown_connections 2024-05-08T23:33:13.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.850+0000 7f5da1874700 1 --2- 172.21.15.81:0/2008737192 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5d88066730 0x7f5d88068bf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.850+0000 7f5da1874700 1 --2- 172.21.15.81:0/2008737192 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d9c160a30 0x7f5d9c09e540 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.850+0000 7f5da1874700 1 --2- 172.21.15.81:0/2008737192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d9c0d3c50 0x7f5d9c0cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.850+0000 7f5da1874700 1 --2- 172.21.15.81:0/2008737192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d9c0d1e90 0x7f5d9c15d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.850+0000 7f5da1874700 1 -- 172.21.15.81:0/2008737192 >> 172.21.15.81:0/2008737192 conn(0x7f5d9c09c500 msgr2=0x7f5d9c15f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:13.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.850+0000 7f5da1874700 1 -- 172.21.15.81:0/2008737192 shutdown_connections 2024-05-08T23:33:13.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.850+0000 7f5da1874700 1 -- 172.21.15.81:0/2008737192 wait complete. 2024-05-08T23:33:13.871 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:33:13.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.982+0000 7f5006144700 1 Processor -- start 2024-05-08T23:33:13.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.982+0000 7f5006144700 1 -- start start 2024-05-08T23:33:13.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.982+0000 7f5006144700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50000d37c0 0x7f50000d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.982+0000 7f5006144700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f50000b2ea0 0x7f50000b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.982+0000 7f5006144700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50000d3c20 con 0x7f50000d1690 2024-05-08T23:33:13.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.982+0000 7f5006144700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50000d3e50 con 0x7f50000b2ea0 2024-05-08T23:33:13.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.982+0000 7f5006144700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50000d1460 con 0x7f50000d37c0 2024-05-08T23:33:13.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.982+0000 7f5005943700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f50000b2ea0 0x7f50000b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.982+0000 7f5005943700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f50000b2ea0 0x7f50000b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36838/0 (socket says 172.21.15.81:36838) 2024-05-08T23:33:13.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.982+0000 7f5005943700 1 -- 172.21.15.81:0/3547283982 learned_addr learned my addr 172.21.15.81:0/3547283982 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:13.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.982+0000 7f5004941700 1 --2- 172.21.15.81:0/3547283982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50000d37c0 0x7f50000d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.988 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5004941700 1 -- 172.21.15.81:0/3547283982 >> v2:172.21.15.81:3300/0 conn(0x7f50000b2ea0 msgr2=0x7f50000b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:13.988 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5004941700 1 --2- 172.21.15.81:0/3547283982 >> v2:172.21.15.81:3300/0 conn(0x7f50000b2ea0 0x7f50000b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.988 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5004941700 1 -- 172.21.15.81:0/3547283982 >> v1:172.21.15.117:6789/0 conn(0x7f50000d1690 legacy=0x7f50000d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.989 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5004941700 1 -- 172.21.15.81:0/3547283982 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50000ce870 con 0x7f50000d37c0 2024-05-08T23:33:13.989 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5005943700 1 --2- 172.21.15.81:0/3547283982 >> v2:172.21.15.81:3300/0 conn(0x7f50000b2ea0 0x7f50000b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:13.989 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5004941700 1 --2- 172.21.15.81:0/3547283982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50000d37c0 0x7f50000d0f20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009850 tx=0x7f4ff4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfa855bd2a8be520 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:13.989 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f4ff3fff700 1 -- 172.21.15.81:0/3547283982 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4011830 con 0x7f50000d37c0 2024-05-08T23:33:13.989 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f4ff3fff700 1 -- 172.21.15.81:0/3547283982 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff4011990 con 0x7f50000d37c0 2024-05-08T23:33:13.989 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f4ff3fff700 1 -- 172.21.15.81:0/3547283982 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4011d40 con 0x7f50000d37c0 2024-05-08T23:33:13.990 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5006144700 1 -- 172.21.15.81:0/3547283982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50000d37c0 msgr2=0x7f50000d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.990 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5006144700 1 --2- 172.21.15.81:0/3547283982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50000d37c0 0x7f50000d0f20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009850 tx=0x7f4ff4009b60 comp rx=0 tx=0).stop 2024-05-08T23:33:13.990 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5006144700 1 -- 172.21.15.81:0/3547283982 shutdown_connections 2024-05-08T23:33:13.990 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5006144700 1 --2- 172.21.15.81:0/3547283982 >> v2:172.21.15.81:3300/0 conn(0x7f50000b2ea0 0x7f50000b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.990 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5006144700 1 --2- 172.21.15.81:0/3547283982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50000d37c0 0x7f50000d0f20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.990 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5006144700 1 -- 172.21.15.81:0/3547283982 >> 172.21.15.81:0/3547283982 conn(0x7f500009c3c0 msgr2=0x7f50000d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:13.990 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5006144700 1 -- 172.21.15.81:0/3547283982 shutdown_connections 2024-05-08T23:33:13.990 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5006144700 1 -- 172.21.15.81:0/3547283982 wait complete. 2024-05-08T23:33:13.991 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5006144700 1 Processor -- start 2024-05-08T23:33:13.991 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.986+0000 7f5006144700 1 -- start start 2024-05-08T23:33:13.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5006144700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50000b2ea0 0x7f50001f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5006144700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50000d1690 0x7f500014f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5006144700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50000d37c0 0x7f5000160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5006144700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50001f69f0 con 0x7f50000b2ea0 2024-05-08T23:33:13.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5006144700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50001f63a0 con 0x7f50000d37c0 2024-05-08T23:33:13.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5006144700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500015fcc0 con 0x7f50000d1690 2024-05-08T23:33:13.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5005142700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50000b2ea0 0x7f50001f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5005142700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50000b2ea0 0x7f50001f6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36852/0 (socket says 172.21.15.81:36852) 2024-05-08T23:33:13.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5005142700 1 -- 172.21.15.81:0/1819247639 learned_addr learned my addr 172.21.15.81:0/1819247639 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:13.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5005943700 1 --2- 172.21.15.81:0/1819247639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50000d37c0 0x7f5000160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5004941700 1 --2- 172.21.15.81:0/1819247639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50000d1690 0x7f500014f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5005142700 1 -- 172.21.15.81:0/1819247639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50000d1690 msgr2=0x7f500014f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5005142700 1 --2- 172.21.15.81:0/1819247639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50000d1690 0x7f500014f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5005142700 1 -- 172.21.15.81:0/1819247639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50000d37c0 msgr2=0x7f5000160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:13.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5005142700 1 --2- 172.21.15.81:0/1819247639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50000d37c0 0x7f5000160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:13.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5005142700 1 -- 172.21.15.81:0/1819247639 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff8009b30 con 0x7f50000b2ea0 2024-05-08T23:33:13.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5004941700 1 --2- 172.21.15.81:0/1819247639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50000d1690 0x7f500014f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:13.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5005142700 1 --2- 172.21.15.81:0/1819247639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50000b2ea0 0x7f50001f6e10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00c500 tx=0x7f4ffc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:13.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f4ff1ffb700 1 -- 172.21.15.81:0/1819247639 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc0052d0 con 0x7f50000b2ea0 2024-05-08T23:33:13.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f4ff1ffb700 1 -- 172.21.15.81:0/1819247639 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ffc005430 con 0x7f50000b2ea0 2024-05-08T23:33:13.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5006144700 1 -- 172.21.15.81:0/1819247639 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff40094b0 con 0x7f50000b2ea0 2024-05-08T23:33:13.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f4ff1ffb700 1 -- 172.21.15.81:0/1819247639 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc0055a0 con 0x7f50000b2ea0 2024-05-08T23:33:13.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5006144700 1 -- 172.21.15.81:0/1819247639 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500015f9d0 con 0x7f50000b2ea0 2024-05-08T23:33:13.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f5006144700 1 -- 172.21.15.81:0/1819247639 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50000c97a0 con 0x7f50000b2ea0 2024-05-08T23:33:13.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.990+0000 7f4ff1ffb700 1 -- 172.21.15.81:0/1819247639 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4ffc0038e0 con 0x7f50000b2ea0 2024-05-08T23:33:13.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.994+0000 7f4ff1ffb700 1 --2- 172.21.15.81:0/1819247639 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4fe4066760 0x7f4fe4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:13.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.994+0000 7f5004941700 1 --2- 172.21.15.81:0/1819247639 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4fe4066760 0x7f4fe4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:13.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.994+0000 7f4ff1ffb700 1 -- 172.21.15.81:0/1819247639 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ffc083540 con 0x7f50000b2ea0 2024-05-08T23:33:13.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:13.994+0000 7f5004941700 1 --2- 172.21.15.81:0/1819247639 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4fe4066760 0x7f4fe4068c20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40090b0 tx=0x7f4ff4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:14.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.002+0000 7f4ff1ffb700 1 -- 172.21.15.81:0/1819247639 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4ffc0508a0 con 0x7f50000b2ea0 2024-05-08T23:33:14.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.178+0000 7f5006144700 1 -- 172.21.15.81:0/1819247639 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f50000684b0 con 0x7f50000b2ea0 2024-05-08T23:33:14.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.178+0000 7f4ff1ffb700 1 -- 172.21.15.81:0/1819247639 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4ffc053ec0 con 0x7f50000b2ea0 2024-05-08T23:33:14.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.182+0000 7f5006144700 1 -- 172.21.15.81:0/1819247639 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4fe4066760 msgr2=0x7f4fe4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.182+0000 7f5006144700 1 --2- 172.21.15.81:0/1819247639 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4fe4066760 0x7f4fe4068c20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40090b0 tx=0x7f4ff4008030 comp rx=0 tx=0).stop 2024-05-08T23:33:14.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.182+0000 7f5006144700 1 -- 172.21.15.81:0/1819247639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50000b2ea0 msgr2=0x7f50001f6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.182+0000 7f5006144700 1 --2- 172.21.15.81:0/1819247639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50000b2ea0 0x7f50001f6e10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00c500 tx=0x7f4ffc00c810 comp rx=0 tx=0).stop 2024-05-08T23:33:14.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.182+0000 7f5006144700 1 -- 172.21.15.81:0/1819247639 shutdown_connections 2024-05-08T23:33:14.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.182+0000 7f5006144700 1 --2- 172.21.15.81:0/1819247639 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f4fe4066760 0x7f4fe4068c20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.182+0000 7f5006144700 1 --2- 172.21.15.81:0/1819247639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50000d37c0 0x7f5000160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.182+0000 7f5006144700 1 --2- 172.21.15.81:0/1819247639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50000d1690 0x7f500014f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.182+0000 7f5006144700 1 --2- 172.21.15.81:0/1819247639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50000b2ea0 0x7f50001f6e10 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.182+0000 7f5006144700 1 -- 172.21.15.81:0/1819247639 >> 172.21.15.81:0/1819247639 conn(0x7f500009c3c0 msgr2=0x7f50000b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:14.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.182+0000 7f5006144700 1 -- 172.21.15.81:0/1819247639 shutdown_connections 2024-05-08T23:33:14.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.182+0000 7f5006144700 1 -- 172.21.15.81:0/1819247639 wait complete. 2024-05-08T23:33:14.186 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2024-05-08T23:33:14.197 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2024-05-08T23:33:14.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:14.197 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T23:33:14.210 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[40026]: fuse finished with error 0 and tester_r 0 2024-05-08T23:33:14.213 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:33:14.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:14.214 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:33:14.234 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:33:14.234 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:33:14.242 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346332k,nr_inodes=4086583,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17609 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T23:33:14.242 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:33:14.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:14.242 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T23:33:14.248 INFO:teuthology.orchestra.run.smithi117.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-08T23:33:14.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:33:14.250 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:33:14.250 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:33:14.282 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346332k,nr_inodes=4086583,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14716 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T23:33:14.282 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:33:14.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:14.282 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-08T23:33:14.289 INFO:teuthology.orchestra.run.smithi081.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-08T23:33:14.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:33:14.290 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:33:14.290 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:33:14.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346332k,nr_inodes=4086583,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17609 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T23:33:14.300 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:33:14.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:14.300 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-08T23:33:14.307 INFO:teuthology.orchestra.run.smithi117.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-08T23:33:14.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:33:14.308 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:33:14.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.426+0000 7fd06518c700 1 Processor -- start 2024-05-08T23:33:14.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.426+0000 7fd06518c700 1 -- start start 2024-05-08T23:33:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.426+0000 7fd06518c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd0600cc850 0x7fd06015bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.426+0000 7fd06518c700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0600ccc60 con 0x7fd06006ba80 2024-05-08T23:33:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.426+0000 7fd06518c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0600cbf00 con 0x7fd0600cbb20 2024-05-08T23:33:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.426+0000 7fd06518c700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06015c430 con 0x7fd0600cc850 2024-05-08T23:33:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.426+0000 7fd05f7fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd06006ba80 0x7fd06006cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34376/0 (socket says 172.21.15.81:34376) 2024-05-08T23:33:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.426+0000 7fd05f7fe700 1 -- 172.21.15.81:0/1087547270 learned_addr learned my addr 172.21.15.81:0/1087547270 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:14.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd06498b700 1 --2- 172.21.15.81:0/1087547270 >> v2:172.21.15.81:3300/0 conn(0x7fd0600cc850 0x7fd06015bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:14.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd05effd700 1 -- 172.21.15.81:0/1087547270 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325711983 0 0) 0x7fd0600ccc60 con 0x7fd06006ba80 2024-05-08T23:33:14.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd05effd700 1 -- 172.21.15.81:0/1087547270 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0480035d0 con 0x7fd06006ba80 2024-05-08T23:33:14.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd05effd700 1 -- 172.21.15.81:0/1087547270 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637113797 0 0) 0x7fd0600cbf00 con 0x7fd0600cbb20 2024-05-08T23:33:14.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd05effd700 1 -- 172.21.15.81:0/1087547270 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0600ccc60 con 0x7fd0600cbb20 2024-05-08T23:33:14.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd06498b700 1 -- 172.21.15.81:0/1087547270 >> v1:172.21.15.117:6789/0 conn(0x7fd0600cbb20 legacy=0x7fd06006b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd06498b700 1 -- 172.21.15.81:0/1087547270 >> v1:172.21.15.81:6789/0 conn(0x7fd06006ba80 legacy=0x7fd06006cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd06498b700 1 -- 172.21.15.81:0/1087547270 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0600ccee0 con 0x7fd0600cc850 2024-05-08T23:33:14.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd06498b700 1 --2- 172.21.15.81:0/1087547270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0600cc850 0x7fd06015bef0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd058009d90 tx=0x7fd05800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7cae54b542b00d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:14.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd05effd700 1 -- 172.21.15.81:0/1087547270 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05800e070 con 0x7fd0600cc850 2024-05-08T23:33:14.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd05effd700 1 -- 172.21.15.81:0/1087547270 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05800bb20 con 0x7fd0600cc850 2024-05-08T23:33:14.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd05effd700 1 -- 172.21.15.81:0/1087547270 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05800be90 con 0x7fd0600cc850 2024-05-08T23:33:14.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd06518c700 1 -- 172.21.15.81:0/1087547270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0600cc850 msgr2=0x7fd06015bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd06518c700 1 --2- 172.21.15.81:0/1087547270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0600cc850 0x7fd06015bef0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd058009d90 tx=0x7fd05800b330 comp rx=0 tx=0).stop 2024-05-08T23:33:14.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd06518c700 1 -- 172.21.15.81:0/1087547270 shutdown_connections 2024-05-08T23:33:14.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd06518c700 1 --2- 172.21.15.81:0/1087547270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0600cc850 0x7fd06015bef0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd06518c700 1 -- 172.21.15.81:0/1087547270 >> 172.21.15.81:0/1087547270 conn(0x7fd060070960 msgr2=0x7fd060070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:14.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd06518c700 1 -- 172.21.15.81:0/1087547270 shutdown_connections 2024-05-08T23:33:14.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd06518c700 1 -- 172.21.15.81:0/1087547270 wait complete. 2024-05-08T23:33:14.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd06518c700 1 Processor -- start 2024-05-08T23:33:14.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.430+0000 7fd06518c700 1 -- start start 2024-05-08T23:33:14.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd06518c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd06006ba80 0x7fd0600b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:14.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd06518c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0600cbb20 0x7fd060144ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:14.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd06518c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0600cc850 0x7fd06015e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:14.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd06518c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0601f6840 con 0x7fd0600cbb20 2024-05-08T23:33:14.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd06518c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0601f6b20 con 0x7fd0600cc850 2024-05-08T23:33:14.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd06518c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0601f6e00 con 0x7fd06006ba80 2024-05-08T23:33:14.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd05f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0600cbb20 0x7fd060144ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:14.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd05f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0600cbb20 0x7fd060144ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36880/0 (socket says 172.21.15.81:36880) 2024-05-08T23:33:14.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd05f7fe700 1 -- 172.21.15.81:0/2327577711 learned_addr learned my addr 172.21.15.81:0/2327577711 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:14.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd05ffff700 1 --2- 172.21.15.81:0/2327577711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd06006ba80 0x7fd0600b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:14.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd06498b700 1 --2- 172.21.15.81:0/2327577711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0600cc850 0x7fd06015e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:14.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd05f7fe700 1 -- 172.21.15.81:0/2327577711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd06006ba80 msgr2=0x7fd0600b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd05f7fe700 1 --2- 172.21.15.81:0/2327577711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd06006ba80 0x7fd0600b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd05f7fe700 1 -- 172.21.15.81:0/2327577711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0600cc850 msgr2=0x7fd06015e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd05f7fe700 1 --2- 172.21.15.81:0/2327577711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0600cc850 0x7fd06015e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd05f7fe700 1 -- 172.21.15.81:0/2327577711 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd058009a70 con 0x7fd0600cbb20 2024-05-08T23:33:14.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd05f7fe700 1 --2- 172.21.15.81:0/2327577711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0600cbb20 0x7fd060144ea0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd05400c560 tx=0x7fd05400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:14.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd05cff9700 1 -- 172.21.15.81:0/2327577711 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd054005d90 con 0x7fd0600cbb20 2024-05-08T23:33:14.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd05cff9700 1 -- 172.21.15.81:0/2327577711 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0540070d0 con 0x7fd0600cbb20 2024-05-08T23:33:14.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd06518c700 1 -- 172.21.15.81:0/2327577711 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0601f70e0 con 0x7fd0600cbb20 2024-05-08T23:33:14.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd05cff9700 1 -- 172.21.15.81:0/2327577711 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd054007460 con 0x7fd0600cbb20 2024-05-08T23:33:14.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd06518c700 1 -- 172.21.15.81:0/2327577711 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0601453e0 con 0x7fd0600cbb20 2024-05-08T23:33:14.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.434+0000 7fd06518c700 1 -- 172.21.15.81:0/2327577711 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06014db30 con 0x7fd0600cbb20 2024-05-08T23:33:14.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.438+0000 7fd05cff9700 1 -- 172.21.15.81:0/2327577711 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd0540075c0 con 0x7fd0600cbb20 2024-05-08T23:33:14.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.438+0000 7fd05cff9700 1 --2- 172.21.15.81:0/2327577711 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd04c066780 0x7fd04c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:14.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.438+0000 7fd05cff9700 1 -- 172.21.15.81:0/2327577711 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd054083570 con 0x7fd0600cbb20 2024-05-08T23:33:14.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.438+0000 7fd05ffff700 1 --2- 172.21.15.81:0/2327577711 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd04c066780 0x7fd04c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:14.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.438+0000 7fd05ffff700 1 --2- 172.21.15.81:0/2327577711 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd04c066780 0x7fd04c068c40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd0500011e0 tx=0x7fd050009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:14.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.442+0000 7fd05cff9700 1 -- 172.21.15.81:0/2327577711 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd06014db30 con 0x7fd0600cbb20 2024-05-08T23:33:14.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.646+0000 7fd06518c700 1 -- 172.21.15.81:0/2327577711 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd06014db30 con 0x7fd0600cbb20 2024-05-08T23:33:14.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.646+0000 7fd05cff9700 1 -- 172.21.15.81:0/2327577711 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4743 (secure 0 0 0) 0x7fd06014db30 con 0x7fd0600cbb20 2024-05-08T23:33:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:14.650 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/2105335102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":2105335102},{"type":"v1","addr":"172.21.15.117:6834","nonce":2105335102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2900127785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2900127785},{"type":"v1","addr":"172.21.15.81:6835","nonce":2900127785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1899281779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1899281779},{"type":"v1","addr":"172.21.15.81:6837","nonce":1899281779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:32:40.852853+0000","modified":"2024-05-08T23:32:43.867414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":15311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_15311":{"gid":15311,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/3401411259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3401411259},{"type":"v1","addr":"172.21.15.117:6835","nonce":3401411259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:33:14.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.650+0000 7fd06518c700 1 -- 172.21.15.81:0/2327577711 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd04c066780 msgr2=0x7fd04c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.650+0000 7fd06518c700 1 --2- 172.21.15.81:0/2327577711 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd04c066780 0x7fd04c068c40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd0500011e0 tx=0x7fd050009040 comp rx=0 tx=0).stop 2024-05-08T23:33:14.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.650+0000 7fd06518c700 1 -- 172.21.15.81:0/2327577711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0600cbb20 msgr2=0x7fd060144ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.650+0000 7fd06518c700 1 --2- 172.21.15.81:0/2327577711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0600cbb20 0x7fd060144ea0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd05400c560 tx=0x7fd05400c870 comp rx=0 tx=0).stop 2024-05-08T23:33:14.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.650+0000 7fd06518c700 1 -- 172.21.15.81:0/2327577711 shutdown_connections 2024-05-08T23:33:14.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.650+0000 7fd06518c700 1 --2- 172.21.15.81:0/2327577711 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd04c066780 0x7fd04c068c40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.650+0000 7fd06518c700 1 --2- 172.21.15.81:0/2327577711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0600cc850 0x7fd06015e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.650+0000 7fd06518c700 1 --2- 172.21.15.81:0/2327577711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0600cbb20 0x7fd060144ea0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.650+0000 7fd06518c700 1 --2- 172.21.15.81:0/2327577711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd06006ba80 0x7fd0600b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.650+0000 7fd06518c700 1 -- 172.21.15.81:0/2327577711 >> 172.21.15.81:0/2327577711 conn(0x7fd060070960 msgr2=0x7fd06006a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:14.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.650+0000 7fd06518c700 1 -- 172.21.15.81:0/2327577711 shutdown_connections 2024-05-08T23:33:14.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.650+0000 7fd06518c700 1 -- 172.21.15.81:0/2327577711 wait complete. 2024-05-08T23:33:14.654 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 23 2024-05-08T23:33:14.669 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:33:14.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278b589700 1 Processor -- start 2024-05-08T23:33:14.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278b589700 1 -- start start 2024-05-08T23:33:14.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278b589700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27840b3c20 0x7f27841624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:14.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278b589700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2784071e70 0x7f27840b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:14.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278b589700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2784158960 con 0x7f2784165070 2024-05-08T23:33:14.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278b589700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27840b1f60 con 0x7f2784071e70 2024-05-08T23:33:14.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278b589700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2784071120 con 0x7f27840b3c20 2024-05-08T23:33:14.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f2789d86700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2784071e70 0x7f27840b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:14.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f2789d86700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2784071e70 0x7f27840b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36894/0 (socket says 172.21.15.81:36894) 2024-05-08T23:33:14.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f2789d86700 1 -- 172.21.15.81:0/1844358404 learned_addr learned my addr 172.21.15.81:0/1844358404 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:14.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278ad88700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2784165070 0x7f2784165500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34384/0 (socket says 172.21.15.81:34384) 2024-05-08T23:33:14.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278a587700 1 --2- 172.21.15.81:0/1844358404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27840b3c20 0x7f27841624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:14.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f2789d86700 1 -- 172.21.15.81:0/1844358404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27840b3c20 msgr2=0x7f27841624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:14.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f2789d86700 1 --2- 172.21.15.81:0/1844358404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27840b3c20 0x7f27841624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f2789d86700 1 -- 172.21.15.81:0/1844358404 >> v1:172.21.15.81:6789/0 conn(0x7f2784165070 legacy=0x7f2784165500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f2789d86700 1 -- 172.21.15.81:0/1844358404 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27840b4540 con 0x7f2784071e70 2024-05-08T23:33:14.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f2789d86700 1 --2- 172.21.15.81:0/1844358404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2784071e70 0x7f27840b4000 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2780009910 tx=0x7f2780009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a1ec16f2a0b9b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:14.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f2789585700 1 -- 172.21.15.81:0/1844358404 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278000ba60 con 0x7f2784071e70 2024-05-08T23:33:14.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f2789585700 1 -- 172.21.15.81:0/1844358404 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278000bbc0 con 0x7f2784071e70 2024-05-08T23:33:14.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f2789585700 1 -- 172.21.15.81:0/1844358404 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780009e60 con 0x7f2784071e70 2024-05-08T23:33:14.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278b589700 1 -- 172.21.15.81:0/1844358404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2784071e70 msgr2=0x7f27840b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278b589700 1 --2- 172.21.15.81:0/1844358404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2784071e70 0x7f27840b4000 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2780009910 tx=0x7f2780009c20 comp rx=0 tx=0).stop 2024-05-08T23:33:14.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278b589700 1 -- 172.21.15.81:0/1844358404 shutdown_connections 2024-05-08T23:33:14.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278b589700 1 --2- 172.21.15.81:0/1844358404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2784071e70 0x7f27840b4000 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278b589700 1 --2- 172.21.15.81:0/1844358404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27840b3c20 0x7f27841624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278b589700 1 -- 172.21.15.81:0/1844358404 >> 172.21.15.81:0/1844358404 conn(0x7f278406c330 msgr2=0x7f278406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:14.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278b589700 1 -- 172.21.15.81:0/1844358404 shutdown_connections 2024-05-08T23:33:14.788 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.782+0000 7f278b589700 1 -- 172.21.15.81:0/1844358404 wait complete. 2024-05-08T23:33:14.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f278b589700 1 Processor -- start 2024-05-08T23:33:14.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f278b589700 1 -- start start 2024-05-08T23:33:14.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f278b589700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2784071e70 0x7f2784147fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:14.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f278b589700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27840b3c20 0x7f27840afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:14.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f278b589700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2784165070 0x7f27840b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:14.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f278b589700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278415a6e0 con 0x7f27840b3c20 2024-05-08T23:33:14.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f278b589700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27841484f0 con 0x7f2784071e70 2024-05-08T23:33:14.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f278b589700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278415c390 con 0x7f2784165070 2024-05-08T23:33:14.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f2789d86700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27840b3c20 0x7f27840afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:14.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f278ad88700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2784165070 0x7f27840b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:14.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f2789d86700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27840b3c20 0x7f27840afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36896/0 (socket says 172.21.15.81:36896) 2024-05-08T23:33:14.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f278ad88700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2784165070 0x7f27840b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.81:40180/0 (socket says 172.21.15.81:40180) 2024-05-08T23:33:14.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f2789d86700 1 -- 172.21.15.81:0/211947626 learned_addr learned my addr 172.21.15.81:0/211947626 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:14.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f278a587700 1 --2- 172.21.15.81:0/211947626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2784071e70 0x7f2784147fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:14.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f2789d86700 1 -- 172.21.15.81:0/211947626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2784165070 msgr2=0x7f27840b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f2789d86700 1 --2- 172.21.15.81:0/211947626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2784165070 0x7f27840b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f2789d86700 1 -- 172.21.15.81:0/211947626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2784071e70 msgr2=0x7f2784147fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f2789d86700 1 --2- 172.21.15.81:0/211947626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2784071e70 0x7f2784147fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f2789d86700 1 -- 172.21.15.81:0/211947626 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2780009570 con 0x7f27840b3c20 2024-05-08T23:33:14.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.786+0000 7f278a587700 1 --2- 172.21.15.81:0/211947626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2784071e70 0x7f2784147fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:14.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.790+0000 7f2789d86700 1 --2- 172.21.15.81:0/211947626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27840b3c20 0x7f27840afe90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2780014040 tx=0x7f2780004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:14.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.790+0000 7f277affd700 1 -- 172.21.15.81:0/211947626 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780004590 con 0x7f27840b3c20 2024-05-08T23:33:14.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.790+0000 7f277affd700 1 -- 172.21.15.81:0/211947626 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278001e070 con 0x7f27840b3c20 2024-05-08T23:33:14.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.790+0000 7f278b589700 1 -- 172.21.15.81:0/211947626 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27841596d0 con 0x7f27840b3c20 2024-05-08T23:33:14.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.790+0000 7f277affd700 1 -- 172.21.15.81:0/211947626 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780003900 con 0x7f27840b3c20 2024-05-08T23:33:14.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.790+0000 7f278b589700 1 -- 172.21.15.81:0/211947626 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278415a020 con 0x7f27840b3c20 2024-05-08T23:33:14.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.790+0000 7f278b589700 1 -- 172.21.15.81:0/211947626 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278415a820 con 0x7f27840b3c20 2024-05-08T23:33:14.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.790+0000 7f277affd700 1 -- 172.21.15.81:0/211947626 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2780003a60 con 0x7f27840b3c20 2024-05-08T23:33:14.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.794+0000 7f277affd700 1 --2- 172.21.15.81:0/211947626 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2768066760 0x7f2768068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:14.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.794+0000 7f278a587700 1 --2- 172.21.15.81:0/211947626 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2768066760 0x7f2768068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:14.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.794+0000 7f277affd700 1 -- 172.21.15.81:0/211947626 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2780020070 con 0x7f27840b3c20 2024-05-08T23:33:14.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.794+0000 7f278a587700 1 --2- 172.21.15.81:0/211947626 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2768066760 0x7f2768068c20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2774007a70 tx=0x7f2774008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:14.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.802+0000 7f277affd700 1 -- 172.21.15.81:0/211947626 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2780052980 con 0x7f27840b3c20 2024-05-08T23:33:14.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.970+0000 7f278b589700 1 -- 172.21.15.81:0/211947626 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27840684b0 con 0x7f27840b3c20 2024-05-08T23:33:14.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.974+0000 7f277affd700 1 -- 172.21.15.81:0/211947626 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f2780055fa0 con 0x7f27840b3c20 2024-05-08T23:33:14.976 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:14.976 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":52,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:32:41.855853+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:39.161796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:39.853963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:33:14.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.974+0000 7f278b589700 1 -- 172.21.15.81:0/211947626 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2768066760 msgr2=0x7f2768068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.974+0000 7f278b589700 1 --2- 172.21.15.81:0/211947626 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2768066760 0x7f2768068c20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2774007a70 tx=0x7f2774008040 comp rx=0 tx=0).stop 2024-05-08T23:33:14.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.974+0000 7f278b589700 1 -- 172.21.15.81:0/211947626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27840b3c20 msgr2=0x7f27840afe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.974+0000 7f278b589700 1 --2- 172.21.15.81:0/211947626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27840b3c20 0x7f27840afe90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2780014040 tx=0x7f2780004770 comp rx=0 tx=0).stop 2024-05-08T23:33:14.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.974+0000 7f278b589700 1 -- 172.21.15.81:0/211947626 shutdown_connections 2024-05-08T23:33:14.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.974+0000 7f278b589700 1 --2- 172.21.15.81:0/211947626 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2768066760 0x7f2768068c20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.974+0000 7f278b589700 1 --2- 172.21.15.81:0/211947626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2784165070 0x7f27840b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.974+0000 7f278b589700 1 --2- 172.21.15.81:0/211947626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27840b3c20 0x7f27840afe90 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.974+0000 7f278b589700 1 --2- 172.21.15.81:0/211947626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2784071e70 0x7f2784147fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.974+0000 7f278b589700 1 -- 172.21.15.81:0/211947626 >> 172.21.15.81:0/211947626 conn(0x7f278406c330 msgr2=0x7f2784165500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:14.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.974+0000 7f278b589700 1 -- 172.21.15.81:0/211947626 shutdown_connections 2024-05-08T23:33:14.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:14.974+0000 7f278b589700 1 -- 172.21.15.81:0/211947626 wait complete. 2024-05-08T23:33:14.992 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:30:25.389387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:32:39.161796+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:32:39.853963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:33:14.992 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:33:15.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.106+0000 7fb0e0b73700 1 Processor -- start 2024-05-08T23:33:15.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.106+0000 7fb0e0b73700 1 -- start start 2024-05-08T23:33:15.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0e0b73700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb0dc15d3b0 0x7fb0dc0b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0e0b73700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb0dc0ae990 0x7fb0dc0b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0e0b73700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0dc0aede0 con 0x7fb0dc0ae530 2024-05-08T23:33:15.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0e0b73700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0dc0b3210 con 0x7fb0dc0ae990 2024-05-08T23:33:15.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0e0b73700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0dc071f40 con 0x7fb0dc15d3b0 2024-05-08T23:33:15.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0dbfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb0dc0ae990 0x7fb0dc0b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0d2ffd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb0dc0ae530 0x7fb0dc06d110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34400/0 (socket says 172.21.15.81:34400) 2024-05-08T23:33:15.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0d2ffd700 1 -- 172.21.15.81:0/3864154815 learned_addr learned my addr 172.21.15.81:0/3864154815 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:15.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0db7fe700 1 --2- 172.21.15.81:0/3864154815 >> v2:172.21.15.117:3300/0 conn(0x7fb0dc15d3b0 0x7fb0dc0b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0dbfff700 1 -- 172.21.15.81:0/3864154815 >> v2:172.21.15.117:3300/0 conn(0x7fb0dc15d3b0 msgr2=0x7fb0dc0b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:15.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0dbfff700 1 --2- 172.21.15.81:0/3864154815 >> v2:172.21.15.117:3300/0 conn(0x7fb0dc15d3b0 0x7fb0dc0b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0dbfff700 1 -- 172.21.15.81:0/3864154815 >> v1:172.21.15.81:6789/0 conn(0x7fb0dc0ae530 legacy=0x7fb0dc06d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0dbfff700 1 -- 172.21.15.81:0/3864154815 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc0ad770 con 0x7fb0dc0ae990 2024-05-08T23:33:15.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0db7fe700 1 --2- 172.21.15.81:0/3864154815 >> v2:172.21.15.117:3300/0 conn(0x7fb0dc15d3b0 0x7fb0dc0b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:15.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0daffd700 1 -- 172.21.15.81:0/3864154815 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633817656 0 0) 0x7fb0dc0aede0 con 0x7fb0dc0ae530 2024-05-08T23:33:15.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0dbfff700 1 --2- 172.21.15.81:0/3864154815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0dc0ae990 0x7fb0dc0b38e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc04ed50 tx=0x7fb0d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1792087a8ab707d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:15.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0daffd700 1 -- 172.21.15.81:0/3864154815 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d400e070 con 0x7fb0dc0ae990 2024-05-08T23:33:15.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0daffd700 1 -- 172.21.15.81:0/3864154815 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d400bbd0 con 0x7fb0dc0ae990 2024-05-08T23:33:15.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0daffd700 1 -- 172.21.15.81:0/3864154815 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4012430 con 0x7fb0dc0ae990 2024-05-08T23:33:15.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0e0b73700 1 -- 172.21.15.81:0/3864154815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0dc0ae990 msgr2=0x7fb0dc0b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0e0b73700 1 --2- 172.21.15.81:0/3864154815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0dc0ae990 0x7fb0dc0b38e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc04ed50 tx=0x7fb0d400b330 comp rx=0 tx=0).stop 2024-05-08T23:33:15.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0e0b73700 1 -- 172.21.15.81:0/3864154815 shutdown_connections 2024-05-08T23:33:15.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0e0b73700 1 --2- 172.21.15.81:0/3864154815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0dc0ae990 0x7fb0dc0b38e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0e0b73700 1 --2- 172.21.15.81:0/3864154815 >> v2:172.21.15.117:3300/0 conn(0x7fb0dc15d3b0 0x7fb0dc0b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0e0b73700 1 -- 172.21.15.81:0/3864154815 >> 172.21.15.81:0/3864154815 conn(0x7fb0dc0cfcc0 msgr2=0x7fb0dc0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:15.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0e0b73700 1 -- 172.21.15.81:0/3864154815 shutdown_connections 2024-05-08T23:33:15.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.110+0000 7fb0e0b73700 1 -- 172.21.15.81:0/3864154815 wait complete. 2024-05-08T23:33:15.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.114+0000 7fb0e0b73700 1 Processor -- start 2024-05-08T23:33:15.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.114+0000 7fb0e0b73700 1 -- start start 2024-05-08T23:33:15.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.114+0000 7fb0e0b73700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0dc0ae530 0x7fb0dc1532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.114+0000 7fb0e0b73700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0dc0ae990 0x7fb0dc1f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.114+0000 7fb0e0b73700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc15d3b0 0x7fb0dc0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.114+0000 7fb0e0b73700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0b07b0 con 0x7fb0dc0ae990 2024-05-08T23:33:15.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.114+0000 7fb0e0b73700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc1537e0 con 0x7fb0dc0ae530 2024-05-08T23:33:15.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.114+0000 7fb0e0b73700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc153cc0 con 0x7fb0dc15d3b0 2024-05-08T23:33:15.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.114+0000 7fb0d2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0dc0ae990 0x7fb0dc1f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0d2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0dc0ae990 0x7fb0dc1f69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36924/0 (socket says 172.21.15.81:36924) 2024-05-08T23:33:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0d2ffd700 1 -- 172.21.15.81:0/1573564456 learned_addr learned my addr 172.21.15.81:0/1573564456 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0dbfff700 1 --2- 172.21.15.81:0/1573564456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc15d3b0 0x7fb0dc0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0db7fe700 1 --2- 172.21.15.81:0/1573564456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0dc0ae530 0x7fb0dc1532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0d2ffd700 1 -- 172.21.15.81:0/1573564456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc15d3b0 msgr2=0x7fb0dc0c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0d2ffd700 1 --2- 172.21.15.81:0/1573564456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc15d3b0 0x7fb0dc0c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0d2ffd700 1 -- 172.21.15.81:0/1573564456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0dc0ae530 msgr2=0x7fb0dc1532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0d2ffd700 1 --2- 172.21.15.81:0/1573564456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0dc0ae530 0x7fb0dc1532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0d2ffd700 1 -- 172.21.15.81:0/1573564456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d4009a70 con 0x7fb0dc0ae990 2024-05-08T23:33:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0db7fe700 1 --2- 172.21.15.81:0/1573564456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0dc0ae530 0x7fb0dc1532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0d2ffd700 1 --2- 172.21.15.81:0/1573564456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0dc0ae990 0x7fb0dc1f69a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400c580 tx=0x7fb0c400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0d8ff9700 1 -- 172.21.15.81:0/1573564456 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4004ca0 con 0x7fb0dc0ae990 2024-05-08T23:33:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0d8ff9700 1 -- 172.21.15.81:0/1573564456 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c4004e20 con 0x7fb0dc0ae990 2024-05-08T23:33:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0d8ff9700 1 -- 172.21.15.81:0/1573564456 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4007310 con 0x7fb0dc0ae990 2024-05-08T23:33:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0e0b73700 1 -- 172.21.15.81:0/1573564456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0dc1f6ee0 con 0x7fb0dc0ae990 2024-05-08T23:33:15.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0e0b73700 1 -- 172.21.15.81:0/1573564456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc0b0ce0 con 0x7fb0dc0ae990 2024-05-08T23:33:15.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0e0b73700 1 -- 172.21.15.81:0/1573564456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0dc0b0ed0 con 0x7fb0dc0ae990 2024-05-08T23:33:15.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.118+0000 7fb0d8ff9700 1 -- 172.21.15.81:0/1573564456 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb0c4007470 con 0x7fb0dc0ae990 2024-05-08T23:33:15.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.122+0000 7fb0d8ff9700 1 --2- 172.21.15.81:0/1573564456 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb0bc066730 0x7fb0bc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.122+0000 7fb0d8ff9700 1 -- 172.21.15.81:0/1573564456 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0c4083e80 con 0x7fb0dc0ae990 2024-05-08T23:33:15.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.122+0000 7fb0db7fe700 1 --2- 172.21.15.81:0/1573564456 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb0bc066730 0x7fb0bc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.122+0000 7fb0db7fe700 1 --2- 172.21.15.81:0/1573564456 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb0bc066730 0x7fb0bc068bf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00c9a0 tx=0x7fb0cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:15.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.130+0000 7fb0d8ff9700 1 -- 172.21.15.81:0/1573564456 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb0c40511f0 con 0x7fb0dc0ae990 2024-05-08T23:33:15.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.290+0000 7fb0e0b73700 1 -- 172.21.15.81:0/1573564456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0dc0672a0 con 0x7fb0dc0ae990 2024-05-08T23:33:15.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.294+0000 7fb0d8ff9700 1 -- 172.21.15.81:0/1573564456 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0c4007720 con 0x7fb0dc0ae990 2024-05-08T23:33:15.296 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-05-08T23:33:15.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.298+0000 7fb0e0b73700 1 -- 172.21.15.81:0/1573564456 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb0bc066730 msgr2=0x7fb0bc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.298+0000 7fb0e0b73700 1 --2- 172.21.15.81:0/1573564456 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb0bc066730 0x7fb0bc068bf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00c9a0 tx=0x7fb0cc008040 comp rx=0 tx=0).stop 2024-05-08T23:33:15.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.298+0000 7fb0e0b73700 1 -- 172.21.15.81:0/1573564456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0dc0ae990 msgr2=0x7fb0dc1f69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.298+0000 7fb0e0b73700 1 --2- 172.21.15.81:0/1573564456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0dc0ae990 0x7fb0dc1f69a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400c580 tx=0x7fb0c400c890 comp rx=0 tx=0).stop 2024-05-08T23:33:15.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.298+0000 7fb0e0b73700 1 -- 172.21.15.81:0/1573564456 shutdown_connections 2024-05-08T23:33:15.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.298+0000 7fb0e0b73700 1 --2- 172.21.15.81:0/1573564456 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb0bc066730 0x7fb0bc068bf0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.298+0000 7fb0e0b73700 1 --2- 172.21.15.81:0/1573564456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0dc15d3b0 0x7fb0dc0c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.298+0000 7fb0e0b73700 1 --2- 172.21.15.81:0/1573564456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0dc0ae990 0x7fb0dc1f69a0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.298+0000 7fb0e0b73700 1 --2- 172.21.15.81:0/1573564456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0dc0ae530 0x7fb0dc1532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.298+0000 7fb0e0b73700 1 -- 172.21.15.81:0/1573564456 >> 172.21.15.81:0/1573564456 conn(0x7fb0dc0cfcc0 msgr2=0x7fb0dc06d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:15.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.298+0000 7fb0e0b73700 1 -- 172.21.15.81:0/1573564456 shutdown_connections 2024-05-08T23:33:15.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.298+0000 7fb0e0b73700 1 -- 172.21.15.81:0/1573564456 wait complete. 2024-05-08T23:33:15.315 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:33:15.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f2064852700 1 Processor -- start 2024-05-08T23:33:15.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f2064852700 1 -- start start 2024-05-08T23:33:15.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f2064852700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2060161470 0x7f20600b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f2064852700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20600b3820 con 0x7f206006c380 2024-05-08T23:33:15.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f2064852700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2060070d20 con 0x7f206006c7e0 2024-05-08T23:33:15.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f2064852700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2060071000 con 0x7f2060161470 2024-05-08T23:33:15.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f2057fff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f206006c380 0x7f2060071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34416/0 (socket says 172.21.15.81:34416) 2024-05-08T23:33:15.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f2057fff700 1 -- 172.21.15.81:0/3085697990 learned_addr learned my addr 172.21.15.81:0/3085697990 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:15.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205f7fe700 1 --2- 172.21.15.81:0/3085697990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2060161470 0x7f20600b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490793610 0 0) 0x7f2048001110 con 0x7f206006c380 2024-05-08T23:33:15.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f204c0035d0 con 0x7f206006c380 2024-05-08T23:33:15.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261036454 0 0) 0x7f2060070d20 con 0x7f206006c7e0 2024-05-08T23:33:15.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2048001110 con 0x7f206006c7e0 2024-05-08T23:33:15.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 95368141 0 0) 0x7f204c0035d0 con 0x7f206006c380 2024-05-08T23:33:15.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2060161470 msgr2=0x7f20600b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:15.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 --2- 172.21.15.81:0/3085697990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2060161470 0x7f20600b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 >> v1:172.21.15.117:6789/0 conn(0x7f206006c7e0 legacy=0x7f20600ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20600ac890 con 0x7f206006c380 2024-05-08T23:33:15.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f2048001800 con 0x7f206006c380 2024-05-08T23:33:15.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 >> v1:172.21.15.81:6789/0 conn(0x7f206006c380 legacy=0x7f2060071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 --2- 172.21.15.81:0/3085697990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f204c003be0 0x7f204c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 --2- 172.21.15.81:0/3085697990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f204c004870 0x7f204c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 --2- 172.21.15.81:0/3085697990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2060161470 0x7f204c005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2060071000 con 0x7f204c003be0 2024-05-08T23:33:15.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c00bb40 con 0x7f204c004870 2024-05-08T23:33:15.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c00bd00 con 0x7f2060161470 2024-05-08T23:33:15.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205f7fe700 1 --2- 172.21.15.81:0/3085697990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f204c003be0 0x7f204c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f2057fff700 1 --2- 172.21.15.81:0/3085697990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f204c004870 0x7f204c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205ffff700 1 --2- 172.21.15.81:0/3085697990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2060161470 0x7f204c005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205f7fe700 1 -- 172.21.15.81:0/3085697990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2060161470 msgr2=0x7f204c005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:15.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205f7fe700 1 --2- 172.21.15.81:0/3085697990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2060161470 0x7f204c005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205f7fe700 1 -- 172.21.15.81:0/3085697990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f204c004870 msgr2=0x7f204c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:15.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205f7fe700 1 --2- 172.21.15.81:0/3085697990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f204c004870 0x7f204c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205f7fe700 1 -- 172.21.15.81:0/3085697990 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c0035d0 con 0x7f204c003be0 2024-05-08T23:33:15.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f2057fff700 1 --2- 172.21.15.81:0/3085697990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f204c004870 0x7f204c004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:33:15.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205ffff700 1 --2- 172.21.15.81:0/3085697990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2060161470 0x7f204c005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:33:15.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205f7fe700 1 --2- 172.21.15.81:0/3085697990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f204c003be0 0x7f204c003fe0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2050009b50 tx=0x7f204c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33069ed72332a6c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:15.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205000e6c0 con 0x7f204c003be0 2024-05-08T23:33:15.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205000e820 con 0x7f204c003be0 2024-05-08T23:33:15.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.430+0000 7f205effd700 1 -- 172.21.15.81:0/3085697990 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205000ebd0 con 0x7f204c003be0 2024-05-08T23:33:15.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2064852700 1 -- 172.21.15.81:0/3085697990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f204c003be0 msgr2=0x7f204c003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2064852700 1 --2- 172.21.15.81:0/3085697990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f204c003be0 0x7f204c003fe0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2050009b50 tx=0x7f204c0058a0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f205ffff700 1 -- 172.21.15.81:0/3085697990 reap_dead start 2024-05-08T23:33:15.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2064852700 1 -- 172.21.15.81:0/3085697990 shutdown_connections 2024-05-08T23:33:15.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2064852700 1 -- 172.21.15.81:0/3085697990 >> 172.21.15.81:0/3085697990 conn(0x7f20600b0de0 msgr2=0x7f20600b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:15.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2064852700 1 -- 172.21.15.81:0/3085697990 shutdown_connections 2024-05-08T23:33:15.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2064852700 1 -- 172.21.15.81:0/3085697990 wait complete. 2024-05-08T23:33:15.440 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2064852700 1 Processor -- start 2024-05-08T23:33:15.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2064852700 1 -- start start 2024-05-08T23:33:15.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2064852700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f204c003be0 0x7f206009afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2064852700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2060099a70 0x7f20600af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2064852700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f206015b8f0 0x7f206015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2064852700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206015c2c0 con 0x7f206015b8f0 2024-05-08T23:33:15.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2064852700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2060099f00 con 0x7f2060099a70 2024-05-08T23:33:15.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2064852700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206009a1e0 con 0x7f204c003be0 2024-05-08T23:33:15.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f205ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f206015b8f0 0x7f206015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f205ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f206015b8f0 0x7f206015bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36942/0 (socket says 172.21.15.81:36942) 2024-05-08T23:33:15.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2057fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2060099a70 0x7f20600af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f205ffff700 1 -- 172.21.15.81:0/3524620167 learned_addr learned my addr 172.21.15.81:0/3524620167 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:15.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f205f7fe700 1 --2- 172.21.15.81:0/3524620167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f204c003be0 0x7f206009afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2057fff700 1 -- 172.21.15.81:0/3524620167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f204c003be0 msgr2=0x7f206009afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2057fff700 1 --2- 172.21.15.81:0/3524620167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f204c003be0 0x7f206009afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2057fff700 1 -- 172.21.15.81:0/3524620167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f206015b8f0 msgr2=0x7f206015bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2057fff700 1 --2- 172.21.15.81:0/3524620167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f206015b8f0 0x7f206015bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2057fff700 1 -- 172.21.15.81:0/3524620167 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206009a4c0 con 0x7f2060099a70 2024-05-08T23:33:15.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.434+0000 7f2057fff700 1 --2- 172.21.15.81:0/3524620167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2060099a70 0x7f20600af4a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2048000c00 tx=0x7f204800ddc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:15.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.438+0000 7f205cff9700 1 -- 172.21.15.81:0/3524620167 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048004ca0 con 0x7f2060099a70 2024-05-08T23:33:15.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.438+0000 7f205cff9700 1 -- 172.21.15.81:0/3524620167 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2048004e00 con 0x7f2060099a70 2024-05-08T23:33:15.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.438+0000 7f205cff9700 1 -- 172.21.15.81:0/3524620167 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048007310 con 0x7f2060099a70 2024-05-08T23:33:15.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.438+0000 7f2064852700 1 -- 172.21.15.81:0/3524620167 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206009a7a0 con 0x7f2060099a70 2024-05-08T23:33:15.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.438+0000 7f2064852700 1 -- 172.21.15.81:0/3524620167 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2060157b20 con 0x7f2060099a70 2024-05-08T23:33:15.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.438+0000 7f2064852700 1 -- 172.21.15.81:0/3524620167 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20601634a0 con 0x7f2060099a70 2024-05-08T23:33:15.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.438+0000 7f205cff9700 1 -- 172.21.15.81:0/3524620167 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2048007470 con 0x7f2060099a70 2024-05-08T23:33:15.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.438+0000 7f205cff9700 1 --2- 172.21.15.81:0/3524620167 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2040066730 0x7f2040068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.438+0000 7f205cff9700 1 -- 172.21.15.81:0/3524620167 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2048083b70 con 0x7f2060099a70 2024-05-08T23:33:15.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.438+0000 7f205f7fe700 1 --2- 172.21.15.81:0/3524620167 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2040066730 0x7f2040068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.442+0000 7f205f7fe700 1 --2- 172.21.15.81:0/3524620167 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2040066730 0x7f2040068bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f20500090b0 tx=0x7f205001f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:15.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.446+0000 7f205cff9700 1 -- 172.21.15.81:0/3524620167 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2048050fc0 con 0x7f2060099a70 2024-05-08T23:33:15.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.614+0000 7f2064852700 1 -- 172.21.15.81:0/3524620167 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20600b3820 con 0x7f2060099a70 2024-05-08T23:33:15.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.614+0000 7f205cff9700 1 -- 172.21.15.81:0/3524620167 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7f20600b3820 con 0x7f2060099a70 2024-05-08T23:33:15.617 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-05-08T23:33:15.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.618+0000 7f2064852700 1 -- 172.21.15.81:0/3524620167 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2040066730 msgr2=0x7f2040068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.618+0000 7f2064852700 1 --2- 172.21.15.81:0/3524620167 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2040066730 0x7f2040068bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f20500090b0 tx=0x7f205001f040 comp rx=0 tx=0).stop 2024-05-08T23:33:15.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.618+0000 7f2064852700 1 -- 172.21.15.81:0/3524620167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2060099a70 msgr2=0x7f20600af4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.618+0000 7f2064852700 1 --2- 172.21.15.81:0/3524620167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2060099a70 0x7f20600af4a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2048000c00 tx=0x7f204800ddc0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.618+0000 7f2064852700 1 -- 172.21.15.81:0/3524620167 shutdown_connections 2024-05-08T23:33:15.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.618+0000 7f2064852700 1 --2- 172.21.15.81:0/3524620167 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2040066730 0x7f2040068bf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.618+0000 7f2064852700 1 --2- 172.21.15.81:0/3524620167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f206015b8f0 0x7f206015bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.618+0000 7f2064852700 1 --2- 172.21.15.81:0/3524620167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2060099a70 0x7f20600af4a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.618+0000 7f2064852700 1 --2- 172.21.15.81:0/3524620167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f204c003be0 0x7f206009afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.618+0000 7f2064852700 1 -- 172.21.15.81:0/3524620167 >> 172.21.15.81:0/3524620167 conn(0x7f20600b0de0 msgr2=0x7f20600cd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:15.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.618+0000 7f2064852700 1 -- 172.21.15.81:0/3524620167 shutdown_connections 2024-05-08T23:33:15.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.618+0000 7f2064852700 1 -- 172.21.15.81:0/3524620167 wait complete. 2024-05-08T23:33:15.632 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:33:15.748 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e3fff700 1 Processor -- start 2024-05-08T23:33:15.748 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e3fff700 1 -- start start 2024-05-08T23:33:15.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e3fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f14e415d180 0x7f14e40afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e3fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f14e40d3510 0x7f14e40b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e3fff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e40b0fa0 con 0x7f14e415d560 2024-05-08T23:33:15.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e3fff700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e40b4990 con 0x7f14e40d3510 2024-05-08T23:33:15.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e3fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e40b0390 con 0x7f14e415d180 2024-05-08T23:33:15.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e37fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f14e40d3510 0x7f14e40b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e37fe700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f14e40d3510 0x7f14e40b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36952/0 (socket says 172.21.15.81:36952) 2024-05-08T23:33:15.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e37fe700 1 -- 172.21.15.81:0/2485368945 learned_addr learned my addr 172.21.15.81:0/2485368945 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:15.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e27fc700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f14e415d560 0x7f14e40d2e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34418/0 (socket says 172.21.15.81:34418) 2024-05-08T23:33:15.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e2ffd700 1 --2- 172.21.15.81:0/2485368945 >> v2:172.21.15.117:3300/0 conn(0x7f14e415d180 0x7f14e40afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e37fe700 1 -- 172.21.15.81:0/2485368945 >> v2:172.21.15.117:3300/0 conn(0x7f14e415d180 msgr2=0x7f14e40afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:15.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e37fe700 1 --2- 172.21.15.81:0/2485368945 >> v2:172.21.15.117:3300/0 conn(0x7f14e415d180 0x7f14e40afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e37fe700 1 -- 172.21.15.81:0/2485368945 >> v1:172.21.15.81:6789/0 conn(0x7f14e415d560 legacy=0x7f14e40d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e37fe700 1 -- 172.21.15.81:0/2485368945 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e40b0670 con 0x7f14e40d3510 2024-05-08T23:33:15.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e37fe700 1 --2- 172.21.15.81:0/2485368945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14e40d3510 0x7f14e40b4450 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f14e408c240 tx=0x7f14dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bcffb1937c24ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:15.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e1ffb700 1 -- 172.21.15.81:0/2485368945 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc00e070 con 0x7f14e40d3510 2024-05-08T23:33:15.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e1ffb700 1 -- 172.21.15.81:0/2485368945 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc00b9c0 con 0x7f14e40d3510 2024-05-08T23:33:15.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e1ffb700 1 -- 172.21.15.81:0/2485368945 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc00bd70 con 0x7f14e40d3510 2024-05-08T23:33:15.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e3fff700 1 -- 172.21.15.81:0/2485368945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14e40d3510 msgr2=0x7f14e40b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e3fff700 1 --2- 172.21.15.81:0/2485368945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14e40d3510 0x7f14e40b4450 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f14e408c240 tx=0x7f14dc00b330 comp rx=0 tx=0).stop 2024-05-08T23:33:15.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e3fff700 1 -- 172.21.15.81:0/2485368945 shutdown_connections 2024-05-08T23:33:15.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e3fff700 1 --2- 172.21.15.81:0/2485368945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14e40d3510 0x7f14e40b4450 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e3fff700 1 --2- 172.21.15.81:0/2485368945 >> v2:172.21.15.117:3300/0 conn(0x7f14e415d180 0x7f14e40afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e3fff700 1 -- 172.21.15.81:0/2485368945 >> 172.21.15.81:0/2485368945 conn(0x7f14e408b000 msgr2=0x7f14e409c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:15.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e3fff700 1 -- 172.21.15.81:0/2485368945 shutdown_connections 2024-05-08T23:33:15.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.746+0000 7f14e3fff700 1 -- 172.21.15.81:0/2485368945 wait complete. 2024-05-08T23:33:15.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.750+0000 7f14e3fff700 1 Processor -- start 2024-05-08T23:33:15.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.750+0000 7f14e3fff700 1 -- start start 2024-05-08T23:33:15.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e3fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e40d3510 0x7f14e415fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14e415d180 0x7f14e40c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e415d560 0x7f14e41523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e3fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e415e790 con 0x7f14e415d180 2024-05-08T23:33:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e3fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e415db50 con 0x7f14e415d560 2024-05-08T23:33:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e3fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e415ddf0 con 0x7f14e40d3510 2024-05-08T23:33:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e27fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14e415d180 0x7f14e40c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e27fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14e415d180 0x7f14e40c8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36966/0 (socket says 172.21.15.81:36966) 2024-05-08T23:33:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e27fc700 1 -- 172.21.15.81:0/966314333 learned_addr learned my addr 172.21.15.81:0/966314333 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e2ffd700 1 --2- 172.21.15.81:0/966314333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e40d3510 0x7f14e415fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e37fe700 1 --2- 172.21.15.81:0/966314333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e415d560 0x7f14e41523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e27fc700 1 -- 172.21.15.81:0/966314333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e40d3510 msgr2=0x7f14e415fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e27fc700 1 --2- 172.21.15.81:0/966314333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e40d3510 0x7f14e415fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e27fc700 1 -- 172.21.15.81:0/966314333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e415d560 msgr2=0x7f14e41523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e27fc700 1 --2- 172.21.15.81:0/966314333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e415d560 0x7f14e41523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e27fc700 1 -- 172.21.15.81:0/966314333 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc009b30 con 0x7f14e415d180 2024-05-08T23:33:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e37fe700 1 --2- 172.21.15.81:0/966314333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e415d560 0x7f14e41523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e27fc700 1 --2- 172.21.15.81:0/966314333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14e415d180 0x7f14e40c8f10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f14d400c4d0 tx=0x7f14d400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:15.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14cbfff700 1 -- 172.21.15.81:0/966314333 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4005cd0 con 0x7f14e415d180 2024-05-08T23:33:15.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14cbfff700 1 -- 172.21.15.81:0/966314333 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d4005e30 con 0x7f14e415d180 2024-05-08T23:33:15.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e3fff700 1 -- 172.21.15.81:0/966314333 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e415e0d0 con 0x7f14e415d180 2024-05-08T23:33:15.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14cbfff700 1 -- 172.21.15.81:0/966314333 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d40072d0 con 0x7f14e415d180 2024-05-08T23:33:15.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e3fff700 1 -- 172.21.15.81:0/966314333 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e4160150 con 0x7f14e415d180 2024-05-08T23:33:15.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.754+0000 7f14e3fff700 1 -- 172.21.15.81:0/966314333 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e40677c0 con 0x7f14e415d180 2024-05-08T23:33:15.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.758+0000 7f14cbfff700 1 -- 172.21.15.81:0/966314333 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f14d4007430 con 0x7f14e415d180 2024-05-08T23:33:15.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.758+0000 7f14cbfff700 1 --2- 172.21.15.81:0/966314333 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f14d0066730 0x7f14d0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:15.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.758+0000 7f14e2ffd700 1 --2- 172.21.15.81:0/966314333 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f14d0066730 0x7f14d0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.758+0000 7f14cbfff700 1 -- 172.21.15.81:0/966314333 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14d4083330 con 0x7f14e415d180 2024-05-08T23:33:15.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.758+0000 7f14e2ffd700 1 --2- 172.21.15.81:0/966314333 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f14d0066730 0x7f14d0068bf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009a00 tx=0x7f14d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:15.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.762+0000 7f14cbfff700 1 -- 172.21.15.81:0/966314333 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f14d40507e0 con 0x7f14e415d180 2024-05-08T23:33:15.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.934+0000 7f14e3fff700 1 -- 172.21.15.81:0/966314333 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f14e415e8d0 con 0x7f14e415d180 2024-05-08T23:33:15.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.934+0000 7f14cbfff700 1 -- 172.21.15.81:0/966314333 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7f14d4017020 con 0x7f14e415d180 2024-05-08T23:33:15.937 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2024-05-08T23:33:15.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.938+0000 7f14e3fff700 1 -- 172.21.15.81:0/966314333 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f14d0066730 msgr2=0x7f14d0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.938+0000 7f14e3fff700 1 --2- 172.21.15.81:0/966314333 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f14d0066730 0x7f14d0068bf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009a00 tx=0x7f14d8008040 comp rx=0 tx=0).stop 2024-05-08T23:33:15.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.938+0000 7f14e3fff700 1 -- 172.21.15.81:0/966314333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14e415d180 msgr2=0x7f14e40c8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:15.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.938+0000 7f14e3fff700 1 --2- 172.21.15.81:0/966314333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14e415d180 0x7f14e40c8f10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f14d400c4d0 tx=0x7f14d400c7e0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.938+0000 7f14e3fff700 1 -- 172.21.15.81:0/966314333 shutdown_connections 2024-05-08T23:33:15.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.938+0000 7f14e3fff700 1 --2- 172.21.15.81:0/966314333 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f14d0066730 0x7f14d0068bf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.938+0000 7f14e3fff700 1 --2- 172.21.15.81:0/966314333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e415d560 0x7f14e41523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.938+0000 7f14e3fff700 1 --2- 172.21.15.81:0/966314333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14e415d180 0x7f14e40c8f10 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.938+0000 7f14e3fff700 1 --2- 172.21.15.81:0/966314333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e40d3510 0x7f14e415fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:15.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.938+0000 7f14e3fff700 1 -- 172.21.15.81:0/966314333 >> 172.21.15.81:0/966314333 conn(0x7f14e408b000 msgr2=0x7f14e40b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:15.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.938+0000 7f14e3fff700 1 -- 172.21.15.81:0/966314333 shutdown_connections 2024-05-08T23:33:15.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:15.938+0000 7f14e3fff700 1 -- 172.21.15.81:0/966314333 wait complete. 2024-05-08T23:33:15.952 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:33:16.065 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbffff700 1 Processor -- start 2024-05-08T23:33:16.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbffff700 1 -- start start 2024-05-08T23:33:16.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbffff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fccc0071e70 0x7fccc01624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fccc0070110 0x7fccc00705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbffff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc0158960 con 0x7fccc00b3c20 2024-05-08T23:33:16.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbffff700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc00b1f60 con 0x7fccc0071e70 2024-05-08T23:33:16.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbffff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc00ae990 con 0x7fccc0070110 2024-05-08T23:33:16.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbe7fc700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fccc0071e70 0x7fccc01624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbe7fc700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fccc0071e70 0x7fccc01624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40030/0 (socket says 172.21.15.81:40030) 2024-05-08T23:33:16.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbe7fc700 1 -- 172.21.15.81:0/3581353720 learned_addr learned my addr 172.21.15.81:0/3581353720 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:16.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbf7fe700 1 --2- 172.21.15.81:0/3581353720 >> v2:172.21.15.117:3300/0 conn(0x7fccc0070110 0x7fccc00705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbe7fc700 1 -- 172.21.15.81:0/3581353720 >> v2:172.21.15.117:3300/0 conn(0x7fccc0070110 msgr2=0x7fccc00705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:16.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbe7fc700 1 --2- 172.21.15.81:0/3581353720 >> v2:172.21.15.117:3300/0 conn(0x7fccc0070110 0x7fccc00705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbe7fc700 1 -- 172.21.15.81:0/3581353720 >> v1:172.21.15.117:6789/0 conn(0x7fccc00b3c20 legacy=0x7fccc00b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbe7fc700 1 -- 172.21.15.81:0/3581353720 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc00b2170 con 0x7fccc0071e70 2024-05-08T23:33:16.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbe7fc700 1 --2- 172.21.15.81:0/3581353720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccc0071e70 0x7fccc01624b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fccc00b51f0 tx=0x7fccb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b37f80ae21ef6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:16.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbdffb700 1 -- 172.21.15.81:0/3581353720 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb800e070 con 0x7fccc0071e70 2024-05-08T23:33:16.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbdffb700 1 -- 172.21.15.81:0/3581353720 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb800b910 con 0x7fccc0071e70 2024-05-08T23:33:16.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbdffb700 1 -- 172.21.15.81:0/3581353720 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb800bcc0 con 0x7fccc0071e70 2024-05-08T23:33:16.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbffff700 1 -- 172.21.15.81:0/3581353720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccc0071e70 msgr2=0x7fccc01624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbffff700 1 --2- 172.21.15.81:0/3581353720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccc0071e70 0x7fccc01624b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fccc00b51f0 tx=0x7fccb800b330 comp rx=0 tx=0).stop 2024-05-08T23:33:16.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbffff700 1 -- 172.21.15.81:0/3581353720 shutdown_connections 2024-05-08T23:33:16.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbffff700 1 --2- 172.21.15.81:0/3581353720 >> v2:172.21.15.117:3300/0 conn(0x7fccc0070110 0x7fccc00705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbffff700 1 --2- 172.21.15.81:0/3581353720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccc0071e70 0x7fccc01624b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbffff700 1 -- 172.21.15.81:0/3581353720 >> 172.21.15.81:0/3581353720 conn(0x7fccc006c330 msgr2=0x7fccc006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:16.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbffff700 1 -- 172.21.15.81:0/3581353720 shutdown_connections 2024-05-08T23:33:16.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.062+0000 7fccbffff700 1 -- 172.21.15.81:0/3581353720 wait complete. 2024-05-08T23:33:16.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbffff700 1 Processor -- start 2024-05-08T23:33:16.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbffff700 1 -- start start 2024-05-08T23:33:16.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc0070110 0x7fccc01fb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccc0071e70 0x7fccc01654c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc00b3c20 0x7fccc00a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbffff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc00ad440 con 0x7fccc0071e70 2024-05-08T23:33:16.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc0165a00 con 0x7fccc0070110 2024-05-08T23:33:16.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc00ac830 con 0x7fccc00b3c20 2024-05-08T23:33:16.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbe7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccc0071e70 0x7fccc01654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbe7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccc0071e70 0x7fccc01654c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40032/0 (socket says 172.21.15.81:40032) 2024-05-08T23:33:16.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbe7fc700 1 -- 172.21.15.81:0/1968182312 learned_addr learned my addr 172.21.15.81:0/1968182312 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:16.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbeffd700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc0070110 0x7fccc01fb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbf7fe700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc00b3c20 0x7fccc00a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbe7fc700 1 -- 172.21.15.81:0/1968182312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc00b3c20 msgr2=0x7fccc00a8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbe7fc700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc00b3c20 0x7fccc00a8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbe7fc700 1 -- 172.21.15.81:0/1968182312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc0070110 msgr2=0x7fccc01fb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbe7fc700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc0070110 0x7fccc01fb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbe7fc700 1 -- 172.21.15.81:0/1968182312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb8009b30 con 0x7fccc0071e70 2024-05-08T23:33:16.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbeffd700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc0070110 0x7fccc01fb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:16.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbf7fe700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc00b3c20 0x7fccc00a8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:33:16.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccbe7fc700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccc0071e70 0x7fccc01654c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fccc006b120 tx=0x7fccb8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:16.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccb7fff700 1 -- 172.21.15.81:0/1968182312 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb80099b0 con 0x7fccc0071e70 2024-05-08T23:33:16.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.066+0000 7fccb7fff700 1 -- 172.21.15.81:0/1968182312 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb80092e0 con 0x7fccc0071e70 2024-05-08T23:33:16.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.070+0000 7fccbffff700 1 -- 172.21.15.81:0/1968182312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc00acb10 con 0x7fccc0071e70 2024-05-08T23:33:16.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.070+0000 7fccb7fff700 1 -- 172.21.15.81:0/1968182312 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb801b650 con 0x7fccc0071e70 2024-05-08T23:33:16.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.070+0000 7fccbffff700 1 -- 172.21.15.81:0/1968182312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc00ad0d0 con 0x7fccc0071e70 2024-05-08T23:33:16.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.070+0000 7fccbffff700 1 -- 172.21.15.81:0/1968182312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc00ad5f0 con 0x7fccc0071e70 2024-05-08T23:33:16.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.070+0000 7fccb7fff700 1 -- 172.21.15.81:0/1968182312 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fccb801b7b0 con 0x7fccc0071e70 2024-05-08T23:33:16.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.070+0000 7fccb7fff700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcca0066760 0x7fcca0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.070+0000 7fccb7fff700 1 -- 172.21.15.81:0/1968182312 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fccb80856a0 con 0x7fccc0071e70 2024-05-08T23:33:16.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.070+0000 7fccbeffd700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcca0066760 0x7fcca0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.075 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.070+0000 7fccbeffd700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcca0066760 0x7fcca0068c20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcca8000f60 tx=0x7fcca8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:16.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.074+0000 7fccb7fff700 1 -- 172.21.15.81:0/1968182312 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fccc00ad5f0 con 0x7fccc0071e70 2024-05-08T23:33:16.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.282+0000 7fccbffff700 1 -- 172.21.15.81:0/1968182312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccc00684b0 con 0x7fccc0071e70 2024-05-08T23:33:16.287 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.286+0000 7fccb7fff700 1 -- 172.21.15.81:0/1968182312 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4743 (secure 0 0 0) 0x7fccb8052a80 con 0x7fccc0071e70 2024-05-08T23:33:16.287 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:16.287 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/2105335102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":2105335102},{"type":"v1","addr":"172.21.15.117:6834","nonce":2105335102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2900127785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2900127785},{"type":"v1","addr":"172.21.15.81:6835","nonce":2900127785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1899281779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1899281779},{"type":"v1","addr":"172.21.15.81:6837","nonce":1899281779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:32:40.852853+0000","modified":"2024-05-08T23:32:43.867414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":15311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_15311":{"gid":15311,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/3401411259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3401411259},{"type":"v1","addr":"172.21.15.117:6835","nonce":3401411259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:33:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.290+0000 7fccbffff700 1 -- 172.21.15.81:0/1968182312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcca0066760 msgr2=0x7fcca0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.290+0000 7fccbffff700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcca0066760 0x7fcca0068c20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcca8000f60 tx=0x7fcca8009040 comp rx=0 tx=0).stop 2024-05-08T23:33:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.290+0000 7fccbffff700 1 -- 172.21.15.81:0/1968182312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccc0071e70 msgr2=0x7fccc01654c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.290+0000 7fccbffff700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccc0071e70 0x7fccc01654c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fccc006b120 tx=0x7fccb8004af0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.290+0000 7fccbffff700 1 -- 172.21.15.81:0/1968182312 shutdown_connections 2024-05-08T23:33:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.290+0000 7fccbffff700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fcca0066760 0x7fcca0068c20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.290+0000 7fccbffff700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccc00b3c20 0x7fccc00a8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.290+0000 7fccbffff700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccc0071e70 0x7fccc01654c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.290+0000 7fccbffff700 1 --2- 172.21.15.81:0/1968182312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc0070110 0x7fccc01fb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.290+0000 7fccbffff700 1 -- 172.21.15.81:0/1968182312 >> 172.21.15.81:0/1968182312 conn(0x7fccc006c330 msgr2=0x7fccc00aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.290+0000 7fccbffff700 1 -- 172.21.15.81:0/1968182312 shutdown_connections 2024-05-08T23:33:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.290+0000 7fccbffff700 1 -- 172.21.15.81:0/1968182312 wait complete. 2024-05-08T23:33:16.293 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 23 2024-05-08T23:33:16.307 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:33:16.424 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f0575639700 1 Processor -- start 2024-05-08T23:33:16.424 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f0575639700 1 -- start start 2024-05-08T23:33:16.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f0575639700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0570165240 0x7f057006c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f0575639700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0570147d20 con 0x7f05700cb3b0 2024-05-08T23:33:16.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f0575639700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05701656b0 con 0x7f05701478c0 2024-05-08T23:33:16.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f0575639700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0570161c50 con 0x7f0570165240 2024-05-08T23:33:16.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056effd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f05700cb3b0 0x7f0570162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46098/0 (socket says 172.21.15.81:46098) 2024-05-08T23:33:16.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056effd700 1 -- 172.21.15.81:0/3620400446 learned_addr learned my addr 172.21.15.81:0/3620400446 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:16.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056f7fe700 1 --2- 172.21.15.81:0/3620400446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0570165240 0x7f057006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 -- 172.21.15.81:0/3620400446 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880831980 0 0) 0x7f0564001110 con 0x7f05700cb3b0 2024-05-08T23:33:16.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 -- 172.21.15.81:0/3620400446 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05580035d0 con 0x7f05700cb3b0 2024-05-08T23:33:16.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 -- 172.21.15.81:0/3620400446 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 305330052 0 0) 0x7f05580035d0 con 0x7f05700cb3b0 2024-05-08T23:33:16.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 -- 172.21.15.81:0/3620400446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0570165240 msgr2=0x7f057006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:16.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 --2- 172.21.15.81:0/3620400446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0570165240 0x7f057006c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 -- 172.21.15.81:0/3620400446 >> v1:172.21.15.117:6789/0 conn(0x7f05701478c0 legacy=0x7f0570144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 -- 172.21.15.81:0/3620400446 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0570165970 con 0x7f05700cb3b0 2024-05-08T23:33:16.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 -- 172.21.15.81:0/3620400446 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7f0564001800 con 0x7f05700cb3b0 2024-05-08T23:33:16.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 -- 172.21.15.81:0/3620400446 >> v1:172.21.15.81:6789/0 conn(0x7f05700cb3b0 legacy=0x7f0570162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 --2- 172.21.15.81:0/3620400446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0558003a50 0x7f0558003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 --2- 172.21.15.81:0/3620400446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0570165240 0x7f0558004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 --2- 172.21.15.81:0/3620400446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0558004db0 0x7f0558005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 -- 172.21.15.81:0/3620400446 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055800ba30 con 0x7f0558003a50 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 -- 172.21.15.81:0/3620400446 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055800bb70 con 0x7f0558004db0 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 -- 172.21.15.81:0/3620400446 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055800bd30 con 0x7f0570165240 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056effd700 1 --2- 172.21.15.81:0/3620400446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0558003a50 0x7f0558003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056e7fc700 1 --2- 172.21.15.81:0/3620400446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0570165240 0x7f0558004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056f7fe700 1 --2- 172.21.15.81:0/3620400446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0558004db0 0x7f0558005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056effd700 1 -- 172.21.15.81:0/3620400446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0570165240 msgr2=0x7f0558004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056effd700 1 --2- 172.21.15.81:0/3620400446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0570165240 0x7f0558004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056effd700 1 -- 172.21.15.81:0/3620400446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0558004db0 msgr2=0x7f0558005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056effd700 1 --2- 172.21.15.81:0/3620400446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0558004db0 0x7f0558005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056effd700 1 -- 172.21.15.81:0/3620400446 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05580035d0 con 0x7f0558003a50 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056effd700 1 --2- 172.21.15.81:0/3620400446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0558003a50 0x7f0558003e30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f056400c320 tx=0x7f05580058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ff7116fab841a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 -- 172.21.15.81:0/3620400446 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056400cda0 con 0x7f0558003a50 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 -- 172.21.15.81:0/3620400446 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0564005980 con 0x7f0558003a50 2024-05-08T23:33:16.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.422+0000 7f056dffb700 1 -- 172.21.15.81:0/3620400446 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0564005d30 con 0x7f0558003a50 2024-05-08T23:33:16.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f0575639700 1 -- 172.21.15.81:0/3620400446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0558003a50 msgr2=0x7f0558003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f0575639700 1 --2- 172.21.15.81:0/3620400446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0558003a50 0x7f0558003e30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f056400c320 tx=0x7f05580058a0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f056f7fe700 1 -- 172.21.15.81:0/3620400446 reap_dead start 2024-05-08T23:33:16.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f0575639700 1 -- 172.21.15.81:0/3620400446 shutdown_connections 2024-05-08T23:33:16.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f0575639700 1 -- 172.21.15.81:0/3620400446 >> 172.21.15.81:0/3620400446 conn(0x7f057006a360 msgr2=0x7f057006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:16.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f0575639700 1 -- 172.21.15.81:0/3620400446 shutdown_connections 2024-05-08T23:33:16.430 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f0575639700 1 -- 172.21.15.81:0/3620400446 wait complete. 2024-05-08T23:33:16.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f0575639700 1 Processor -- start 2024-05-08T23:33:16.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f0575639700 1 -- start start 2024-05-08T23:33:16.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f0575639700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0558003a50 0x7f0570143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f0575639700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0570147ec0 0x7f0570148340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f0575639700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05700b03e0 0x7f05700b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f0575639700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057014b230 con 0x7f05700b03e0 2024-05-08T23:33:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f0575639700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05700b0da0 con 0x7f0570147ec0 2024-05-08T23:33:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f0575639700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0570158f80 con 0x7f0558003a50 2024-05-08T23:33:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f056f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05700b03e0 0x7f05700b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f056e7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0570147ec0 0x7f0570148340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f056f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05700b03e0 0x7f05700b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40056/0 (socket says 172.21.15.81:40056) 2024-05-08T23:33:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f056f7fe700 1 -- 172.21.15.81:0/398647568 learned_addr learned my addr 172.21.15.81:0/398647568 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.426+0000 7f056effd700 1 --2- 172.21.15.81:0/398647568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0558003a50 0x7f0570143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.430+0000 7f056e7fc700 1 -- 172.21.15.81:0/398647568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0558003a50 msgr2=0x7f0570143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.430+0000 7f056e7fc700 1 --2- 172.21.15.81:0/398647568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0558003a50 0x7f0570143f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.430+0000 7f056e7fc700 1 -- 172.21.15.81:0/398647568 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05700b03e0 msgr2=0x7f05700b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.430+0000 7f056e7fc700 1 --2- 172.21.15.81:0/398647568 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05700b03e0 0x7f05700b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.430+0000 7f056e7fc700 1 -- 172.21.15.81:0/398647568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057014a990 con 0x7f0570147ec0 2024-05-08T23:33:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.430+0000 7f056f7fe700 1 --2- 172.21.15.81:0/398647568 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05700b03e0 0x7f05700b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:16.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.430+0000 7f056e7fc700 1 --2- 172.21.15.81:0/398647568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0570147ec0 0x7f0570148340 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f056000f7f0 tx=0x7f056000fb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:16.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.430+0000 7f05577fe700 1 -- 172.21.15.81:0/398647568 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05600052e0 con 0x7f0570147ec0 2024-05-08T23:33:16.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.430+0000 7f0575639700 1 -- 172.21.15.81:0/398647568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057014ac70 con 0x7f0570147ec0 2024-05-08T23:33:16.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.430+0000 7f05577fe700 1 -- 172.21.15.81:0/398647568 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0560005440 con 0x7f0570147ec0 2024-05-08T23:33:16.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.430+0000 7f05577fe700 1 -- 172.21.15.81:0/398647568 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05600038c0 con 0x7f0570147ec0 2024-05-08T23:33:16.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.430+0000 7f0575639700 1 -- 172.21.15.81:0/398647568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057014a640 con 0x7f0570147ec0 2024-05-08T23:33:16.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.430+0000 7f0575639700 1 -- 172.21.15.81:0/398647568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05700684b0 con 0x7f0570147ec0 2024-05-08T23:33:16.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.430+0000 7f05577fe700 1 -- 172.21.15.81:0/398647568 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0560003a20 con 0x7f0570147ec0 2024-05-08T23:33:16.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.434+0000 7f05577fe700 1 --2- 172.21.15.81:0/398647568 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f055c066730 0x7f055c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.434+0000 7f05577fe700 1 -- 172.21.15.81:0/398647568 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0560084820 con 0x7f0570147ec0 2024-05-08T23:33:16.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.434+0000 7f056effd700 1 --2- 172.21.15.81:0/398647568 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f055c066730 0x7f055c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.434+0000 7f056effd700 1 --2- 172.21.15.81:0/398647568 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f055c066730 0x7f055c068bf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0564015040 tx=0x7f0564007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:16.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.442+0000 7f05577fe700 1 -- 172.21.15.81:0/398647568 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0560051c20 con 0x7f0570147ec0 2024-05-08T23:33:16.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.610+0000 7f0575639700 1 -- 172.21.15.81:0/398647568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0570158d70 con 0x7f0570147ec0 2024-05-08T23:33:16.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.614+0000 7f05577fe700 1 -- 172.21.15.81:0/398647568 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f0560084070 con 0x7f0570147ec0 2024-05-08T23:33:16.618 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:16.618 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":52,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:32:41.855853+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:39.161796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:39.853963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:33:16.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.618+0000 7f0575639700 1 -- 172.21.15.81:0/398647568 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f055c066730 msgr2=0x7f055c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.618+0000 7f0575639700 1 --2- 172.21.15.81:0/398647568 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f055c066730 0x7f055c068bf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0564015040 tx=0x7f0564007000 comp rx=0 tx=0).stop 2024-05-08T23:33:16.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.618+0000 7f0575639700 1 -- 172.21.15.81:0/398647568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0570147ec0 msgr2=0x7f0570148340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.618+0000 7f0575639700 1 --2- 172.21.15.81:0/398647568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0570147ec0 0x7f0570148340 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f056000f7f0 tx=0x7f056000fb00 comp rx=0 tx=0).stop 2024-05-08T23:33:16.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.618+0000 7f0575639700 1 -- 172.21.15.81:0/398647568 shutdown_connections 2024-05-08T23:33:16.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.618+0000 7f0575639700 1 --2- 172.21.15.81:0/398647568 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f055c066730 0x7f055c068bf0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.618+0000 7f0575639700 1 --2- 172.21.15.81:0/398647568 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05700b03e0 0x7f05700b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.618+0000 7f0575639700 1 --2- 172.21.15.81:0/398647568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0570147ec0 0x7f0570148340 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.618+0000 7f0575639700 1 --2- 172.21.15.81:0/398647568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0558003a50 0x7f0570143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.618+0000 7f0575639700 1 -- 172.21.15.81:0/398647568 >> 172.21.15.81:0/398647568 conn(0x7f057006a360 msgr2=0x7f0570162b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:16.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.618+0000 7f0575639700 1 -- 172.21.15.81:0/398647568 shutdown_connections 2024-05-08T23:33:16.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.618+0000 7f0575639700 1 -- 172.21.15.81:0/398647568 wait complete. 2024-05-08T23:33:16.638 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T23:33:16.638 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:33:16.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd06fb1700 1 Processor -- start 2024-05-08T23:33:16.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd06fb1700 1 -- start start 2024-05-08T23:33:16.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd06fb1700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdd0015ac70 0x7fdd0013d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd06fb1700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd000ce080 con 0x7fdd000ce510 2024-05-08T23:33:16.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd06fb1700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd000cd7c0 con 0x7fdd00071ab0 2024-05-08T23:33:16.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd06fb1700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd000cda30 con 0x7fdd0015ac70 2024-05-08T23:33:16.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd057ae700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fdd000ce510 0x7fdd0015a560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46108/0 (socket says 172.21.15.81:46108) 2024-05-08T23:33:16.754 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd057ae700 1 -- 172.21.15.81:0/3619388437 learned_addr learned my addr 172.21.15.81:0/3619388437 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:16.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd067b0700 1 --2- 172.21.15.81:0/3619388437 >> v2:172.21.15.117:3300/0 conn(0x7fdd0015ac70 0x7fdd0013d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd04fad700 1 -- 172.21.15.81:0/3619388437 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812871215 0 0) 0x7fdd000ce080 con 0x7fdd000ce510 2024-05-08T23:33:16.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd04fad700 1 -- 172.21.15.81:0/3619388437 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce80035d0 con 0x7fdd000ce510 2024-05-08T23:33:16.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd04fad700 1 -- 172.21.15.81:0/3619388437 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4137090054 0 0) 0x7fdce80035d0 con 0x7fdd000ce510 2024-05-08T23:33:16.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd04fad700 1 -- 172.21.15.81:0/3619388437 >> v2:172.21.15.117:3300/0 conn(0x7fdd0015ac70 msgr2=0x7fdd0013d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:16.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd04fad700 1 --2- 172.21.15.81:0/3619388437 >> v2:172.21.15.117:3300/0 conn(0x7fdd0015ac70 0x7fdd0013d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd04fad700 1 -- 172.21.15.81:0/3619388437 >> v1:172.21.15.117:6789/0 conn(0x7fdd00071ab0 legacy=0x7fdd00071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd04fad700 1 -- 172.21.15.81:0/3619388437 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd000cdd10 con 0x7fdd000ce510 2024-05-08T23:33:16.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd04fad700 1 -- 172.21.15.81:0/3619388437 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7fdcf8001670 con 0x7fdd000ce510 2024-05-08T23:33:16.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd04fad700 1 -- 172.21.15.81:0/3619388437 >> v1:172.21.15.81:6789/0 conn(0x7fdd000ce510 legacy=0x7fdd0015a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd04fad700 1 --2- 172.21.15.81:0/3619388437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdce8003a20 0x7fdce8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd04fad700 1 --2- 172.21.15.81:0/3619388437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8004870 0x7fdce8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd04fad700 1 --2- 172.21.15.81:0/3619388437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce8005230 0x7fdce800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd04fad700 1 -- 172.21.15.81:0/3619388437 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd000cda30 con 0x7fdce8003a20 2024-05-08T23:33:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd04fad700 1 -- 172.21.15.81:0/3619388437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce800c0d0 con 0x7fdce8004870 2024-05-08T23:33:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.750+0000 7fdd04fad700 1 -- 172.21.15.81:0/3619388437 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce800c290 con 0x7fdce8005230 2024-05-08T23:33:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd05faf700 1 --2- 172.21.15.81:0/3619388437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdce8003a20 0x7fdce8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd057ae700 1 --2- 172.21.15.81:0/3619388437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8004870 0x7fdce8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd067b0700 1 --2- 172.21.15.81:0/3619388437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce8005230 0x7fdce800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd057ae700 1 -- 172.21.15.81:0/3619388437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce8005230 msgr2=0x7fdce800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd057ae700 1 --2- 172.21.15.81:0/3619388437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce8005230 0x7fdce800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd057ae700 1 -- 172.21.15.81:0/3619388437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdce8003a20 msgr2=0x7fdce8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd057ae700 1 --2- 172.21.15.81:0/3619388437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdce8003a20 0x7fdce8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd067b0700 1 -- 172.21.15.81:0/3619388437 reap_dead start 2024-05-08T23:33:16.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd057ae700 1 -- 172.21.15.81:0/3619388437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce80035d0 con 0x7fdce8004870 2024-05-08T23:33:16.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd057ae700 1 --2- 172.21.15.81:0/3619388437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8004870 0x7fdce8004cf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c320 tx=0x7fdce80058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9602675897caded server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:16.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd04fad700 1 -- 172.21.15.81:0/3619388437 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8007c30 con 0x7fdce8004870 2024-05-08T23:33:16.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd04fad700 1 -- 172.21.15.81:0/3619388437 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf800ce00 con 0x7fdce8004870 2024-05-08T23:33:16.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd04fad700 1 -- 172.21.15.81:0/3619388437 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8005bc0 con 0x7fdce8004870 2024-05-08T23:33:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd06fb1700 1 -- 172.21.15.81:0/3619388437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8004870 msgr2=0x7fdce8004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd06fb1700 1 --2- 172.21.15.81:0/3619388437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8004870 0x7fdce8004cf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c320 tx=0x7fdce80058a0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd06fb1700 1 -- 172.21.15.81:0/3619388437 shutdown_connections 2024-05-08T23:33:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd06fb1700 1 --2- 172.21.15.81:0/3619388437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8004870 0x7fdce8004cf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd06fb1700 1 -- 172.21.15.81:0/3619388437 >> 172.21.15.81:0/3619388437 conn(0x7fdd0006c330 msgr2=0x7fdd0006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd06fb1700 1 -- 172.21.15.81:0/3619388437 shutdown_connections 2024-05-08T23:33:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.754+0000 7fdd06fb1700 1 -- 172.21.15.81:0/3619388437 wait complete. 2024-05-08T23:33:16.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.758+0000 7fdd06fb1700 1 Processor -- start 2024-05-08T23:33:16.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.758+0000 7fdd06fb1700 1 -- start start 2024-05-08T23:33:16.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.758+0000 7fdd06fb1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8004870 0x7fdd000ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.758+0000 7fdd06fb1700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd00153050 0x7fdd000b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.758+0000 7fdd06fb1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd001534d0 0x7fdd000b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.758+0000 7fdd06fb1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd00134550 con 0x7fdd001534d0 2024-05-08T23:33:16.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.758+0000 7fdd06fb1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd00134830 con 0x7fdce8004870 2024-05-08T23:33:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.758+0000 7fdd06fb1700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd00134b10 con 0x7fdd00153050 2024-05-08T23:33:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd067b0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd001534d0 0x7fdd000b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd067b0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd001534d0 0x7fdd000b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40070/0 (socket says 172.21.15.81:40070) 2024-05-08T23:33:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd067b0700 1 -- 172.21.15.81:0/1764258235 learned_addr learned my addr 172.21.15.81:0/1764258235 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd057ae700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd00153050 0x7fdd000b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd05faf700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8004870 0x7fdd000ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd067b0700 1 -- 172.21.15.81:0/1764258235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd00153050 msgr2=0x7fdd000b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd067b0700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd00153050 0x7fdd000b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd067b0700 1 -- 172.21.15.81:0/1764258235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8004870 msgr2=0x7fdd000ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd067b0700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8004870 0x7fdd000ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd067b0700 1 -- 172.21.15.81:0/1764258235 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd00134df0 con 0x7fdd001534d0 2024-05-08T23:33:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd05faf700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8004870 0x7fdd000ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd057ae700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd00153050 0x7fdd000b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd067b0700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd001534d0 0x7fdd000b4010 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fdd0004f630 tx=0x7fdcf0011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdcf67fc700 1 -- 172.21.15.81:0/1764258235 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000e040 con 0x7fdd001534d0 2024-05-08T23:33:16.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdcf67fc700 1 -- 172.21.15.81:0/1764258235 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf000f040 con 0x7fdd001534d0 2024-05-08T23:33:16.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd06fb1700 1 -- 172.21.15.81:0/1764258235 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd001350d0 con 0x7fdd001534d0 2024-05-08T23:33:16.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdcf67fc700 1 -- 172.21.15.81:0/1764258235 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000fd00 con 0x7fdd001534d0 2024-05-08T23:33:16.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd06fb1700 1 -- 172.21.15.81:0/1764258235 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd000adf00 con 0x7fdd001534d0 2024-05-08T23:33:16.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdd06fb1700 1 -- 172.21.15.81:0/1764258235 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd001352f0 con 0x7fdd001534d0 2024-05-08T23:33:16.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdcf67fc700 1 -- 172.21.15.81:0/1764258235 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdcf0015430 con 0x7fdd001534d0 2024-05-08T23:33:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdcf67fc700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdcec066730 0x7fdcec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.762+0000 7fdcf67fc700 1 -- 172.21.15.81:0/1764258235 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdcf0084940 con 0x7fdd001534d0 2024-05-08T23:33:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.766+0000 7fdd05faf700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdcec066730 0x7fdcec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.766+0000 7fdd05faf700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdcec066730 0x7fdcec068bf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdd000ae430 tx=0x7fdcfc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:16.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.770+0000 7fdcf67fc700 1 -- 172.21.15.81:0/1764258235 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdcf0051df0 con 0x7fdd001534d0 2024-05-08T23:33:16.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.974+0000 7fdd06fb1700 1 -- 172.21.15.81:0/1764258235 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd000684b0 con 0x7fdd001534d0 2024-05-08T23:33:16.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.974+0000 7fdcf67fc700 1 -- 172.21.15.81:0/1764258235 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4743 (secure 0 0 0) 0x7fdcf0055410 con 0x7fdd001534d0 2024-05-08T23:33:16.978 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:16.978 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/2105335102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":2105335102},{"type":"v1","addr":"172.21.15.117:6834","nonce":2105335102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2900127785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2900127785},{"type":"v1","addr":"172.21.15.81:6835","nonce":2900127785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1899281779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1899281779},{"type":"v1","addr":"172.21.15.81:6837","nonce":1899281779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:32:40.852853+0000","modified":"2024-05-08T23:32:43.867414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":15311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_15311":{"gid":15311,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/3401411259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3401411259},{"type":"v1","addr":"172.21.15.117:6835","nonce":3401411259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:33:16.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.978+0000 7fdd06fb1700 1 -- 172.21.15.81:0/1764258235 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdcec066730 msgr2=0x7fdcec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.978+0000 7fdd06fb1700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdcec066730 0x7fdcec068bf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdd000ae430 tx=0x7fdcfc009040 comp rx=0 tx=0).stop 2024-05-08T23:33:16.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.978+0000 7fdd06fb1700 1 -- 172.21.15.81:0/1764258235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd001534d0 msgr2=0x7fdd000b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.978+0000 7fdd06fb1700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd001534d0 0x7fdd000b4010 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fdd0004f630 tx=0x7fdcf0011ed0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.978+0000 7fdd06fb1700 1 -- 172.21.15.81:0/1764258235 shutdown_connections 2024-05-08T23:33:16.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.978+0000 7fdd06fb1700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdcec066730 0x7fdcec068bf0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.978+0000 7fdd06fb1700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd001534d0 0x7fdd000b4010 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.978+0000 7fdd06fb1700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd00153050 0x7fdd000b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.978+0000 7fdd06fb1700 1 --2- 172.21.15.81:0/1764258235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8004870 0x7fdd000ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.978+0000 7fdd06fb1700 1 -- 172.21.15.81:0/1764258235 >> 172.21.15.81:0/1764258235 conn(0x7fdd0006c330 msgr2=0x7fdd0015ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:16.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.978+0000 7fdd06fb1700 1 -- 172.21.15.81:0/1764258235 shutdown_connections 2024-05-08T23:33:16.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:16.978+0000 7fdd06fb1700 1 -- 172.21.15.81:0/1764258235 wait complete. 2024-05-08T23:33:16.982 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 23 2024-05-08T23:33:16.994 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:33:17.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68bac97700 1 Processor -- start 2024-05-08T23:33:17.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68bac97700 1 -- start start 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68bac97700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b40ce990 0x7f68b415f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68bac97700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f68b415f770 0x7f68b40b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68bac97700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b415eda0 con 0x7f68b415afa0 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68bac97700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b415ea70 con 0x7f68b415f770 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68bac97700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b415ba90 con 0x7f68b40ce990 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68ba496700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f68b415f770 0x7f68b40b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68ba496700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f68b415f770 0x7f68b40b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40084/0 (socket says 172.21.15.81:40084) 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68ba496700 1 -- 172.21.15.81:0/3249082985 learned_addr learned my addr 172.21.15.81:0/3249082985 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68b9494700 1 --2- 172.21.15.81:0/3249082985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b40ce990 0x7f68b415f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68ba496700 1 -- 172.21.15.81:0/3249082985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b40ce990 msgr2=0x7f68b415f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68ba496700 1 --2- 172.21.15.81:0/3249082985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b40ce990 0x7f68b415f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68ba496700 1 -- 172.21.15.81:0/3249082985 >> v1:172.21.15.117:6789/0 conn(0x7f68b415afa0 legacy=0x7f68b415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68ba496700 1 -- 172.21.15.81:0/3249082985 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b415bd40 con 0x7f68b415f770 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68ba496700 1 --2- 172.21.15.81:0/3249082985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68b415f770 0x7f68b40b4270 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009910 tx=0x7f68ac009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f03cd3fa92bd8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68b8c93700 1 -- 172.21.15.81:0/3249082985 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac004230 con 0x7f68b415f770 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68b8c93700 1 -- 172.21.15.81:0/3249082985 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ac004390 con 0x7f68b415f770 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68b8c93700 1 -- 172.21.15.81:0/3249082985 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac009e60 con 0x7f68b415f770 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68bac97700 1 -- 172.21.15.81:0/3249082985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68b415f770 msgr2=0x7f68b40b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68bac97700 1 --2- 172.21.15.81:0/3249082985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68b415f770 0x7f68b40b4270 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009910 tx=0x7f68ac009c20 comp rx=0 tx=0).stop 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68bac97700 1 -- 172.21.15.81:0/3249082985 shutdown_connections 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68bac97700 1 --2- 172.21.15.81:0/3249082985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68b415f770 0x7f68b40b4270 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68bac97700 1 --2- 172.21.15.81:0/3249082985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b40ce990 0x7f68b415f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.110+0000 7f68bac97700 1 -- 172.21.15.81:0/3249082985 >> 172.21.15.81:0/3249082985 conn(0x7f68b406c330 msgr2=0x7f68b406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68bac97700 1 -- 172.21.15.81:0/3249082985 shutdown_connections 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68bac97700 1 -- 172.21.15.81:0/3249082985 wait complete. 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68bac97700 1 Processor -- start 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68bac97700 1 -- start start 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68bac97700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68b40ce990 0x7f68b40ad610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68bac97700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b415afa0 0x7f68b414c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68bac97700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68b415f770 0x7f68b4143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68bac97700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40ad1f0 con 0x7f68b415f770 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68bac97700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40adb50 con 0x7f68b40ce990 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68bac97700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40ac8e0 con 0x7f68b415afa0 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68ba496700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68b415f770 0x7f68b4143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68ba496700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68b415f770 0x7f68b4143520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40090/0 (socket says 172.21.15.81:40090) 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68ba496700 1 -- 172.21.15.81:0/3499625594 learned_addr learned my addr 172.21.15.81:0/3499625594 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68b9c95700 1 --2- 172.21.15.81:0/3499625594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68b40ce990 0x7f68b40ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68b9494700 1 --2- 172.21.15.81:0/3499625594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b415afa0 0x7f68b414c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68ba496700 1 -- 172.21.15.81:0/3499625594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b415afa0 msgr2=0x7f68b414c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68ba496700 1 --2- 172.21.15.81:0/3499625594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b415afa0 0x7f68b414c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68ba496700 1 -- 172.21.15.81:0/3499625594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68b40ce990 msgr2=0x7f68b40ad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68ba496700 1 --2- 172.21.15.81:0/3499625594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68b40ce990 0x7f68b40ad610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68ba496700 1 -- 172.21.15.81:0/3499625594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac009570 con 0x7f68b415f770 2024-05-08T23:33:17.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68b9c95700 1 --2- 172.21.15.81:0/3499625594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68b40ce990 0x7f68b40ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:17.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68ba496700 1 --2- 172.21.15.81:0/3499625594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68b415f770 0x7f68b4143520 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009910 tx=0x7f68ac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.114+0000 7f68a67fc700 1 -- 172.21.15.81:0/3499625594 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00bca0 con 0x7f68b415f770 2024-05-08T23:33:17.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.118+0000 7f68a67fc700 1 -- 172.21.15.81:0/3499625594 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ac00be20 con 0x7f68b415f770 2024-05-08T23:33:17.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.118+0000 7f68bac97700 1 -- 172.21.15.81:0/3499625594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b414bb30 con 0x7f68b415f770 2024-05-08T23:33:17.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.118+0000 7f68a67fc700 1 -- 172.21.15.81:0/3499625594 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac003900 con 0x7f68b415f770 2024-05-08T23:33:17.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.118+0000 7f68bac97700 1 -- 172.21.15.81:0/3499625594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b40ae7e0 con 0x7f68b415f770 2024-05-08T23:33:17.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.118+0000 7f68bac97700 1 -- 172.21.15.81:0/3499625594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b40cd530 con 0x7f68b415f770 2024-05-08T23:33:17.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.118+0000 7f68a67fc700 1 -- 172.21.15.81:0/3499625594 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f68ac021070 con 0x7f68b415f770 2024-05-08T23:33:17.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.118+0000 7f68a67fc700 1 --2- 172.21.15.81:0/3499625594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6898066760 0x7f6898068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.118+0000 7f68a67fc700 1 -- 172.21.15.81:0/3499625594 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68ac085670 con 0x7f68b415f770 2024-05-08T23:33:17.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.118+0000 7f68b9c95700 1 --2- 172.21.15.81:0/3499625594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6898066760 0x7f6898068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.122+0000 7f68b9c95700 1 --2- 172.21.15.81:0/3499625594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6898066760 0x7f6898068c20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f68b0002230 tx=0x7f68b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.126+0000 7f68a67fc700 1 -- 172.21.15.81:0/3499625594 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f68ac052a70 con 0x7f68b415f770 2024-05-08T23:33:17.336 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.334+0000 7f68bac97700 1 -- 172.21.15.81:0/3499625594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68b40684b0 con 0x7f68b415f770 2024-05-08T23:33:17.337 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.334+0000 7f68a67fc700 1 -- 172.21.15.81:0/3499625594 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f68ac056090 con 0x7f68b415f770 2024-05-08T23:33:17.338 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:17.338 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/2105335102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":2105335102},{"type":"v1","addr":"172.21.15.117:6834","nonce":2105335102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2900127785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2900127785},{"type":"v1","addr":"172.21.15.81:6835","nonce":2900127785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1899281779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1899281779},{"type":"v1","addr":"172.21.15.81:6837","nonce":1899281779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:32:40.852853+0000","modified":"2024-05-08T23:32:43.867414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":15311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_15311":{"gid":15311,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/3401411259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3401411259},{"type":"v1","addr":"172.21.15.117:6835","nonce":3401411259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-08T23:33:17.340 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.338+0000 7f68bac97700 1 -- 172.21.15.81:0/3499625594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6898066760 msgr2=0x7f6898068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.340 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.338+0000 7f68bac97700 1 --2- 172.21.15.81:0/3499625594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6898066760 0x7f6898068c20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f68b0002230 tx=0x7f68b0009040 comp rx=0 tx=0).stop 2024-05-08T23:33:17.340 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.338+0000 7f68bac97700 1 -- 172.21.15.81:0/3499625594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68b415f770 msgr2=0x7f68b4143520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.340 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.338+0000 7f68bac97700 1 --2- 172.21.15.81:0/3499625594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68b415f770 0x7f68b4143520 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009910 tx=0x7f68ac004770 comp rx=0 tx=0).stop 2024-05-08T23:33:17.340 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.338+0000 7f68bac97700 1 -- 172.21.15.81:0/3499625594 shutdown_connections 2024-05-08T23:33:17.340 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.338+0000 7f68bac97700 1 --2- 172.21.15.81:0/3499625594 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6898066760 0x7f6898068c20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.340 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.338+0000 7f68bac97700 1 --2- 172.21.15.81:0/3499625594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68b415f770 0x7f68b4143520 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.338+0000 7f68bac97700 1 --2- 172.21.15.81:0/3499625594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68b415afa0 0x7f68b414c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.338+0000 7f68bac97700 1 --2- 172.21.15.81:0/3499625594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68b40ce990 0x7f68b40ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.338+0000 7f68bac97700 1 -- 172.21.15.81:0/3499625594 >> 172.21.15.81:0/3499625594 conn(0x7f68b406c330 msgr2=0x7f68b40af5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:17.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.338+0000 7f68bac97700 1 -- 172.21.15.81:0/3499625594 shutdown_connections 2024-05-08T23:33:17.341 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.338+0000 7f68bac97700 1 -- 172.21.15.81:0/3499625594 wait complete. 2024-05-08T23:33:17.341 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 23 2024-05-08T23:33:17.353 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:33:17.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.462+0000 7f4216ca6700 1 Processor -- start 2024-05-08T23:33:17.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.462+0000 7f4216ca6700 1 -- start start 2024-05-08T23:33:17.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.462+0000 7f4216ca6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42101624b0 0x7f4210070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.462+0000 7f4216ca6700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4210158960 con 0x7f4210071e70 2024-05-08T23:33:17.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.462+0000 7f4216ca6700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42100ae990 con 0x7f42100b3c20 2024-05-08T23:33:17.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.462+0000 7f4216ca6700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42100b1f60 con 0x7f42101624b0 2024-05-08T23:33:17.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.462+0000 7f42154a3700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f4210071e70 0x7f4210070110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46118/0 (socket says 172.21.15.81:46118) 2024-05-08T23:33:17.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.462+0000 7f42154a3700 1 -- 172.21.15.81:0/757574962 learned_addr learned my addr 172.21.15.81:0/757574962 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:17.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.462+0000 7f42164a5700 1 --2- 172.21.15.81:0/757574962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42101624b0 0x7f4210070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.462+0000 7f4214ca2700 1 -- 172.21.15.81:0/757574962 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510236331 0 0) 0x7f4210158960 con 0x7f4210071e70 2024-05-08T23:33:17.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4214ca2700 1 -- 172.21.15.81:0/757574962 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41fc0035d0 con 0x7f4210071e70 2024-05-08T23:33:17.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4214ca2700 1 -- 172.21.15.81:0/757574962 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943830681 0 0) 0x7f42100ae990 con 0x7f42100b3c20 2024-05-08T23:33:17.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4214ca2700 1 -- 172.21.15.81:0/757574962 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4210158960 con 0x7f42100b3c20 2024-05-08T23:33:17.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f42164a5700 1 -- 172.21.15.81:0/757574962 >> v1:172.21.15.117:6789/0 conn(0x7f42100b3c20 legacy=0x7f42100b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f42164a5700 1 -- 172.21.15.81:0/757574962 >> v1:172.21.15.81:6789/0 conn(0x7f4210071e70 legacy=0x7f4210070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f42164a5700 1 -- 172.21.15.81:0/757574962 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42100b2170 con 0x7f42101624b0 2024-05-08T23:33:17.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f42164a5700 1 --2- 172.21.15.81:0/757574962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42101624b0 0x7f4210070820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4208009dc0 tx=0x7f420800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa878fa5a9b3f273 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4214ca2700 1 -- 172.21.15.81:0/757574962 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420800e070 con 0x7f42101624b0 2024-05-08T23:33:17.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4214ca2700 1 -- 172.21.15.81:0/757574962 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420800bb20 con 0x7f42101624b0 2024-05-08T23:33:17.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4214ca2700 1 -- 172.21.15.81:0/757574962 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420800be90 con 0x7f42101624b0 2024-05-08T23:33:17.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4216ca6700 1 -- 172.21.15.81:0/757574962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42101624b0 msgr2=0x7f4210070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4216ca6700 1 --2- 172.21.15.81:0/757574962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42101624b0 0x7f4210070820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4208009dc0 tx=0x7f420800b330 comp rx=0 tx=0).stop 2024-05-08T23:33:17.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4216ca6700 1 -- 172.21.15.81:0/757574962 shutdown_connections 2024-05-08T23:33:17.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4216ca6700 1 --2- 172.21.15.81:0/757574962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42101624b0 0x7f4210070820 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4216ca6700 1 -- 172.21.15.81:0/757574962 >> 172.21.15.81:0/757574962 conn(0x7f421006c330 msgr2=0x7f421006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:17.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4216ca6700 1 -- 172.21.15.81:0/757574962 shutdown_connections 2024-05-08T23:33:17.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4216ca6700 1 -- 172.21.15.81:0/757574962 wait complete. 2024-05-08T23:33:17.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4216ca6700 1 Processor -- start 2024-05-08T23:33:17.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4216ca6700 1 -- start start 2024-05-08T23:33:17.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4216ca6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4210071e70 0x7f4210142e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4216ca6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42100b3c20 0x7f421015b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4216ca6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42101624b0 0x7f421015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4216ca6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421014d440 con 0x7f42101624b0 2024-05-08T23:33:17.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4216ca6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421014feb0 con 0x7f4210071e70 2024-05-08T23:33:17.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.466+0000 7f4216ca6700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4210150190 con 0x7f42100b3c20 2024-05-08T23:33:17.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f42164a5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42101624b0 0x7f421015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f42164a5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42101624b0 0x7f421015b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40104/0 (socket says 172.21.15.81:40104) 2024-05-08T23:33:17.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f4215ca4700 1 --2- 172.21.15.81:0/4238129613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4210071e70 0x7f4210142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f42164a5700 1 -- 172.21.15.81:0/4238129613 learned_addr learned my addr 172.21.15.81:0/4238129613 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:17.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f42154a3700 1 --2- 172.21.15.81:0/4238129613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42100b3c20 0x7f421015b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f4215ca4700 1 -- 172.21.15.81:0/4238129613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42100b3c20 msgr2=0x7f421015b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f4215ca4700 1 --2- 172.21.15.81:0/4238129613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42100b3c20 0x7f421015b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f4215ca4700 1 -- 172.21.15.81:0/4238129613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42101624b0 msgr2=0x7f421015b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f4215ca4700 1 --2- 172.21.15.81:0/4238129613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42101624b0 0x7f421015b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f4215ca4700 1 -- 172.21.15.81:0/4238129613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4208009a70 con 0x7f4210071e70 2024-05-08T23:33:17.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f42164a5700 1 --2- 172.21.15.81:0/4238129613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42101624b0 0x7f421015b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:17.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f4215ca4700 1 --2- 172.21.15.81:0/4238129613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4210071e70 0x7f4210142e30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f420400c3c0 tx=0x7f420400c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f42027fc700 1 -- 172.21.15.81:0/4238129613 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42040052d0 con 0x7f4210071e70 2024-05-08T23:33:17.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f42027fc700 1 -- 172.21.15.81:0/4238129613 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4204005450 con 0x7f4210071e70 2024-05-08T23:33:17.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f4216ca6700 1 -- 172.21.15.81:0/4238129613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4210150470 con 0x7f4210071e70 2024-05-08T23:33:17.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f42027fc700 1 -- 172.21.15.81:0/4238129613 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42040038e0 con 0x7f4210071e70 2024-05-08T23:33:17.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f4216ca6700 1 -- 172.21.15.81:0/4238129613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4210143370 con 0x7f4210071e70 2024-05-08T23:33:17.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f4216ca6700 1 -- 172.21.15.81:0/4238129613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42100684b0 con 0x7f4210071e70 2024-05-08T23:33:17.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f42027fc700 1 -- 172.21.15.81:0/4238129613 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4204003a40 con 0x7f4210071e70 2024-05-08T23:33:17.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.470+0000 7f42027fc700 1 --2- 172.21.15.81:0/4238129613 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f41f4066730 0x7f41f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.474+0000 7f42027fc700 1 -- 172.21.15.81:0/4238129613 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4204083750 con 0x7f4210071e70 2024-05-08T23:33:17.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.474+0000 7f42154a3700 1 --2- 172.21.15.81:0/4238129613 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f41f4066730 0x7f41f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.474+0000 7f42154a3700 1 --2- 172.21.15.81:0/4238129613 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f41f4066730 0x7f41f4068bf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f42101fa9f0 tx=0x7f420c007e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.482+0000 7f42027fc700 1 -- 172.21.15.81:0/4238129613 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4204050a30 con 0x7f4210071e70 2024-05-08T23:33:17.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.658+0000 7f4216ca6700 1 -- 172.21.15.81:0/4238129613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4210158af0 con 0x7f4210071e70 2024-05-08T23:33:17.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.662+0000 7f42027fc700 1 -- 172.21.15.81:0/4238129613 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f4204017020 con 0x7f4210071e70 2024-05-08T23:33:17.663 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:17.664 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":52,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:32:41.855853+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:39.161796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:39.853963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:33:17.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.662+0000 7f4216ca6700 1 -- 172.21.15.81:0/4238129613 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f41f4066730 msgr2=0x7f41f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.662+0000 7f4216ca6700 1 --2- 172.21.15.81:0/4238129613 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f41f4066730 0x7f41f4068bf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f42101fa9f0 tx=0x7f420c007e00 comp rx=0 tx=0).stop 2024-05-08T23:33:17.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.662+0000 7f4216ca6700 1 -- 172.21.15.81:0/4238129613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4210071e70 msgr2=0x7f4210142e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.662+0000 7f4216ca6700 1 --2- 172.21.15.81:0/4238129613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4210071e70 0x7f4210142e30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f420400c3c0 tx=0x7f420400c6d0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.666+0000 7f4216ca6700 1 -- 172.21.15.81:0/4238129613 shutdown_connections 2024-05-08T23:33:17.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.666+0000 7f4216ca6700 1 --2- 172.21.15.81:0/4238129613 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f41f4066730 0x7f41f4068bf0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.666+0000 7f4216ca6700 1 --2- 172.21.15.81:0/4238129613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42101624b0 0x7f421015b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.666+0000 7f4216ca6700 1 --2- 172.21.15.81:0/4238129613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42100b3c20 0x7f421015b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.666+0000 7f4216ca6700 1 --2- 172.21.15.81:0/4238129613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4210071e70 0x7f4210142e30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.666+0000 7f4216ca6700 1 -- 172.21.15.81:0/4238129613 >> 172.21.15.81:0/4238129613 conn(0x7f421006c330 msgr2=0x7f421006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:17.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.666+0000 7f4216ca6700 1 -- 172.21.15.81:0/4238129613 shutdown_connections 2024-05-08T23:33:17.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.666+0000 7f4216ca6700 1 -- 172.21.15.81:0/4238129613 wait complete. 2024-05-08T23:33:17.685 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T23:33:17.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.794+0000 7f9c0edcd700 1 Processor -- start 2024-05-08T23:33:17.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.794+0000 7f9c0edcd700 1 -- start start 2024-05-08T23:33:17.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.794+0000 7f9c0edcd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9c0815afa0 0x7f9c0815b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0edcd700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0815eda0 con 0x7f9c0815b8c0 2024-05-08T23:33:17.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0edcd700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0815ea70 con 0x7f9c080ce990 2024-05-08T23:33:17.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0edcd700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c080ced70 con 0x7f9c0815afa0 2024-05-08T23:33:17.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0ddcb700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9c0815afa0 0x7f9c0815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0d5ca700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f9c0815b8c0 0x7f9c0815f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46128/0 (socket says 172.21.15.81:46128) 2024-05-08T23:33:17.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0d5ca700 1 -- 172.21.15.81:0/2955218333 learned_addr learned my addr 172.21.15.81:0/2955218333 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:17.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0ddcb700 1 -- 172.21.15.81:0/2955218333 >> v1:172.21.15.117:6789/0 conn(0x7f9c080ce990 legacy=0x7f9c080b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0cdc9700 1 -- 172.21.15.81:0/2955218333 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090030035 0 0) 0x7f9c0815eda0 con 0x7f9c0815b8c0 2024-05-08T23:33:17.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0ddcb700 1 -- 172.21.15.81:0/2955218333 >> v1:172.21.15.81:6789/0 conn(0x7f9c0815b8c0 legacy=0x7f9c0815f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0ddcb700 1 -- 172.21.15.81:0/2955218333 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0815bd40 con 0x7f9c0815afa0 2024-05-08T23:33:17.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0ddcb700 1 --2- 172.21.15.81:0/2955218333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c0815afa0 0x7f9c0815b380 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009850 tx=0x7f9c04009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc7413365f7feb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0cdc9700 1 -- 172.21.15.81:0/2955218333 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c04011830 con 0x7f9c0815afa0 2024-05-08T23:33:17.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0cdc9700 1 -- 172.21.15.81:0/2955218333 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c04011990 con 0x7f9c0815afa0 2024-05-08T23:33:17.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0cdc9700 1 -- 172.21.15.81:0/2955218333 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c04011d20 con 0x7f9c0815afa0 2024-05-08T23:33:17.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0edcd700 1 -- 172.21.15.81:0/2955218333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c0815afa0 msgr2=0x7f9c0815b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0edcd700 1 --2- 172.21.15.81:0/2955218333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c0815afa0 0x7f9c0815b380 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009850 tx=0x7f9c04009b60 comp rx=0 tx=0).stop 2024-05-08T23:33:17.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0edcd700 1 -- 172.21.15.81:0/2955218333 shutdown_connections 2024-05-08T23:33:17.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0edcd700 1 --2- 172.21.15.81:0/2955218333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c0815afa0 0x7f9c0815b380 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0edcd700 1 -- 172.21.15.81:0/2955218333 >> 172.21.15.81:0/2955218333 conn(0x7f9c0806c330 msgr2=0x7f9c0806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:17.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0edcd700 1 -- 172.21.15.81:0/2955218333 shutdown_connections 2024-05-08T23:33:17.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0edcd700 1 -- 172.21.15.81:0/2955218333 wait complete. 2024-05-08T23:33:17.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.798+0000 7f9c0edcd700 1 Processor -- start 2024-05-08T23:33:17.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0edcd700 1 -- start start 2024-05-08T23:33:17.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0edcd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c080ce990 0x7f9c08145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0edcd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c0815b8c0 0x7f9c08146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0edcd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c08146a40 0x7f9c0815d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0edcd700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08147b60 con 0x7f9c08146a40 2024-05-08T23:33:17.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0edcd700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08146200 con 0x7f9c0815b8c0 2024-05-08T23:33:17.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0edcd700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08146f50 con 0x7f9c080ce990 2024-05-08T23:33:17.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0e5cc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c08146a40 0x7f9c0815d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0e5cc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c08146a40 0x7f9c0815d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40120/0 (socket says 172.21.15.81:40120) 2024-05-08T23:33:17.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0e5cc700 1 -- 172.21.15.81:0/278103573 learned_addr learned my addr 172.21.15.81:0/278103573 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:17.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0d5ca700 1 --2- 172.21.15.81:0/278103573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c0815b8c0 0x7f9c08146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0ddcb700 1 --2- 172.21.15.81:0/278103573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c080ce990 0x7f9c08145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0e5cc700 1 -- 172.21.15.81:0/278103573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c080ce990 msgr2=0x7f9c08145cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0e5cc700 1 --2- 172.21.15.81:0/278103573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c080ce990 0x7f9c08145cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0e5cc700 1 -- 172.21.15.81:0/278103573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c0815b8c0 msgr2=0x7f9c08146500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0e5cc700 1 --2- 172.21.15.81:0/278103573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c0815b8c0 0x7f9c08146500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0e5cc700 1 -- 172.21.15.81:0/278103573 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c040094b0 con 0x7f9c08146a40 2024-05-08T23:33:17.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0e5cc700 1 --2- 172.21.15.81:0/278103573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c08146a40 0x7f9c0815d440 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000cc50 tx=0x7f9c0000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9bfa7fc700 1 -- 172.21.15.81:0/278103573 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000ad80 con 0x7f9c08146a40 2024-05-08T23:33:17.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9bfa7fc700 1 -- 172.21.15.81:0/278103573 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0000ce50 con 0x7f9c08146a40 2024-05-08T23:33:17.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0edcd700 1 -- 172.21.15.81:0/278103573 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c08147230 con 0x7f9c08146a40 2024-05-08T23:33:17.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9bfa7fc700 1 -- 172.21.15.81:0/278103573 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00003c60 con 0x7f9c08146a40 2024-05-08T23:33:17.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.802+0000 7f9c0edcd700 1 -- 172.21.15.81:0/278103573 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c081477f0 con 0x7f9c08146a40 2024-05-08T23:33:17.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.806+0000 7f9c0edcd700 1 -- 172.21.15.81:0/278103573 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c080684b0 con 0x7f9c08146a40 2024-05-08T23:33:17.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.806+0000 7f9bfa7fc700 1 -- 172.21.15.81:0/278103573 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9c000050d0 con 0x7f9c08146a40 2024-05-08T23:33:17.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.806+0000 7f9bfa7fc700 1 --2- 172.21.15.81:0/278103573 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9bec066760 0x7f9bec068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.806+0000 7f9bfa7fc700 1 -- 172.21.15.81:0/278103573 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c000842a0 con 0x7f9c08146a40 2024-05-08T23:33:17.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.806+0000 7f9c0ddcb700 1 --2- 172.21.15.81:0/278103573 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9bec066760 0x7f9bec068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.806+0000 7f9c0ddcb700 1 --2- 172.21.15.81:0/278103573 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9bec066760 0x7f9bec068c20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9c0806b800 tx=0x7f9c04008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:17.814+0000 7f9bfa7fc700 1 -- 172.21.15.81:0/278103573 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9c00051610 con 0x7f9c08146a40 2024-05-08T23:33:18.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.018+0000 7f9c0edcd700 1 -- 172.21.15.81:0/278103573 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9c0806e220 con 0x7f9c08146a40 2024-05-08T23:33:18.023 INFO:tasks.ceph.mon.a.smithi081.stderr:2024-05-08T23:33:18.018+0000 7f7c02c75700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:33:18.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.034+0000 7f9bfa7fc700 1 -- 172.21.15.81:0/278103573 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9c00054c30 con 0x7f9c08146a40 2024-05-08T23:33:18.038 INFO:tasks.ceph.mds.b.smithi117.stderr: -2796> 2024-05-08T23:32:46.788+0000 7faf28d89700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:33:18.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.038+0000 7f9c0edcd700 1 -- 172.21.15.81:0/278103573 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9bec066760 msgr2=0x7f9bec068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:18.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.038+0000 7f9c0edcd700 1 --2- 172.21.15.81:0/278103573 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9bec066760 0x7f9bec068c20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9c0806b800 tx=0x7f9c04008030 comp rx=0 tx=0).stop 2024-05-08T23:33:18.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.038+0000 7f9c0edcd700 1 -- 172.21.15.81:0/278103573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c08146a40 msgr2=0x7f9c0815d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:18.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.038+0000 7f9c0edcd700 1 --2- 172.21.15.81:0/278103573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c08146a40 0x7f9c0815d440 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000cc50 tx=0x7f9c0000a340 comp rx=0 tx=0).stop 2024-05-08T23:33:18.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.038+0000 7f9c0edcd700 1 -- 172.21.15.81:0/278103573 shutdown_connections 2024-05-08T23:33:18.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.038+0000 7f9c0edcd700 1 --2- 172.21.15.81:0/278103573 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f9bec066760 0x7f9bec068c20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:18.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.038+0000 7f9c0edcd700 1 --2- 172.21.15.81:0/278103573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c08146a40 0x7f9c0815d440 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:18.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.038+0000 7f9c0edcd700 1 --2- 172.21.15.81:0/278103573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c0815b8c0 0x7f9c08146500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:18.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.038+0000 7f9c0edcd700 1 --2- 172.21.15.81:0/278103573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c080ce990 0x7f9c08145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:18.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.038+0000 7f9c0edcd700 1 -- 172.21.15.81:0/278103573 >> 172.21.15.81:0/278103573 conn(0x7f9c0806c330 msgr2=0x7f9c080b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:18.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.038+0000 7f9c0edcd700 1 -- 172.21.15.81:0/278103573 shutdown_connections 2024-05-08T23:33:18.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.038+0000 7f9c0edcd700 1 -- 172.21.15.81:0/278103573 wait complete. 2024-05-08T23:33:18.041 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T23:33:18.052 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T23:33:18.071 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-05-08T23:33:18.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.166+0000 7fd61d995700 1 Processor -- start 2024-05-08T23:33:18.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.166+0000 7fd61d995700 1 -- start start 2024-05-08T23:33:18.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d995700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd61815af20 0x7fd6180b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:18.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d995700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd61806ae90 0x7fd6180b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:18.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d995700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6180adb50 con 0x7fd6180abe80 2024-05-08T23:33:18.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d995700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6180ade30 con 0x7fd61806ae90 2024-05-08T23:33:18.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d995700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180ae110 con 0x7fd61815af20 2024-05-08T23:33:18.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d194700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd61806ae90 0x7fd6180b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:18.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd617fff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd6180abe80 0x7fd61806a780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46144/0 (socket says 172.21.15.81:46144) 2024-05-08T23:33:18.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd617fff700 1 -- 172.21.15.81:0/988646718 learned_addr learned my addr 172.21.15.81:0/988646718 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:18.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61c993700 1 --2- 172.21.15.81:0/988646718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd61815af20 0x7fd6180b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:18.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd6177fe700 1 -- 172.21.15.81:0/988646718 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514933387 0 0) 0x7fd6180adb50 con 0x7fd6180abe80 2024-05-08T23:33:18.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d194700 1 -- 172.21.15.81:0/988646718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd61815af20 msgr2=0x7fd6180b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:18.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d194700 1 --2- 172.21.15.81:0/988646718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd61815af20 0x7fd6180b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:18.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d194700 1 -- 172.21.15.81:0/988646718 >> v1:172.21.15.81:6789/0 conn(0x7fd6180abe80 legacy=0x7fd61806a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:18.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d194700 1 -- 172.21.15.81:0/988646718 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6180ae3f0 con 0x7fd61806ae90 2024-05-08T23:33:18.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d194700 1 --2- 172.21.15.81:0/988646718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd61806ae90 0x7fd6180b1de0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd610009d90 tx=0x7fd61000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0edf982eb9720e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:18.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd6177fe700 1 -- 172.21.15.81:0/988646718 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61000e070 con 0x7fd61806ae90 2024-05-08T23:33:18.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd6177fe700 1 -- 172.21.15.81:0/988646718 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61000bb20 con 0x7fd61806ae90 2024-05-08T23:33:18.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd6177fe700 1 -- 172.21.15.81:0/988646718 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610012430 con 0x7fd61806ae90 2024-05-08T23:33:18.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d995700 1 -- 172.21.15.81:0/988646718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd61806ae90 msgr2=0x7fd6180b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:18.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d995700 1 --2- 172.21.15.81:0/988646718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd61806ae90 0x7fd6180b1de0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd610009d90 tx=0x7fd61000b330 comp rx=0 tx=0).stop 2024-05-08T23:33:18.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d995700 1 -- 172.21.15.81:0/988646718 shutdown_connections 2024-05-08T23:33:18.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d995700 1 --2- 172.21.15.81:0/988646718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd61806ae90 0x7fd6180b1de0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:18.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d995700 1 --2- 172.21.15.81:0/988646718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd61815af20 0x7fd6180b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:18.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d995700 1 -- 172.21.15.81:0/988646718 >> 172.21.15.81:0/988646718 conn(0x7fd6180b0e00 msgr2=0x7fd6180b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:18.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d995700 1 -- 172.21.15.81:0/988646718 shutdown_connections 2024-05-08T23:33:18.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.170+0000 7fd61d995700 1 -- 172.21.15.81:0/988646718 wait complete. 2024-05-08T23:33:18.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.174+0000 7fd61d995700 1 Processor -- start 2024-05-08T23:33:18.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.174+0000 7fd61d995700 1 -- start start 2024-05-08T23:33:18.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61d995700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd61806ae90 0x7fd61806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:18.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61d995700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6180abe80 0x7fd6181f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:18.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61d995700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd61815af20 0x7fd6181f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:18.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61d995700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618099ec0 con 0x7fd61806ae90 2024-05-08T23:33:18.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61d995700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618099b90 con 0x7fd61815af20 2024-05-08T23:33:18.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61d995700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61806d380 con 0x7fd6180abe80 2024-05-08T23:33:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61c993700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd61806ae90 0x7fd61806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61c993700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd61806ae90 0x7fd61806cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40134/0 (socket says 172.21.15.81:40134) 2024-05-08T23:33:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61c993700 1 -- 172.21.15.81:0/2819598029 learned_addr learned my addr 172.21.15.81:0/2819598029 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd617fff700 1 --2- 172.21.15.81:0/2819598029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6180abe80 0x7fd6181f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61d194700 1 --2- 172.21.15.81:0/2819598029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd61815af20 0x7fd6181f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61c993700 1 -- 172.21.15.81:0/2819598029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6180abe80 msgr2=0x7fd6181f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61c993700 1 --2- 172.21.15.81:0/2819598029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6180abe80 0x7fd6181f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61c993700 1 -- 172.21.15.81:0/2819598029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd61815af20 msgr2=0x7fd6181f5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61c993700 1 --2- 172.21.15.81:0/2819598029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd61815af20 0x7fd6181f5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61c993700 1 -- 172.21.15.81:0/2819598029 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd610009a70 con 0x7fd61806ae90 2024-05-08T23:33:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61c993700 1 --2- 172.21.15.81:0/2819598029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd61806ae90 0x7fd61806cb30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00e840 tx=0x7fd60c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd6157fa700 1 -- 172.21.15.81:0/2819598029 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c0095a0 con 0x7fd61806ae90 2024-05-08T23:33:18.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd6157fa700 1 -- 172.21.15.81:0/2819598029 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c013680 con 0x7fd61806ae90 2024-05-08T23:33:18.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd6157fa700 1 -- 172.21.15.81:0/2819598029 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c0139f0 con 0x7fd61806ae90 2024-05-08T23:33:18.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61d995700 1 -- 172.21.15.81:0/2819598029 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61815b480 con 0x7fd61806ae90 2024-05-08T23:33:18.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.178+0000 7fd61d995700 1 -- 172.21.15.81:0/2819598029 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61806d940 con 0x7fd61806ae90 2024-05-08T23:33:18.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.182+0000 7fd61d995700 1 -- 172.21.15.81:0/2819598029 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6180677c0 con 0x7fd61806ae90 2024-05-08T23:33:18.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.182+0000 7fd6157fa700 1 -- 172.21.15.81:0/2819598029 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd60c013b50 con 0x7fd61806ae90 2024-05-08T23:33:18.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.182+0000 7fd6157fa700 1 --2- 172.21.15.81:0/2819598029 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd604066730 0x7fd604068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:18.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.182+0000 7fd617fff700 1 --2- 172.21.15.81:0/2819598029 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd604066730 0x7fd604068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:18.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.182+0000 7fd6157fa700 1 -- 172.21.15.81:0/2819598029 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd60c084010 con 0x7fd61806ae90 2024-05-08T23:33:18.187 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.182+0000 7fd617fff700 1 --2- 172.21.15.81:0/2819598029 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd604066730 0x7fd604068bf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd6180cc8f0 tx=0x7fd608007e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:18.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.190+0000 7fd6157fa700 1 -- 172.21.15.81:0/2819598029 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd60c051320 con 0x7fd61806ae90 2024-05-08T23:33:18.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:18.398+0000 7fd61d995700 1 -- 172.21.15.81:0/2819598029 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd6180684b0 con 0x7fd61806ae90 2024-05-08T23:33:19.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.034+0000 7fd6157fa700 1 -- 172.21.15.81:0/2819598029 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd60c054940 con 0x7fd61806ae90 2024-05-08T23:33:19.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.038+0000 7fd61d995700 1 -- 172.21.15.81:0/2819598029 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd604066730 msgr2=0x7fd604068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:19.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.038+0000 7fd61d995700 1 --2- 172.21.15.81:0/2819598029 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd604066730 0x7fd604068bf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd6180cc8f0 tx=0x7fd608007e00 comp rx=0 tx=0).stop 2024-05-08T23:33:19.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.038+0000 7fd61d995700 1 -- 172.21.15.81:0/2819598029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd61806ae90 msgr2=0x7fd61806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:19.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.038+0000 7fd61d995700 1 --2- 172.21.15.81:0/2819598029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd61806ae90 0x7fd61806cb30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00e840 tx=0x7fd60c00eb50 comp rx=0 tx=0).stop 2024-05-08T23:33:19.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.038+0000 7fd61d995700 1 -- 172.21.15.81:0/2819598029 shutdown_connections 2024-05-08T23:33:19.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.038+0000 7fd61d995700 1 --2- 172.21.15.81:0/2819598029 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd604066730 0x7fd604068bf0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:19.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.038+0000 7fd61d995700 1 --2- 172.21.15.81:0/2819598029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd61815af20 0x7fd6181f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:19.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.042+0000 7fd61d995700 1 --2- 172.21.15.81:0/2819598029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6180abe80 0x7fd6181f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:19.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.042+0000 7fd61d995700 1 --2- 172.21.15.81:0/2819598029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd61806ae90 0x7fd61806cb30 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:19.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.042+0000 7fd61d995700 1 -- 172.21.15.81:0/2819598029 >> 172.21.15.81:0/2819598029 conn(0x7fd6180b0e00 msgr2=0x7fd618155810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:19.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.042+0000 7fd61d995700 1 -- 172.21.15.81:0/2819598029 shutdown_connections 2024-05-08T23:33:19.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.042+0000 7fd61d995700 1 -- 172.21.15.81:0/2819598029 wait complete. 2024-05-08T23:33:19.060 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T23:33:19.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.182+0000 7f8129dac700 1 Processor -- start 2024-05-08T23:33:19.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.182+0000 7f8129dac700 1 -- start start 2024-05-08T23:33:19.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.182+0000 7f8129dac700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f812415d230 0x7f812406f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:19.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.182+0000 7f8129dac700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f812406f9a0 0x7f812406a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:19.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8129dac700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812406ace0 con 0x7f812406a8a0 2024-05-08T23:33:19.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8129dac700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81240b31f0 con 0x7f812406f9a0 2024-05-08T23:33:19.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8129dac700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81240b34d0 con 0x7f812415d230 2024-05-08T23:33:19.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f811bfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f812406f9a0 0x7f812406a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:19.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8123fff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f812406a8a0 0x7f81240712c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46150/0 (socket says 172.21.15.81:46150) 2024-05-08T23:33:19.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8123fff700 1 -- 172.21.15.81:0/1055086895 learned_addr learned my addr 172.21.15.81:0/1055086895 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:19.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f81237fe700 1 --2- 172.21.15.81:0/1055086895 >> v2:172.21.15.117:3300/0 conn(0x7f812415d230 0x7f812406f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:19.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f811bfff700 1 -- 172.21.15.81:0/1055086895 >> v2:172.21.15.117:3300/0 conn(0x7f812415d230 msgr2=0x7f812406f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:19.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f811bfff700 1 --2- 172.21.15.81:0/1055086895 >> v2:172.21.15.117:3300/0 conn(0x7f812415d230 0x7f812406f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:19.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f811bfff700 1 -- 172.21.15.81:0/1055086895 >> v1:172.21.15.81:6789/0 conn(0x7f812406a8a0 legacy=0x7f81240712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:19.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f811bfff700 1 -- 172.21.15.81:0/1055086895 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81240b37b0 con 0x7f812406f9a0 2024-05-08T23:33:19.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f81237fe700 1 --2- 172.21.15.81:0/1055086895 >> v2:172.21.15.117:3300/0 conn(0x7f812415d230 0x7f812406f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:19.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f811bfff700 1 --2- 172.21.15.81:0/1055086895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f812406f9a0 0x7f812406a360 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f81240b41d0 tx=0x7f810c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e292183c443cc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:19.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8122ffd700 1 -- 172.21.15.81:0/1055086895 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c011830 con 0x7f812406f9a0 2024-05-08T23:33:19.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8122ffd700 1 -- 172.21.15.81:0/1055086895 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810c011990 con 0x7f812406f9a0 2024-05-08T23:33:19.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8122ffd700 1 -- 172.21.15.81:0/1055086895 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c011d40 con 0x7f812406f9a0 2024-05-08T23:33:19.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8129dac700 1 -- 172.21.15.81:0/1055086895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f812406f9a0 msgr2=0x7f812406a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:19.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8129dac700 1 --2- 172.21.15.81:0/1055086895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f812406f9a0 0x7f812406a360 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f81240b41d0 tx=0x7f810c009b30 comp rx=0 tx=0).stop 2024-05-08T23:33:19.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8129dac700 1 -- 172.21.15.81:0/1055086895 shutdown_connections 2024-05-08T23:33:19.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8129dac700 1 --2- 172.21.15.81:0/1055086895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f812406f9a0 0x7f812406a360 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:19.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8129dac700 1 --2- 172.21.15.81:0/1055086895 >> v2:172.21.15.117:3300/0 conn(0x7f812415d230 0x7f812406f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:19.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8129dac700 1 -- 172.21.15.81:0/1055086895 >> 172.21.15.81:0/1055086895 conn(0x7f812406d520 msgr2=0x7f812406d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:19.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8129dac700 1 -- 172.21.15.81:0/1055086895 shutdown_connections 2024-05-08T23:33:19.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.186+0000 7f8129dac700 1 -- 172.21.15.81:0/1055086895 wait complete. 2024-05-08T23:33:19.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8129dac700 1 Processor -- start 2024-05-08T23:33:19.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8129dac700 1 -- start start 2024-05-08T23:33:19.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8129dac700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f812406a8a0 0x7f81240af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:19.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8129dac700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f812406f9a0 0x7f81240b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:19.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8129dac700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f812415d230 0x7f81241598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8129dac700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8124153950 con 0x7f812415d230 2024-05-08T23:33:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8129dac700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81240b2520 con 0x7f812406a8a0 2024-05-08T23:33:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8129dac700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81240b2750 con 0x7f812406f9a0 2024-05-08T23:33:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8123fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f812415d230 0x7f81241598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8123fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f812415d230 0x7f81241598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40146/0 (socket says 172.21.15.81:40146) 2024-05-08T23:33:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8123fff700 1 -- 172.21.15.81:0/1040728280 learned_addr learned my addr 172.21.15.81:0/1040728280 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f81237fe700 1 --2- 172.21.15.81:0/1040728280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f812406a8a0 0x7f81240af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f811bfff700 1 --2- 172.21.15.81:0/1040728280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f812406f9a0 0x7f81240b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8123fff700 1 -- 172.21.15.81:0/1040728280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f812406f9a0 msgr2=0x7f81240b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8123fff700 1 --2- 172.21.15.81:0/1040728280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f812406f9a0 0x7f81240b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8123fff700 1 -- 172.21.15.81:0/1040728280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f812406a8a0 msgr2=0x7f81240af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8123fff700 1 --2- 172.21.15.81:0/1040728280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f812406a8a0 0x7f81240af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8123fff700 1 -- 172.21.15.81:0/1040728280 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c0094b0 con 0x7f812415d230 2024-05-08T23:33:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8123fff700 1 --2- 172.21.15.81:0/1040728280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f812415d230 0x7f81241598d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f811c00cbc0 tx=0x7f811c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8120ff9700 1 -- 172.21.15.81:0/1040728280 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c007480 con 0x7f812415d230 2024-05-08T23:33:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8120ff9700 1 -- 172.21.15.81:0/1040728280 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c0075e0 con 0x7f812415d230 2024-05-08T23:33:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8129dac700 1 -- 172.21.15.81:0/1040728280 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81240b2a30 con 0x7f812415d230 2024-05-08T23:33:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8120ff9700 1 -- 172.21.15.81:0/1040728280 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c007950 con 0x7f812415d230 2024-05-08T23:33:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.190+0000 7f8129dac700 1 -- 172.21.15.81:0/1040728280 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8124153e40 con 0x7f812415d230 2024-05-08T23:33:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.194+0000 7f8129dac700 1 -- 172.21.15.81:0/1040728280 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8124154030 con 0x7f812415d230 2024-05-08T23:33:19.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.194+0000 7f8120ff9700 1 -- 172.21.15.81:0/1040728280 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f811c012400 con 0x7f812415d230 2024-05-08T23:33:19.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.194+0000 7f8120ff9700 1 --2- 172.21.15.81:0/1040728280 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8104066730 0x7f8104068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:19.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.194+0000 7f8120ff9700 1 -- 172.21.15.81:0/1040728280 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f811c084160 con 0x7f812415d230 2024-05-08T23:33:19.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.194+0000 7f81237fe700 1 --2- 172.21.15.81:0/1040728280 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8104066730 0x7f8104068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:19.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.194+0000 7f81237fe700 1 --2- 172.21.15.81:0/1040728280 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8104066730 0x7f8104068bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f812415af20 tx=0x7f8114008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:19.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.202+0000 7f8120ff9700 1 -- 172.21.15.81:0/1040728280 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f811c0515b0 con 0x7f812415d230 2024-05-08T23:33:19.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:19.370+0000 7f8129dac700 1 -- 172.21.15.81:0/1040728280 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8124068630 con 0x7f812415d230 2024-05-08T23:33:20.045 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.042+0000 7f8120ff9700 1 -- 172.21.15.81:0/1040728280 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v54) v1 ==== 181+0+0 (secure 0 0 0) 0x7f811c054bd0 con 0x7f812415d230 2024-05-08T23:33:20.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.050+0000 7f8129dac700 1 -- 172.21.15.81:0/1040728280 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8104066730 msgr2=0x7f8104068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:20.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.050+0000 7f8129dac700 1 --2- 172.21.15.81:0/1040728280 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8104066730 0x7f8104068bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f812415af20 tx=0x7f8114008040 comp rx=0 tx=0).stop 2024-05-08T23:33:20.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.050+0000 7f8129dac700 1 -- 172.21.15.81:0/1040728280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f812415d230 msgr2=0x7f81241598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:20.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.050+0000 7f8129dac700 1 --2- 172.21.15.81:0/1040728280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f812415d230 0x7f81241598d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f811c00cbc0 tx=0x7f811c00cbf0 comp rx=0 tx=0).stop 2024-05-08T23:33:20.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.050+0000 7f8129dac700 1 -- 172.21.15.81:0/1040728280 shutdown_connections 2024-05-08T23:33:20.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.050+0000 7f8129dac700 1 --2- 172.21.15.81:0/1040728280 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8104066730 0x7f8104068bf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:20.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.050+0000 7f8129dac700 1 --2- 172.21.15.81:0/1040728280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f812415d230 0x7f81241598d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:20.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.050+0000 7f8129dac700 1 --2- 172.21.15.81:0/1040728280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f812406f9a0 0x7f81240b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:20.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.050+0000 7f8129dac700 1 --2- 172.21.15.81:0/1040728280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f812406a8a0 0x7f81240af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:20.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.050+0000 7f8129dac700 1 -- 172.21.15.81:0/1040728280 >> 172.21.15.81:0/1040728280 conn(0x7f812406d520 msgr2=0x7f81240ad7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:20.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.054+0000 7f8129dac700 1 -- 172.21.15.81:0/1040728280 shutdown_connections 2024-05-08T23:33:20.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.054+0000 7f8129dac700 1 -- 172.21.15.81:0/1040728280 wait complete. 2024-05-08T23:33:20.059 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' removed 2024-05-08T23:33:20.072 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T23:33:20.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.186+0000 7fb65e23a700 1 Processor -- start 2024-05-08T23:33:20.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.186+0000 7fb65e23a700 1 -- start start 2024-05-08T23:33:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.186+0000 7fb65e23a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6580ceae0 0x7fb65815af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.186+0000 7fb65e23a700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6580cde80 con 0x7fb6580ce700 2024-05-08T23:33:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.186+0000 7fb65e23a700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65815def0 con 0x7fb65815dab0 2024-05-08T23:33:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.186+0000 7fb65e23a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65815ea30 con 0x7fb6580ceae0 2024-05-08T23:33:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.186+0000 7fb65d238700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb6580ce700 0x7fb6580cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46166/0 (socket says 172.21.15.81:46166) 2024-05-08T23:33:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.186+0000 7fb65d238700 1 -- 172.21.15.81:0/531429376 learned_addr learned my addr 172.21.15.81:0/531429376 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65ca37700 1 --2- 172.21.15.81:0/531429376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6580ceae0 0x7fb65815af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:20.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983214120 0 0) 0x7fb6580cde80 con 0x7fb6580ce700 2024-05-08T23:33:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6440035d0 con 0x7fb6580ce700 2024-05-08T23:33:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006162338 0 0) 0x7fb65815def0 con 0x7fb65815dab0 2024-05-08T23:33:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6580cde80 con 0x7fb65815dab0 2024-05-08T23:33:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2447602039 0 0) 0x7fb6440035d0 con 0x7fb6580ce700 2024-05-08T23:33:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6580ceae0 msgr2=0x7fb65815af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 --2- 172.21.15.81:0/531429376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6580ceae0 0x7fb65815af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 >> v1:172.21.15.117:6789/0 conn(0x7fb65815dab0 legacy=0x7fb65806c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65815e190 con 0x7fb6580ce700 2024-05-08T23:33:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7fb650001a10 con 0x7fb6580ce700 2024-05-08T23:33:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 >> v1:172.21.15.81:6789/0 conn(0x7fb6580ce700 legacy=0x7fb6580cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 --2- 172.21.15.81:0/531429376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6580ceae0 0x7fb644003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 --2- 172.21.15.81:0/531429376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb644004140 0x7fb644004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 --2- 172.21.15.81:0/531429376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb644004db0 0x7fb644005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65815ea30 con 0x7fb644004db0 2024-05-08T23:33:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64400bb10 con 0x7fb644004140 2024-05-08T23:33:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64400bcd0 con 0x7fb6580ceae0 2024-05-08T23:33:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65da39700 1 --2- 172.21.15.81:0/531429376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb644004db0 0x7fb644005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65ca37700 1 --2- 172.21.15.81:0/531429376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb644004140 0x7fb644004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65d238700 1 --2- 172.21.15.81:0/531429376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6580ceae0 0x7fb644003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65da39700 1 -- 172.21.15.81:0/531429376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6580ceae0 msgr2=0x7fb644003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65da39700 1 --2- 172.21.15.81:0/531429376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6580ceae0 0x7fb644003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65da39700 1 -- 172.21.15.81:0/531429376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb644004140 msgr2=0x7fb644004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65da39700 1 --2- 172.21.15.81:0/531429376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb644004140 0x7fb644004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65da39700 1 -- 172.21.15.81:0/531429376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6440035d0 con 0x7fb644004db0 2024-05-08T23:33:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65da39700 1 --2- 172.21.15.81:0/531429376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb644004db0 0x7fb644005230 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb654001960 tx=0x7fb6440058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a960fb07167fdff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb654001450 con 0x7fb644004db0 2024-05-08T23:33:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6540052f0 con 0x7fb644004db0 2024-05-08T23:33:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb64bfff700 1 -- 172.21.15.81:0/531429376 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6540036a0 con 0x7fb644004db0 2024-05-08T23:33:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65e23a700 1 -- 172.21.15.81:0/531429376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb644004db0 msgr2=0x7fb644005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65e23a700 1 --2- 172.21.15.81:0/531429376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb644004db0 0x7fb644005230 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb654001960 tx=0x7fb6440058a0 comp rx=0 tx=0).stop 2024-05-08T23:33:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65da39700 1 -- 172.21.15.81:0/531429376 reap_dead start 2024-05-08T23:33:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65e23a700 1 -- 172.21.15.81:0/531429376 shutdown_connections 2024-05-08T23:33:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65e23a700 1 -- 172.21.15.81:0/531429376 >> 172.21.15.81:0/531429376 conn(0x7fb65806d650 msgr2=0x7fb65806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65e23a700 1 -- 172.21.15.81:0/531429376 shutdown_connections 2024-05-08T23:33:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.190+0000 7fb65e23a700 1 -- 172.21.15.81:0/531429376 wait complete. 2024-05-08T23:33:20.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65e23a700 1 Processor -- start 2024-05-08T23:33:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65e23a700 1 -- start start 2024-05-08T23:33:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65e23a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb644004db0 0x7fb6581f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65e23a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6581409b0 0x7fb6580aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65e23a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb65814ad80 0x7fb65814b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65e23a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65814a480 con 0x7fb644004db0 2024-05-08T23:33:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65e23a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65814b740 con 0x7fb65814ad80 2024-05-08T23:33:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65e23a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6581f4090 con 0x7fb6581409b0 2024-05-08T23:33:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65d238700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb644004db0 0x7fb6581f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65d238700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb644004db0 0x7fb6581f2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40154/0 (socket says 172.21.15.81:40154) 2024-05-08T23:33:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65d238700 1 -- 172.21.15.81:0/259373380 learned_addr learned my addr 172.21.15.81:0/259373380 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65da39700 1 --2- 172.21.15.81:0/259373380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb65814ad80 0x7fb65814b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65ca37700 1 --2- 172.21.15.81:0/259373380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6581409b0 0x7fb6580aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65d238700 1 -- 172.21.15.81:0/259373380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6581409b0 msgr2=0x7fb6580aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65d238700 1 --2- 172.21.15.81:0/259373380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6581409b0 0x7fb6580aec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65d238700 1 -- 172.21.15.81:0/259373380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb65814ad80 msgr2=0x7fb65814b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65d238700 1 --2- 172.21.15.81:0/259373380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb65814ad80 0x7fb65814b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65d238700 1 -- 172.21.15.81:0/259373380 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb658140e30 con 0x7fb644004db0 2024-05-08T23:33:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65ca37700 1 --2- 172.21.15.81:0/259373380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6581409b0 0x7fb6580aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:33:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65d238700 1 --2- 172.21.15.81:0/259373380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb644004db0 0x7fb6581f2f00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb65000e440 tx=0x7fb65000e520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb649ffb700 1 -- 172.21.15.81:0/259373380 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65000ed40 con 0x7fb644004db0 2024-05-08T23:33:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb649ffb700 1 -- 172.21.15.81:0/259373380 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb650013420 con 0x7fb644004db0 2024-05-08T23:33:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb649ffb700 1 -- 172.21.15.81:0/259373380 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6500137d0 con 0x7fb644004db0 2024-05-08T23:33:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65e23a700 1 -- 172.21.15.81:0/259373380 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6580b05f0 con 0x7fb644004db0 2024-05-08T23:33:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.194+0000 7fb65e23a700 1 -- 172.21.15.81:0/259373380 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6580af1c0 con 0x7fb644004db0 2024-05-08T23:33:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.198+0000 7fb65e23a700 1 -- 172.21.15.81:0/259373380 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb658148420 con 0x7fb644004db0 2024-05-08T23:33:20.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.198+0000 7fb649ffb700 1 -- 172.21.15.81:0/259373380 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb650013930 con 0x7fb644004db0 2024-05-08T23:33:20.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.198+0000 7fb649ffb700 1 --2- 172.21.15.81:0/259373380 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb63c066730 0x7fb63c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:20.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.198+0000 7fb649ffb700 1 -- 172.21.15.81:0/259373380 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5415+0+0 (secure 0 0 0) 0x7fb6500847c0 con 0x7fb644004db0 2024-05-08T23:33:20.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.198+0000 7fb65ca37700 1 --2- 172.21.15.81:0/259373380 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb63c066730 0x7fb63c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:20.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.198+0000 7fb65ca37700 1 --2- 172.21.15.81:0/259373380 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb63c066730 0x7fb63c068bf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb65815c2a0 tx=0x7fb64c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:20.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.206+0000 7fb649ffb700 1 -- 172.21.15.81:0/259373380 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb658148420 con 0x7fb644004db0 2024-05-08T23:33:20.377 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:20.374+0000 7fb65e23a700 1 -- 172.21.15.81:0/259373380 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb658148420 con 0x7fb644004db0 2024-05-08T23:33:21.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.058+0000 7fb649ffb700 1 -- 172.21.15.81:0/259373380 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v55) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb658148420 con 0x7fb644004db0 2024-05-08T23:33:21.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.066+0000 7fb65e23a700 1 -- 172.21.15.81:0/259373380 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb63c066730 msgr2=0x7fb63c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:21.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.066+0000 7fb65e23a700 1 --2- 172.21.15.81:0/259373380 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb63c066730 0x7fb63c068bf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb65815c2a0 tx=0x7fb64c008040 comp rx=0 tx=0).stop 2024-05-08T23:33:21.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.066+0000 7fb65e23a700 1 -- 172.21.15.81:0/259373380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb644004db0 msgr2=0x7fb6581f2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:21.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.066+0000 7fb65e23a700 1 --2- 172.21.15.81:0/259373380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb644004db0 0x7fb6581f2f00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb65000e440 tx=0x7fb65000e520 comp rx=0 tx=0).stop 2024-05-08T23:33:21.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.066+0000 7fb65e23a700 1 -- 172.21.15.81:0/259373380 shutdown_connections 2024-05-08T23:33:21.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.066+0000 7fb65e23a700 1 --2- 172.21.15.81:0/259373380 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb63c066730 0x7fb63c068bf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.066+0000 7fb65e23a700 1 --2- 172.21.15.81:0/259373380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb65814ad80 0x7fb65814b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.069 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.066+0000 7fb65e23a700 1 --2- 172.21.15.81:0/259373380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6581409b0 0x7fb6580aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.066+0000 7fb65e23a700 1 --2- 172.21.15.81:0/259373380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb644004db0 0x7fb6581f2f00 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.066+0000 7fb65e23a700 1 -- 172.21.15.81:0/259373380 >> 172.21.15.81:0/259373380 conn(0x7fb65806d650 msgr2=0x7fb6580700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:21.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.066+0000 7fb65e23a700 1 -- 172.21.15.81:0/259373380 shutdown_connections 2024-05-08T23:33:21.070 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.066+0000 7fb65e23a700 1 -- 172.21.15.81:0/259373380 wait complete. 2024-05-08T23:33:21.071 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' removed 2024-05-08T23:33:21.086 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T23:33:21.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.194+0000 7f7625c55700 1 Processor -- start 2024-05-08T23:33:21.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.194+0000 7f7625c55700 1 -- start start 2024-05-08T23:33:21.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.194+0000 7f7625c55700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f762015d470 0x7f76200ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:21.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.194+0000 7f7625c55700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f762015d850 con 0x7f76200b1f30 2024-05-08T23:33:21.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.194+0000 7f7625c55700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76200d00a0 con 0x7f76200ced00 2024-05-08T23:33:21.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.194+0000 7f7625c55700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76200d0380 con 0x7f762015d470 2024-05-08T23:33:21.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f761ffff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f76200b1f30 0x7f762006af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46174/0 (socket says 172.21.15.81:46174) 2024-05-08T23:33:21.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f761ffff700 1 -- 172.21.15.81:0/1605385435 learned_addr learned my addr 172.21.15.81:0/1605385435 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:21.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f7624c53700 1 --2- 172.21.15.81:0/1605385435 >> v2:172.21.15.81:3300/0 conn(0x7f762015d470 0x7f76200ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:21.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f761f7fe700 1 -- 172.21.15.81:0/1605385435 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129488184 0 0) 0x7f762015d850 con 0x7f76200b1f30 2024-05-08T23:33:21.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f761f7fe700 1 -- 172.21.15.81:0/1605385435 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f760c0035d0 con 0x7f76200b1f30 2024-05-08T23:33:21.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f7624c53700 1 -- 172.21.15.81:0/1605385435 >> v1:172.21.15.117:6789/0 conn(0x7f76200ced00 legacy=0x7f76200d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:21.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f7624c53700 1 -- 172.21.15.81:0/1605385435 >> v1:172.21.15.81:6789/0 conn(0x7f76200b1f30 legacy=0x7f762006af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:21.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f7624c53700 1 -- 172.21.15.81:0/1605385435 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76200cdfc0 con 0x7f762015d470 2024-05-08T23:33:21.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f7624c53700 1 --2- 172.21.15.81:0/1605385435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f762015d470 0x7f76200ce7c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7608009850 tx=0x7f7608009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e825b43266363f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:21.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f761f7fe700 1 -- 172.21.15.81:0/1605385435 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7608011830 con 0x7f762015d470 2024-05-08T23:33:21.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f761f7fe700 1 -- 172.21.15.81:0/1605385435 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7608011990 con 0x7f762015d470 2024-05-08T23:33:21.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f761f7fe700 1 -- 172.21.15.81:0/1605385435 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7608011d00 con 0x7f762015d470 2024-05-08T23:33:21.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f7625c55700 1 -- 172.21.15.81:0/1605385435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f762015d470 msgr2=0x7f76200ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:21.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f7625c55700 1 --2- 172.21.15.81:0/1605385435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f762015d470 0x7f76200ce7c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7608009850 tx=0x7f7608009b60 comp rx=0 tx=0).stop 2024-05-08T23:33:21.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f7625c55700 1 -- 172.21.15.81:0/1605385435 shutdown_connections 2024-05-08T23:33:21.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f7625c55700 1 --2- 172.21.15.81:0/1605385435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f762015d470 0x7f76200ce7c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f7625c55700 1 -- 172.21.15.81:0/1605385435 >> 172.21.15.81:0/1605385435 conn(0x7f762009c370 msgr2=0x7f76200d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:21.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f7625c55700 1 -- 172.21.15.81:0/1605385435 shutdown_connections 2024-05-08T23:33:21.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.198+0000 7f7625c55700 1 -- 172.21.15.81:0/1605385435 wait complete. 2024-05-08T23:33:21.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f7625c55700 1 Processor -- start 2024-05-08T23:33:21.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f7625c55700 1 -- start start 2024-05-08T23:33:21.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f7625c55700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76200b1f30 0x7f76200c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:21.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f7625c55700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76200ced00 0x7f762014b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:21.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f7625c55700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f762015d470 0x7f76201600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:21.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f7625c55700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762015e430 con 0x7f76200ced00 2024-05-08T23:33:21.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f7625c55700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76200cb930 con 0x7f762015d470 2024-05-08T23:33:21.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f7625c55700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762015e610 con 0x7f76200b1f30 2024-05-08T23:33:21.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f761ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76200ced00 0x7f762014b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:21.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f761ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76200ced00 0x7f762014b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40174/0 (socket says 172.21.15.81:40174) 2024-05-08T23:33:21.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f761ffff700 1 -- 172.21.15.81:0/749853896 learned_addr learned my addr 172.21.15.81:0/749853896 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:21.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f7625454700 1 --2- 172.21.15.81:0/749853896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f762015d470 0x7f76201600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:21.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f7624c53700 1 --2- 172.21.15.81:0/749853896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76200b1f30 0x7f76200c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:21.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f761ffff700 1 -- 172.21.15.81:0/749853896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76200b1f30 msgr2=0x7f76200c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:21.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f761ffff700 1 --2- 172.21.15.81:0/749853896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76200b1f30 0x7f76200c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f761ffff700 1 -- 172.21.15.81:0/749853896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f762015d470 msgr2=0x7f76201600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:21.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f761ffff700 1 --2- 172.21.15.81:0/749853896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f762015d470 0x7f76201600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f761ffff700 1 -- 172.21.15.81:0/749853896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76080094b0 con 0x7f76200ced00 2024-05-08T23:33:21.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f7624c53700 1 --2- 172.21.15.81:0/749853896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76200b1f30 0x7f76200c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:21.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f761ffff700 1 --2- 172.21.15.81:0/749853896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76200ced00 0x7f762014b2c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f761000c4d0 tx=0x7f761000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:21.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f761d7fa700 1 -- 172.21.15.81:0/749853896 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610005d90 con 0x7f76200ced00 2024-05-08T23:33:21.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f761d7fa700 1 -- 172.21.15.81:0/749853896 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76100070d0 con 0x7f76200ced00 2024-05-08T23:33:21.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f7625c55700 1 -- 172.21.15.81:0/749853896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762015e8f0 con 0x7f76200ced00 2024-05-08T23:33:21.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f761d7fa700 1 -- 172.21.15.81:0/749853896 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610007480 con 0x7f76200ced00 2024-05-08T23:33:21.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.202+0000 7f7625c55700 1 -- 172.21.15.81:0/749853896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762014b800 con 0x7f76200ced00 2024-05-08T23:33:21.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.206+0000 7f7625c55700 1 -- 172.21.15.81:0/749853896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76200684b0 con 0x7f76200ced00 2024-05-08T23:33:21.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.206+0000 7f761d7fa700 1 -- 172.21.15.81:0/749853896 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f76100075e0 con 0x7f76200ced00 2024-05-08T23:33:21.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.206+0000 7f761d7fa700 1 --2- 172.21.15.81:0/749853896 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7600066760 0x7f7600068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:21.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.206+0000 7f7624c53700 1 --2- 172.21.15.81:0/749853896 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7600066760 0x7f7600068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:21.209 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.206+0000 7f761d7fa700 1 -- 172.21.15.81:0/749853896 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7610083260 con 0x7f76200ced00 2024-05-08T23:33:21.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.206+0000 7f7624c53700 1 --2- 172.21.15.81:0/749853896 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7600066760 0x7f7600068c20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7608009ba0 tx=0x7f7608008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:21.214 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.210+0000 7f761d7fa700 1 -- 172.21.15.81:0/749853896 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7610050960 con 0x7f76200ced00 2024-05-08T23:33:21.389 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.386+0000 7f7625c55700 1 -- 172.21.15.81:0/749853896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f76200d0920 con 0x7f76200ced00 2024-05-08T23:33:21.394 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.390+0000 7f761d7fa700 1 -- 172.21.15.81:0/749853896 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7610053f80 con 0x7f76200ced00 2024-05-08T23:33:21.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.394+0000 7f7625c55700 1 -- 172.21.15.81:0/749853896 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7600066760 msgr2=0x7f7600068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:21.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.394+0000 7f7625c55700 1 --2- 172.21.15.81:0/749853896 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7600066760 0x7f7600068c20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7608009ba0 tx=0x7f7608008030 comp rx=0 tx=0).stop 2024-05-08T23:33:21.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.394+0000 7f7625c55700 1 -- 172.21.15.81:0/749853896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76200ced00 msgr2=0x7f762014b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:21.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.394+0000 7f7625c55700 1 --2- 172.21.15.81:0/749853896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76200ced00 0x7f762014b2c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f761000c4d0 tx=0x7f761000c7e0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.394+0000 7f7625c55700 1 -- 172.21.15.81:0/749853896 shutdown_connections 2024-05-08T23:33:21.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.394+0000 7f7625c55700 1 --2- 172.21.15.81:0/749853896 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f7600066760 0x7f7600068c20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.394+0000 7f7625c55700 1 --2- 172.21.15.81:0/749853896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f762015d470 0x7f76201600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.394+0000 7f7625c55700 1 --2- 172.21.15.81:0/749853896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76200ced00 0x7f762014b2c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.394+0000 7f7625c55700 1 --2- 172.21.15.81:0/749853896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76200b1f30 0x7f76200c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.394+0000 7f7625c55700 1 -- 172.21.15.81:0/749853896 >> 172.21.15.81:0/749853896 conn(0x7f762009c370 msgr2=0x7f76200b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:21.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.394+0000 7f7625c55700 1 -- 172.21.15.81:0/749853896 shutdown_connections 2024-05-08T23:33:21.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.394+0000 7f7625c55700 1 -- 172.21.15.81:0/749853896 wait complete. 2024-05-08T23:33:21.408 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2024-05-08T23:33:21.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.514+0000 7fec21e82700 1 Processor -- start 2024-05-08T23:33:21.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.514+0000 7fec21e82700 1 -- start start 2024-05-08T23:33:21.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.514+0000 7fec21e82700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fec1c1596a0 0x7fec1c0b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:21.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.514+0000 7fec21e82700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fec1c0b4e50 0x7fec1c159a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:21.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.514+0000 7fec21e82700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec1c0b3590 con 0x7fec1c159fc0 2024-05-08T23:33:21.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.514+0000 7fec21e82700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1c0b24f0 con 0x7fec1c1596a0 2024-05-08T23:33:21.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.514+0000 7fec21e82700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1c0b27d0 con 0x7fec1c0b4e50 2024-05-08T23:33:21.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec20e80700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fec1c1596a0 0x7fec1c0b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:21.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec20e80700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fec1c1596a0 0x7fec1c0b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40184/0 (socket says 172.21.15.81:40184) 2024-05-08T23:33:21.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec20e80700 1 -- 172.21.15.81:0/2264730893 learned_addr learned my addr 172.21.15.81:0/2264730893 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:21.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec1bfff700 1 --2- 172.21.15.81:0/2264730893 >> v2:172.21.15.117:3300/0 conn(0x7fec1c0b4e50 0x7fec1c159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:21.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec20e80700 1 -- 172.21.15.81:0/2264730893 >> v2:172.21.15.117:3300/0 conn(0x7fec1c0b4e50 msgr2=0x7fec1c159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:21.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec20e80700 1 --2- 172.21.15.81:0/2264730893 >> v2:172.21.15.117:3300/0 conn(0x7fec1c0b4e50 0x7fec1c159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec20e80700 1 -- 172.21.15.81:0/2264730893 >> v1:172.21.15.117:6789/0 conn(0x7fec1c159fc0 legacy=0x7fec1c15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:21.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec20e80700 1 -- 172.21.15.81:0/2264730893 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c0b2ab0 con 0x7fec1c1596a0 2024-05-08T23:33:21.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec1bfff700 1 --2- 172.21.15.81:0/2264730893 >> v2:172.21.15.117:3300/0 conn(0x7fec1c0b4e50 0x7fec1c159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:21.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec20e80700 1 --2- 172.21.15.81:0/2264730893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec1c1596a0 0x7fec1c0b4910 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0cee40 tx=0x7fec1000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e80bd9e725be41 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:21.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec1b7fe700 1 -- 172.21.15.81:0/2264730893 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1000dd90 con 0x7fec1c1596a0 2024-05-08T23:33:21.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec1b7fe700 1 -- 172.21.15.81:0/2264730893 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec10010470 con 0x7fec1c1596a0 2024-05-08T23:33:21.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec1b7fe700 1 -- 172.21.15.81:0/2264730893 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec10010800 con 0x7fec1c1596a0 2024-05-08T23:33:21.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec21e82700 1 -- 172.21.15.81:0/2264730893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec1c1596a0 msgr2=0x7fec1c0b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:21.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec21e82700 1 --2- 172.21.15.81:0/2264730893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec1c1596a0 0x7fec1c0b4910 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0cee40 tx=0x7fec1000be90 comp rx=0 tx=0).stop 2024-05-08T23:33:21.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec21e82700 1 -- 172.21.15.81:0/2264730893 shutdown_connections 2024-05-08T23:33:21.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec21e82700 1 --2- 172.21.15.81:0/2264730893 >> v2:172.21.15.117:3300/0 conn(0x7fec1c0b4e50 0x7fec1c159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec21e82700 1 --2- 172.21.15.81:0/2264730893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec1c1596a0 0x7fec1c0b4910 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec21e82700 1 -- 172.21.15.81:0/2264730893 >> 172.21.15.81:0/2264730893 conn(0x7fec1c06c330 msgr2=0x7fec1c06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:21.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec21e82700 1 -- 172.21.15.81:0/2264730893 shutdown_connections 2024-05-08T23:33:21.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.518+0000 7fec21e82700 1 -- 172.21.15.81:0/2264730893 wait complete. 2024-05-08T23:33:21.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec21e82700 1 Processor -- start 2024-05-08T23:33:21.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec21e82700 1 -- start start 2024-05-08T23:33:21.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec21e82700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec1c0b4e50 0x7fec1c14fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec21e82700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec1c1596a0 0x7fec1c154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec21e82700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec1c159fc0 0x7fec1c155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec21e82700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c14ff70 con 0x7fec1c0b4e50 2024-05-08T23:33:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec21e82700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0af8c0 con 0x7fec1c1596a0 2024-05-08T23:33:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec21e82700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0afba0 con 0x7fec1c159fc0 2024-05-08T23:33:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec20e80700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec1c0b4e50 0x7fec1c14fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec1bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec1c1596a0 0x7fec1c154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec20e80700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec1c0b4e50 0x7fec1c14fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40186/0 (socket says 172.21.15.81:40186) 2024-05-08T23:33:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec20e80700 1 -- 172.21.15.81:0/1099159035 learned_addr learned my addr 172.21.15.81:0/1099159035 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec21681700 1 --2- 172.21.15.81:0/1099159035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec1c159fc0 0x7fec1c155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec20e80700 1 -- 172.21.15.81:0/1099159035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec1c159fc0 msgr2=0x7fec1c155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec20e80700 1 --2- 172.21.15.81:0/1099159035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec1c159fc0 0x7fec1c155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec20e80700 1 -- 172.21.15.81:0/1099159035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec1c1596a0 msgr2=0x7fec1c154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec20e80700 1 --2- 172.21.15.81:0/1099159035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec1c1596a0 0x7fec1c154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:21.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec20e80700 1 -- 172.21.15.81:0/1099159035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1000b810 con 0x7fec1c0b4e50 2024-05-08T23:33:21.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec20e80700 1 --2- 172.21.15.81:0/1099159035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec1c0b4e50 0x7fec1c14fa30 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fec10000c00 tx=0x7fec10004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:21.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec197fa700 1 -- 172.21.15.81:0/1099159035 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1001d070 con 0x7fec1c0b4e50 2024-05-08T23:33:21.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec197fa700 1 -- 172.21.15.81:0/1099159035 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec10010e20 con 0x7fec1c0b4e50 2024-05-08T23:33:21.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec197fa700 1 -- 172.21.15.81:0/1099159035 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec10004490 con 0x7fec1c0b4e50 2024-05-08T23:33:21.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec21e82700 1 -- 172.21.15.81:0/1099159035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c0afe80 con 0x7fec1c0b4e50 2024-05-08T23:33:21.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.522+0000 7fec21e82700 1 -- 172.21.15.81:0/1099159035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1c0b0440 con 0x7fec1c0b4e50 2024-05-08T23:33:21.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.526+0000 7fec21e82700 1 -- 172.21.15.81:0/1099159035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1c0684b0 con 0x7fec1c0b4e50 2024-05-08T23:33:21.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.526+0000 7fec197fa700 1 -- 172.21.15.81:0/1099159035 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fec100036c0 con 0x7fec1c0b4e50 2024-05-08T23:33:21.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.526+0000 7fec197fa700 1 --2- 172.21.15.81:0/1099159035 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fec08066730 0x7fec08068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:21.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.526+0000 7fec1bfff700 1 --2- 172.21.15.81:0/1099159035 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fec08066730 0x7fec08068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:21.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.526+0000 7fec197fa700 1 -- 172.21.15.81:0/1099159035 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fec10085860 con 0x7fec1c0b4e50 2024-05-08T23:33:21.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.526+0000 7fec1bfff700 1 --2- 172.21.15.81:0/1099159035 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fec08066730 0x7fec08068bf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fec1c1525c0 tx=0x7fec0c0098f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:21.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.534+0000 7fec197fa700 1 -- 172.21.15.81:0/1099159035 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fec10052ec0 con 0x7fec1c0b4e50 2024-05-08T23:33:21.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:21.770+0000 7fec21e82700 1 -- 172.21.15.81:0/1099159035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7fec1c06d930 con 0x7fec1c0b4e50 2024-05-08T23:33:22.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.062+0000 7fec197fa700 1 -- 172.21.15.81:0/1099159035 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v164) v1 ==== 134+0+0 (secure 0 0 0) 0x7fec100564e0 con 0x7fec1c0b4e50 2024-05-08T23:33:22.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.066+0000 7fec21e82700 1 -- 172.21.15.81:0/1099159035 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fec08066730 msgr2=0x7fec08068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:22.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.066+0000 7fec21e82700 1 --2- 172.21.15.81:0/1099159035 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fec08066730 0x7fec08068bf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fec1c1525c0 tx=0x7fec0c0098f0 comp rx=0 tx=0).stop 2024-05-08T23:33:22.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.066+0000 7fec21e82700 1 -- 172.21.15.81:0/1099159035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec1c0b4e50 msgr2=0x7fec1c14fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:22.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.066+0000 7fec21e82700 1 --2- 172.21.15.81:0/1099159035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec1c0b4e50 0x7fec1c14fa30 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fec10000c00 tx=0x7fec10004770 comp rx=0 tx=0).stop 2024-05-08T23:33:22.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.066+0000 7fec21e82700 1 -- 172.21.15.81:0/1099159035 shutdown_connections 2024-05-08T23:33:22.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.066+0000 7fec21e82700 1 --2- 172.21.15.81:0/1099159035 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fec08066730 0x7fec08068bf0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:22.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.066+0000 7fec21e82700 1 --2- 172.21.15.81:0/1099159035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec1c159fc0 0x7fec1c155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:22.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.066+0000 7fec21e82700 1 --2- 172.21.15.81:0/1099159035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec1c1596a0 0x7fec1c154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:22.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.066+0000 7fec21e82700 1 --2- 172.21.15.81:0/1099159035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec1c0b4e50 0x7fec1c14fa30 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:22.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.066+0000 7fec21e82700 1 -- 172.21.15.81:0/1099159035 >> 172.21.15.81:0/1099159035 conn(0x7fec1c06c330 msgr2=0x7fec1c070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:22.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.066+0000 7fec21e82700 1 -- 172.21.15.81:0/1099159035 shutdown_connections 2024-05-08T23:33:22.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.066+0000 7fec21e82700 1 -- 172.21.15.81:0/1099159035 wait complete. 2024-05-08T23:33:22.088 INFO:tasks.cephfs_test_runner:test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ok 2024-05-08T23:33:22.088 INFO:tasks.cephfs_test_runner: 2024-05-08T23:33:22.089 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-08T23:33:22.089 INFO:tasks.cephfs_test_runner:Ran 2 tests in 134.439s 2024-05-08T23:33:22.089 INFO:tasks.cephfs_test_runner: 2024-05-08T23:33:22.089 INFO:tasks.cephfs_test_runner:OK 2024-05-08T23:33:22.089 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-08T23:33:22.100 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-08T23:33:22.126 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-08T23:33:22.126 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:33:22.126 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:33:22.134 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346332k,nr_inodes=4086583,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14716 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T23:33:22.134 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T23:33:22.134 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:33:22.142 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346332k,nr_inodes=4086583,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14716 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T23:33:22.142 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:33:22.143 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:33:22.151 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346332k,nr_inodes=4086583,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17609 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T23:33:22.151 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-05-08T23:33:22.151 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:33:22.200 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346332k,nr_inodes=4086583,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17609 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T23:33:22.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:22.200 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-05-08T23:33:22.208 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T23:33:22.208 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:33:22.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:22.209 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T23:33:22.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:22.267 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:33:22.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:22.284 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link delete ceph-brx 2024-05-08T23:33:22.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:22.478 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-05-08T23:33:22.485 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-08T23:33:22.485 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T23:33:22.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:22.486 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T23:33:22.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:22.503 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T23:33:22.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:33:22.563 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-05-08T23:33:22.715 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-08T23:33:22.724 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T23:33:22.724 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:33:22.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87f59e700 1 Processor -- start 2024-05-08T23:33:22.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87f59e700 1 -- start start 2024-05-08T23:33:22.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87f59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc8801596a0 0x7fc8800b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:22.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87f59e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc8800b4e50 0x7fc880159a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:22.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87f59e700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8800b3590 con 0x7fc880159fc0 2024-05-08T23:33:22.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87f59e700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc880071ab0 con 0x7fc8800b4e50 2024-05-08T23:33:22.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87f59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc880070820 con 0x7fc8801596a0 2024-05-08T23:33:22.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87dd9b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc8800b4e50 0x7fc880159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:22.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87dd9b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc8800b4e50 0x7fc880159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40194/0 (socket says 172.21.15.81:40194) 2024-05-08T23:33:22.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87dd9b700 1 -- 172.21.15.81:0/3389921812 learned_addr learned my addr 172.21.15.81:0/3389921812 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:22.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87e59c700 1 --2- 172.21.15.81:0/3389921812 >> v2:172.21.15.117:3300/0 conn(0x7fc8801596a0 0x7fc8800b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:22.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87dd9b700 1 -- 172.21.15.81:0/3389921812 >> v2:172.21.15.117:3300/0 conn(0x7fc8801596a0 msgr2=0x7fc8800b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:22.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87dd9b700 1 --2- 172.21.15.81:0/3389921812 >> v2:172.21.15.117:3300/0 conn(0x7fc8801596a0 0x7fc8800b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:22.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87dd9b700 1 -- 172.21.15.81:0/3389921812 >> v1:172.21.15.81:6789/0 conn(0x7fc880159fc0 legacy=0x7fc880070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:22.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87dd9b700 1 -- 172.21.15.81:0/3389921812 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8800b24f0 con 0x7fc8800b4e50 2024-05-08T23:33:22.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87e59c700 1 --2- 172.21.15.81:0/3389921812 >> v2:172.21.15.117:3300/0 conn(0x7fc8801596a0 0x7fc8800b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:33:22.852 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87dd9b700 1 --2- 172.21.15.81:0/3389921812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8800b4e50 0x7fc880159a80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc88004f790 tx=0x7fc868009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c887b4d0e8c49599 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:22.852 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87d59a700 1 -- 172.21.15.81:0/3389921812 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86800b8a0 con 0x7fc8800b4e50 2024-05-08T23:33:22.852 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87d59a700 1 -- 172.21.15.81:0/3389921812 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86800ba00 con 0x7fc8800b4e50 2024-05-08T23:33:22.852 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87d59a700 1 -- 172.21.15.81:0/3389921812 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86800bd90 con 0x7fc8800b4e50 2024-05-08T23:33:22.853 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87f59e700 1 -- 172.21.15.81:0/3389921812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8800b4e50 msgr2=0x7fc880159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:22.853 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.846+0000 7fc87f59e700 1 --2- 172.21.15.81:0/3389921812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8800b4e50 0x7fc880159a80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc88004f790 tx=0x7fc868009bf0 comp rx=0 tx=0).stop 2024-05-08T23:33:22.853 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87f59e700 1 -- 172.21.15.81:0/3389921812 shutdown_connections 2024-05-08T23:33:22.853 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87f59e700 1 --2- 172.21.15.81:0/3389921812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8800b4e50 0x7fc880159a80 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:22.853 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87f59e700 1 --2- 172.21.15.81:0/3389921812 >> v2:172.21.15.117:3300/0 conn(0x7fc8801596a0 0x7fc8800b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:22.853 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87f59e700 1 -- 172.21.15.81:0/3389921812 >> 172.21.15.81:0/3389921812 conn(0x7fc88006c330 msgr2=0x7fc88006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:22.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87f59e700 1 -- 172.21.15.81:0/3389921812 shutdown_connections 2024-05-08T23:33:22.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87f59e700 1 -- 172.21.15.81:0/3389921812 wait complete. 2024-05-08T23:33:22.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87f59e700 1 Processor -- start 2024-05-08T23:33:22.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87f59e700 1 -- start start 2024-05-08T23:33:22.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87f59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8800b4e50 0x7fc8801f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:22.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87f59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8801596a0 0x7fc8801560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:22.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87f59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880159fc0 0x7fc8800cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:22.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87f59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8801f08b0 con 0x7fc8801596a0 2024-05-08T23:33:22.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87f59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8800a8d80 con 0x7fc880159fc0 2024-05-08T23:33:22.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87f59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8800a9060 con 0x7fc8800b4e50 2024-05-08T23:33:22.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87dd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8801596a0 0x7fc8801560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:22.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87dd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8801596a0 0x7fc8801560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40202/0 (socket says 172.21.15.81:40202) 2024-05-08T23:33:22.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87dd9b700 1 -- 172.21.15.81:0/3137517024 learned_addr learned my addr 172.21.15.81:0/3137517024 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:22.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87e59c700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8800b4e50 0x7fc8801f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:22.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87ed9d700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880159fc0 0x7fc8800cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:22.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87dd9b700 1 -- 172.21.15.81:0/3137517024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8800b4e50 msgr2=0x7fc8801f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:22.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87dd9b700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8800b4e50 0x7fc8801f3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:22.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87dd9b700 1 -- 172.21.15.81:0/3137517024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880159fc0 msgr2=0x7fc8800cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:22.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87dd9b700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880159fc0 0x7fc8800cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:22.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87dd9b700 1 -- 172.21.15.81:0/3137517024 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc868009570 con 0x7fc8801596a0 2024-05-08T23:33:22.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87e59c700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8800b4e50 0x7fc8801f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:33:22.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.850+0000 7fc87ed9d700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880159fc0 0x7fc8800cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:33:22.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.854+0000 7fc87dd9b700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8801596a0 0x7fc8801560b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc868000c00 tx=0x7fc868004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:22.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.854+0000 7fc876ffd700 1 -- 172.21.15.81:0/3137517024 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868003ad0 con 0x7fc8801596a0 2024-05-08T23:33:22.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.854+0000 7fc876ffd700 1 -- 172.21.15.81:0/3137517024 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86801e070 con 0x7fc8801596a0 2024-05-08T23:33:22.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.854+0000 7fc87f59e700 1 -- 172.21.15.81:0/3137517024 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8800a9340 con 0x7fc8801596a0 2024-05-08T23:33:22.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.854+0000 7fc876ffd700 1 -- 172.21.15.81:0/3137517024 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868003e90 con 0x7fc8801596a0 2024-05-08T23:33:22.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.854+0000 7fc87f59e700 1 -- 172.21.15.81:0/3137517024 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8800a89a0 con 0x7fc8801596a0 2024-05-08T23:33:22.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.854+0000 7fc87f59e700 1 -- 172.21.15.81:0/3137517024 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8800684b0 con 0x7fc8801596a0 2024-05-08T23:33:22.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.854+0000 7fc876ffd700 1 -- 172.21.15.81:0/3137517024 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc868011420 con 0x7fc8801596a0 2024-05-08T23:33:22.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.858+0000 7fc876ffd700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc860066730 0x7fc860068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:22.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.858+0000 7fc876ffd700 1 -- 172.21.15.81:0/3137517024 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc868020070 con 0x7fc8801596a0 2024-05-08T23:33:22.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.858+0000 7fc87e59c700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc860066730 0x7fc860068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:22.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.858+0000 7fc87e59c700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc860066730 0x7fc860068bf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc8700089d0 tx=0x7fc870008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:22.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:22.862+0000 7fc876ffd700 1 -- 172.21.15.81:0/3137517024 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc868052e20 con 0x7fc8801596a0 2024-05-08T23:33:23.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.030+0000 7fc87f59e700 1 -- 172.21.15.81:0/3137517024 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc8800b2d90 con 0x7fc860066730 2024-05-08T23:33:23.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.034+0000 7fc876ffd700 1 -- 172.21.15.81:0/3137517024 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49780 (secure 0 0 0) 0x7fc87000c040 con 0x7fc860066730 2024-05-08T23:33:23.040 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:23.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.038+0000 7fc87f59e700 1 -- 172.21.15.81:0/3137517024 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc860066730 msgr2=0x7fc860068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.038+0000 7fc87f59e700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc860066730 0x7fc860068bf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc8700089d0 tx=0x7fc870008040 comp rx=0 tx=0).stop 2024-05-08T23:33:23.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.038+0000 7fc87f59e700 1 -- 172.21.15.81:0/3137517024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8801596a0 msgr2=0x7fc8801560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.038+0000 7fc87f59e700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8801596a0 0x7fc8801560b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc868000c00 tx=0x7fc868004770 comp rx=0 tx=0).stop 2024-05-08T23:33:23.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.038+0000 7fc87f59e700 1 -- 172.21.15.81:0/3137517024 shutdown_connections 2024-05-08T23:33:23.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.038+0000 7fc87f59e700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc860066730 0x7fc860068bf0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.038+0000 7fc87f59e700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc880159fc0 0x7fc8800cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.038+0000 7fc87f59e700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8801596a0 0x7fc8801560b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.038+0000 7fc87f59e700 1 --2- 172.21.15.81:0/3137517024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8800b4e50 0x7fc8801f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.038+0000 7fc87f59e700 1 -- 172.21.15.81:0/3137517024 >> 172.21.15.81:0/3137517024 conn(0x7fc88006c330 msgr2=0x7fc880152a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:23.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.038+0000 7fc87f59e700 1 -- 172.21.15.81:0/3137517024 shutdown_connections 2024-05-08T23:33:23.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.038+0000 7fc87f59e700 1 -- 172.21.15.81:0/3137517024 wait complete. 2024-05-08T23:33:23.044 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:33:23.054 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-05-08T23:33:21.379789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230428,"kb_used_data":6068,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":754744292,"statfs":{"total":773094113280,"available":772858155008,"internally_reserved":0,"allocated":6213632,"data_stored":3260120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":229625400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12549,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-62,"num_write_kb":-32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001668"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":191,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485798+0000","last_change":"2024-05-08T23:30:27.226484+0000","last_active":"2024-05-08T23:32:42.485798+0000","last_peered":"2024-05-08T23:32:42.485798+0000","last_clean":"2024-05-08T23:32:42.485798+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:32:42.485798+0000","last_undegraded":"2024-05-08T23:32:42.485798+0000","last_fullsized":"2024-05-08T23:32:42.485798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_clean_scrub_stamp":"2024-05-08T23:30:26.204118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:14.267287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":259859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.89400000000000002,"15min":0.90500000000000003},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1719999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.96899999999999997,"15min":0.97899999999999998},"min":{"1min":0.58299999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1910000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.75800000000000001}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.59099999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6579999999999999,"5min":2.004,"15min":2.004},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.92600000000000005,"15min":0.92400000000000004},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.327,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.0369999999999999}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88300000000000001,"15min":0.88500000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.95999999999999996,"15min":0.97099999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.218,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.56200000000000006}]},{"osd":3,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.89200000000000002,"15min":0.88400000000000001},"min":{"1min":0.65100000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.88},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96899999999999997,"15min":0.97799999999999998},"min":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2290000000000001,"5min":1.272,"15min":1.272},"last":0.84299999999999997}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.71499999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.1439999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0249999999999999,"15min":1.0269999999999999},"min":{"1min":0.67800000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.1919999999999999}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0469999999999999,"15min":1.056},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4450000000000001,"5min":2.109,"15min":2.109},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1240000000000001,"15min":1.1439999999999999},"min":{"1min":0.75800000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.996}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1970000000000001,"15min":1.2070000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5660000000000001,"5min":2.544,"15min":2.544},"last":1.266},{"interface":"front","average":{"1min":1.069,"5min":1.103,"15min":1.109},"min":{"1min":0.81200000000000006,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.2230000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509480,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":253126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2969999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.97099999999999997,"15min":0.98299999999999998},"min":{"1min":0.63700000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1859999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.82699999999999996}]},{"osd":1,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.90900000000000003,"15min":0.90400000000000003},"min":{"1min":0.67300000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88400000000000001,"15min":0.878},"min":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.042}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.91800000000000004,"15min":0.91700000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.385,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.93700000000000006,"15min":0.92500000000000004},"min":{"1min":0.70299999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.454,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.99399999999999999}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.97199999999999998,"15min":0.98399999999999999},"min":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.252,"5min":1.395,"15min":1.395},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.004,"5min":0.96099999999999997,"15min":0.95399999999999996},"min":{"1min":0.63700000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.101}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.995,"15min":1.0129999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.877},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.1339999999999999,"15min":1.1599999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.484,"5min":2.464,"15min":2.464},"last":1.155}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0800000000000001,"15min":1.0820000000000001},"min":{"1min":0.82099999999999995,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.133}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1930000000000001,"15min":1.216},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.21},{"interface":"front","average":{"1min":1.163,"5min":1.1759999999999999,"15min":1.1779999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.2589999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":614400,"data_stored":253708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.84099999999999997,"15min":0.85099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2030000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77900000000000003,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.085}]},{"osd":1,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.70999999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1459999999999999,"5min":1.268,"15min":1.268},"last":1.163},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.88800000000000001,"15min":0.90000000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.125,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.59899999999999998}]},{"osd":2,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.88800000000000001,"15min":0.89800000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89900000000000002,"15min":0.90700000000000003},"min":{"1min":0.65400000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0249999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.85299999999999998,"15min":0.84899999999999998},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1759999999999999,"5min":1.23,"15min":1.23},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2390000000000001,"5min":1.431,"15min":1.431},"last":1.1160000000000001}]},{"osd":5,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3420000000000001,"5min":1.413,"15min":1.413},"last":0.873},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95899999999999996,"15min":0.95099999999999996},"min":{"1min":0.73999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.278,"5min":1.504,"15min":1.504},"last":0.98299999999999998}]},{"osd":6,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0269999999999999,"15min":1.0309999999999999},"min":{"1min":0.73899999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1419999999999999,"15min":1.1559999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3859999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.83499999999999996}]},{"osd":7,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0800000000000001,"15min":1.0840000000000001},"min":{"1min":0.69899999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4379999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.03},{"interface":"front","average":{"1min":1.113,"5min":1.143,"15min":1.1479999999999999},"min":{"1min":0.94799999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4099999999999999,"5min":1.583,"15min":1.583},"last":1.246}]}]},{"osd":3,"up_from":15,"seq":64424509478,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":638976,"data_stored":269404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.49099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.46300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":1.032,"15min":1.032},"last":0.80400000000000005}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70599999999999996,"15min":0.71099999999999997},"min":{"1min":0.50700000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.89300000000000002,"5min":1,"15min":1},"last":0.76400000000000001}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0469999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80200000000000005,"15min":0.79900000000000004},"min":{"1min":0.56699999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0229999999999999,"5min":1.242,"15min":1.242},"last":0.88600000000000001}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.65600000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.129,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.96699999999999997,"15min":0.97399999999999998},"min":{"1min":0.78000000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.244,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.73999999999999999}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.014,"15min":1.022},"min":{"1min":0.80300000000000005,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.161,"5min":1.623,"15min":1.623},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.995,"15min":1.004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1359999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.96499999999999997}]},{"osd":6,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.997,"15min":1.0009999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.232,"5min":1.482,"15min":1.482},"last":1.3080000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.093,"15min":1.111},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3480000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.85299999999999998}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0800000000000001,"15min":1.0860000000000001},"min":{"1min":0.78400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.456,"5min":1.514,"15min":1.514},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0580000000000001,"15min":1.0600000000000001},"min":{"1min":0.75,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.2869999999999999,"5min":1.552,"15min":1.552},"last":1.2250000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":638976,"data_stored":265326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80400000000000005,"5min":1.099,"15min":1.099},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0660000000000001,"5min":1.129,"15min":1.129},"last":0.66700000000000004}]},{"osd":1,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70999999999999996,"15min":0.70899999999999996},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.90200000000000002}]},{"osd":3,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0509999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.79500000000000004}]},{"osd":4,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.96799999999999997,"15min":0.97799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.27,"5min":1.367,"15min":1.367},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.95599999999999996,"15min":0.96399999999999997},"min":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.84099999999999997}]},{"osd":5,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98999999999999999,"15min":0.98599999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.874},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.98099999999999998}]},{"osd":6,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0620000000000001,"15min":1.0740000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4239999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.075,"15min":1.075},"min":{"1min":0.85699999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.95599999999999996}]},{"osd":7,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1220000000000001,"15min":1.1240000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":1.028},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1000000000000001,"15min":1.1020000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.492,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.93000000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":259367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.51300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.44,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.68899999999999995,"15min":0.67000000000000004},"min":{"1min":0.53900000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.73099999999999998}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73499999999999999,"15min":0.72599999999999998},"min":{"1min":0.57999999999999996,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.872},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.42799999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.038,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.63900000000000001}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.95099999999999996,"15min":0.95199999999999996},"min":{"1min":0.67600000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.256,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.95499999999999996,"15min":0.94299999999999995},"min":{"1min":0.77300000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.2709999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":1.0900000000000001}]},{"osd":5,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0109999999999999,"15min":1.0089999999999999},"min":{"1min":0.72299999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.403,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.077,"15min":1.079},"min":{"1min":0.85499999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3879999999999999,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.77200000000000002}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0369999999999999,"15min":1.034},"min":{"1min":0.80000000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.4330000000000001,"5min":1.633,"15min":1.633},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.135,"15min":1.1419999999999999},"min":{"1min":0.80500000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.4750000000000001,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.98099999999999998}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.0209999999999999,"15min":1.0009999999999999},"min":{"1min":0.83099999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.371,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.081,"15min":1.0820000000000001},"min":{"1min":0.84999999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.306,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.95499999999999996}]}]},{"osd":7,"up_from":14,"seq":60129542182,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1224704,"data_stored":849699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.88300000000000001,"15min":0.89700000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.86399999999999999,"15min":0.86699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1379999999999999,"5min":1.359,"15min":1.359},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91200000000000003,"15min":0.91600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.23,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.83899999999999997}]},{"osd":2,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88,"15min":0.873},"min":{"1min":0.57499999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88,"15min":0.88500000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.133,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.76000000000000001}]},{"osd":3,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.91300000000000003,"15min":0.91100000000000003},"min":{"1min":0.60699999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.99299999999999999,"15min":0.99099999999999999},"min":{"1min":0.80100000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2470000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.55700000000000005}]},{"osd":4,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.90400000000000003,"15min":0.91200000000000003},"min":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.73999999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.1899999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.018}]},{"osd":5,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.028,"15min":1.028},"min":{"1min":0.64100000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.405,"5min":1.714,"15min":1.714},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.1950000000000001,"5min":1.325,"15min":1.325},"last":0.91800000000000004}]},{"osd":6,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.048,"15min":1.048},"min":{"1min":0.86599999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.4430000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.091,"5min":1.105,"15min":1.107},"min":{"1min":0.83999999999999997,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.359,"5min":1.496,"15min":1.496},"last":1.0760000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542182,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1224704,"data_stored":849631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.47499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":1.351},"last":0.44500000000000001}]},{"osd":2,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.51500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.238,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.546,"5min":1.869,"15min":1.869},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.84199999999999997,"15min":0.85599999999999998},"min":{"1min":0.55200000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.94199999999999995,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.626}]},{"osd":4,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97199999999999998,"15min":0.97299999999999998},"min":{"1min":0.71999999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.357,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.90800000000000003,"15min":0.89600000000000002},"min":{"1min":0.69899999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.82399999999999995}]},{"osd":5,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.052,"15min":1.0600000000000001},"min":{"1min":0.76800000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3819999999999999,"5min":1.474,"15min":1.474},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.92500000000000004,"15min":0.91400000000000003},"min":{"1min":0.72699999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.67400000000000004}]},{"osd":6,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.056,"15min":1.0720000000000001},"min":{"1min":0.76900000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.171,"5min":1.736,"15min":1.736},"last":0.751},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1180000000000001,"15min":1.121},"min":{"1min":0.89000000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4830000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.65900000000000003}]},{"osd":7,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.113,"15min":1.117},"min":{"1min":0.66600000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.397,"5min":1.72,"15min":1.72},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.097,"5min":1.109,"15min":1.111},"min":{"1min":0.86399999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.69099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:33:23.056 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:33:23.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e64c7a700 1 Processor -- start 2024-05-08T23:33:23.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e64c7a700 1 -- start start 2024-05-08T23:33:23.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e64c7a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e6013cda0 0x7f1e6006c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e64c7a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1e6013d180 0x7f1e6006c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e64c7a700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e600b3380 con 0x7f1e6015a950 2024-05-08T23:33:23.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e64c7a700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6015b6a0 con 0x7f1e6013d180 2024-05-08T23:33:23.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e64c7a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e600b3660 con 0x7f1e6013cda0 2024-05-08T23:33:23.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e5ed9d700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f1e6015a950 0x7f1e6015ade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46190/0 (socket says 172.21.15.81:46190) 2024-05-08T23:33:23.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e5ed9d700 1 -- 172.21.15.81:0/688993948 learned_addr learned my addr 172.21.15.81:0/688993948 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:23.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e5dd9b700 1 --2- 172.21.15.81:0/688993948 >> v2:172.21.15.117:3300/0 conn(0x7f1e6013d180 0x7f1e6006c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e5e59c700 1 --2- 172.21.15.81:0/688993948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e6013cda0 0x7f1e6006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e5d59a700 1 -- 172.21.15.81:0/688993948 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506736173 0 0) 0x7f1e600b3380 con 0x7f1e6015a950 2024-05-08T23:33:23.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e5d59a700 1 -- 172.21.15.81:0/688993948 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e400057e0 con 0x7f1e6015a950 2024-05-08T23:33:23.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e5dd9b700 1 -- 172.21.15.81:0/688993948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e6013cda0 msgr2=0x7f1e6006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:23.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e5dd9b700 1 --2- 172.21.15.81:0/688993948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e6013cda0 0x7f1e6006c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e5dd9b700 1 -- 172.21.15.81:0/688993948 >> v1:172.21.15.81:6789/0 conn(0x7f1e6015a950 legacy=0x7f1e6015ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e5dd9b700 1 -- 172.21.15.81:0/688993948 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e600b3940 con 0x7f1e6013d180 2024-05-08T23:33:23.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e5dd9b700 1 --2- 172.21.15.81:0/688993948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e6013d180 0x7f1e6006c890 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009850 tx=0x7f1e48009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b970b17c456dae4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:23.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e5d59a700 1 -- 172.21.15.81:0/688993948 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e48011830 con 0x7f1e6013d180 2024-05-08T23:33:23.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e5d59a700 1 -- 172.21.15.81:0/688993948 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e48011990 con 0x7f1e6013d180 2024-05-08T23:33:23.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.166+0000 7f1e5d59a700 1 -- 172.21.15.81:0/688993948 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e48011d00 con 0x7f1e6013d180 2024-05-08T23:33:23.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.170+0000 7f1e64c7a700 1 -- 172.21.15.81:0/688993948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e6013d180 msgr2=0x7f1e6006c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.170+0000 7f1e64c7a700 1 --2- 172.21.15.81:0/688993948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e6013d180 0x7f1e6006c890 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009850 tx=0x7f1e48009b60 comp rx=0 tx=0).stop 2024-05-08T23:33:23.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.170+0000 7f1e64c7a700 1 -- 172.21.15.81:0/688993948 shutdown_connections 2024-05-08T23:33:23.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.170+0000 7f1e64c7a700 1 --2- 172.21.15.81:0/688993948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e6013d180 0x7f1e6006c890 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.170+0000 7f1e64c7a700 1 --2- 172.21.15.81:0/688993948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e6013cda0 0x7f1e6006c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.170+0000 7f1e64c7a700 1 -- 172.21.15.81:0/688993948 >> 172.21.15.81:0/688993948 conn(0x7f1e6006a360 msgr2=0x7f1e6006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:23.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.170+0000 7f1e64c7a700 1 -- 172.21.15.81:0/688993948 shutdown_connections 2024-05-08T23:33:23.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.170+0000 7f1e64c7a700 1 -- 172.21.15.81:0/688993948 wait complete. 2024-05-08T23:33:23.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.170+0000 7f1e64c7a700 1 Processor -- start 2024-05-08T23:33:23.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e64c7a700 1 -- start start 2024-05-08T23:33:23.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e64c7a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e6013cda0 0x7f1e601f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e64c7a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e6013d180 0x7f1e6014e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e64c7a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6015a950 0x7f1e601f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e64c7a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e600710c0 con 0x7f1e6015a950 2024-05-08T23:33:23.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e64c7a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e600713a0 con 0x7f1e6013d180 2024-05-08T23:33:23.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e64c7a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e60071680 con 0x7f1e6013cda0 2024-05-08T23:33:23.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e5ed9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6015a950 0x7f1e601f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e5ed9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6015a950 0x7f1e601f1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40208/0 (socket says 172.21.15.81:40208) 2024-05-08T23:33:23.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e5ed9d700 1 -- 172.21.15.81:0/3248959690 learned_addr learned my addr 172.21.15.81:0/3248959690 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:23.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e5dd9b700 1 --2- 172.21.15.81:0/3248959690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e6013d180 0x7f1e6014e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e5e59c700 1 --2- 172.21.15.81:0/3248959690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e6013cda0 0x7f1e601f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e5ed9d700 1 -- 172.21.15.81:0/3248959690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e6013cda0 msgr2=0x7f1e601f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e5ed9d700 1 --2- 172.21.15.81:0/3248959690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e6013cda0 0x7f1e601f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e5ed9d700 1 -- 172.21.15.81:0/3248959690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e6013d180 msgr2=0x7f1e6014e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e5ed9d700 1 --2- 172.21.15.81:0/3248959690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e6013d180 0x7f1e6014e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e5ed9d700 1 -- 172.21.15.81:0/3248959690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e480094b0 con 0x7f1e6015a950 2024-05-08T23:33:23.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e5ed9d700 1 --2- 172.21.15.81:0/3248959690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6015a950 0x7f1e601f1e80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800ca20 tx=0x7f1e5800ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:23.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e56ffd700 1 -- 172.21.15.81:0/3248959690 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e58007480 con 0x7f1e6015a950 2024-05-08T23:33:23.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e56ffd700 1 -- 172.21.15.81:0/3248959690 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e580075e0 con 0x7f1e6015a950 2024-05-08T23:33:23.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e56ffd700 1 -- 172.21.15.81:0/3248959690 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e58007970 con 0x7f1e6015a950 2024-05-08T23:33:23.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e64c7a700 1 -- 172.21.15.81:0/3248959690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e60071960 con 0x7f1e6015a950 2024-05-08T23:33:23.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.174+0000 7f1e64c7a700 1 -- 172.21.15.81:0/3248959690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e60071f20 con 0x7f1e6015a950 2024-05-08T23:33:23.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.178+0000 7f1e64c7a700 1 -- 172.21.15.81:0/3248959690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e600684b0 con 0x7f1e6015a950 2024-05-08T23:33:23.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.178+0000 7f1e56ffd700 1 -- 172.21.15.81:0/3248959690 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e5801b3b0 con 0x7f1e6015a950 2024-05-08T23:33:23.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.178+0000 7f1e56ffd700 1 --2- 172.21.15.81:0/3248959690 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1e44066730 0x7f1e44068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.178+0000 7f1e56ffd700 1 -- 172.21.15.81:0/3248959690 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1e58083810 con 0x7f1e6015a950 2024-05-08T23:33:23.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.178+0000 7f1e5e59c700 1 --2- 172.21.15.81:0/3248959690 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1e44066730 0x7f1e44068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.178+0000 7f1e5e59c700 1 --2- 172.21.15.81:0/3248959690 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1e44066730 0x7f1e44068bf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009bf0 tx=0x7f1e50009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:23.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.182+0000 7f1e56ffd700 1 -- 172.21.15.81:0/3248959690 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1e58050d60 con 0x7f1e6015a950 2024-05-08T23:33:23.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.350+0000 7f1e64c7a700 1 -- 172.21.15.81:0/3248959690 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e600b20d0 con 0x7f1e44066730 2024-05-08T23:33:23.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.358+0000 7f1e56ffd700 1 -- 172.21.15.81:0/3248959690 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49780 (secure 0 0 0) 0x7f1e500118c0 con 0x7f1e44066730 2024-05-08T23:33:23.361 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:23.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.358+0000 7f1e64c7a700 1 -- 172.21.15.81:0/3248959690 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1e44066730 msgr2=0x7f1e44068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.358+0000 7f1e64c7a700 1 --2- 172.21.15.81:0/3248959690 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1e44066730 0x7f1e44068bf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009bf0 tx=0x7f1e50009240 comp rx=0 tx=0).stop 2024-05-08T23:33:23.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.358+0000 7f1e64c7a700 1 -- 172.21.15.81:0/3248959690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6015a950 msgr2=0x7f1e601f1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.358+0000 7f1e64c7a700 1 --2- 172.21.15.81:0/3248959690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6015a950 0x7f1e601f1e80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800ca20 tx=0x7f1e5800ca50 comp rx=0 tx=0).stop 2024-05-08T23:33:23.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.358+0000 7f1e64c7a700 1 -- 172.21.15.81:0/3248959690 shutdown_connections 2024-05-08T23:33:23.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.358+0000 7f1e64c7a700 1 --2- 172.21.15.81:0/3248959690 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f1e44066730 0x7f1e44068bf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.358+0000 7f1e64c7a700 1 --2- 172.21.15.81:0/3248959690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6015a950 0x7f1e601f1e80 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.358+0000 7f1e64c7a700 1 --2- 172.21.15.81:0/3248959690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e6013d180 0x7f1e6014e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.358+0000 7f1e64c7a700 1 --2- 172.21.15.81:0/3248959690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e6013cda0 0x7f1e601f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.358+0000 7f1e64c7a700 1 -- 172.21.15.81:0/3248959690 >> 172.21.15.81:0/3248959690 conn(0x7f1e6006a360 msgr2=0x7f1e6015cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:23.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.358+0000 7f1e64c7a700 1 -- 172.21.15.81:0/3248959690 shutdown_connections 2024-05-08T23:33:23.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.358+0000 7f1e64c7a700 1 -- 172.21.15.81:0/3248959690 wait complete. 2024-05-08T23:33:23.364 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:33:23.376 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-05-08T23:33:21.379789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230428,"kb_used_data":6068,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":754744292,"statfs":{"total":773094113280,"available":772858155008,"internally_reserved":0,"allocated":6213632,"data_stored":3260120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":229625400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12549,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-62,"num_write_kb":-32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001668"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":191,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485798+0000","last_change":"2024-05-08T23:30:27.226484+0000","last_active":"2024-05-08T23:32:42.485798+0000","last_peered":"2024-05-08T23:32:42.485798+0000","last_clean":"2024-05-08T23:32:42.485798+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:32:42.485798+0000","last_undegraded":"2024-05-08T23:32:42.485798+0000","last_fullsized":"2024-05-08T23:32:42.485798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_clean_scrub_stamp":"2024-05-08T23:30:26.204118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:14.267287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":259859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.89400000000000002,"15min":0.90500000000000003},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1719999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.96899999999999997,"15min":0.97899999999999998},"min":{"1min":0.58299999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1910000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.75800000000000001}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.59099999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6579999999999999,"5min":2.004,"15min":2.004},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.92600000000000005,"15min":0.92400000000000004},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.327,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.0369999999999999}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88300000000000001,"15min":0.88500000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.95999999999999996,"15min":0.97099999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.218,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.56200000000000006}]},{"osd":3,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.89200000000000002,"15min":0.88400000000000001},"min":{"1min":0.65100000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.88},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96899999999999997,"15min":0.97799999999999998},"min":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2290000000000001,"5min":1.272,"15min":1.272},"last":0.84299999999999997}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.71499999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.1439999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0249999999999999,"15min":1.0269999999999999},"min":{"1min":0.67800000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.1919999999999999}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0469999999999999,"15min":1.056},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4450000000000001,"5min":2.109,"15min":2.109},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1240000000000001,"15min":1.1439999999999999},"min":{"1min":0.75800000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.996}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1970000000000001,"15min":1.2070000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5660000000000001,"5min":2.544,"15min":2.544},"last":1.266},{"interface":"front","average":{"1min":1.069,"5min":1.103,"15min":1.109},"min":{"1min":0.81200000000000006,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.2230000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509480,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":253126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2969999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.97099999999999997,"15min":0.98299999999999998},"min":{"1min":0.63700000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1859999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.82699999999999996}]},{"osd":1,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.90900000000000003,"15min":0.90400000000000003},"min":{"1min":0.67300000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88400000000000001,"15min":0.878},"min":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.042}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.91800000000000004,"15min":0.91700000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.385,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.93700000000000006,"15min":0.92500000000000004},"min":{"1min":0.70299999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.454,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.99399999999999999}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.97199999999999998,"15min":0.98399999999999999},"min":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.252,"5min":1.395,"15min":1.395},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.004,"5min":0.96099999999999997,"15min":0.95399999999999996},"min":{"1min":0.63700000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.101}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.995,"15min":1.0129999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.877},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.1339999999999999,"15min":1.1599999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.484,"5min":2.464,"15min":2.464},"last":1.155}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0800000000000001,"15min":1.0820000000000001},"min":{"1min":0.82099999999999995,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.133}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1930000000000001,"15min":1.216},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.21},{"interface":"front","average":{"1min":1.163,"5min":1.1759999999999999,"15min":1.1779999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.2589999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":614400,"data_stored":253708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.84099999999999997,"15min":0.85099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2030000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77900000000000003,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.085}]},{"osd":1,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.70999999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1459999999999999,"5min":1.268,"15min":1.268},"last":1.163},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.88800000000000001,"15min":0.90000000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.125,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.59899999999999998}]},{"osd":2,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.88800000000000001,"15min":0.89800000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89900000000000002,"15min":0.90700000000000003},"min":{"1min":0.65400000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0249999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.85299999999999998,"15min":0.84899999999999998},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1759999999999999,"5min":1.23,"15min":1.23},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2390000000000001,"5min":1.431,"15min":1.431},"last":1.1160000000000001}]},{"osd":5,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3420000000000001,"5min":1.413,"15min":1.413},"last":0.873},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95899999999999996,"15min":0.95099999999999996},"min":{"1min":0.73999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.278,"5min":1.504,"15min":1.504},"last":0.98299999999999998}]},{"osd":6,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0269999999999999,"15min":1.0309999999999999},"min":{"1min":0.73899999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1419999999999999,"15min":1.1559999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3859999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.83499999999999996}]},{"osd":7,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0800000000000001,"15min":1.0840000000000001},"min":{"1min":0.69899999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4379999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.03},{"interface":"front","average":{"1min":1.113,"5min":1.143,"15min":1.1479999999999999},"min":{"1min":0.94799999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4099999999999999,"5min":1.583,"15min":1.583},"last":1.246}]}]},{"osd":3,"up_from":15,"seq":64424509478,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":638976,"data_stored":269404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.49099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.46300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":1.032,"15min":1.032},"last":0.80400000000000005}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70599999999999996,"15min":0.71099999999999997},"min":{"1min":0.50700000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.89300000000000002,"5min":1,"15min":1},"last":0.76400000000000001}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0469999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80200000000000005,"15min":0.79900000000000004},"min":{"1min":0.56699999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0229999999999999,"5min":1.242,"15min":1.242},"last":0.88600000000000001}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.65600000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.129,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.96699999999999997,"15min":0.97399999999999998},"min":{"1min":0.78000000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.244,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.73999999999999999}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.014,"15min":1.022},"min":{"1min":0.80300000000000005,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.161,"5min":1.623,"15min":1.623},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.995,"15min":1.004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1359999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.96499999999999997}]},{"osd":6,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.997,"15min":1.0009999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.232,"5min":1.482,"15min":1.482},"last":1.3080000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.093,"15min":1.111},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3480000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.85299999999999998}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0800000000000001,"15min":1.0860000000000001},"min":{"1min":0.78400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.456,"5min":1.514,"15min":1.514},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0580000000000001,"15min":1.0600000000000001},"min":{"1min":0.75,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.2869999999999999,"5min":1.552,"15min":1.552},"last":1.2250000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":638976,"data_stored":265326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80400000000000005,"5min":1.099,"15min":1.099},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0660000000000001,"5min":1.129,"15min":1.129},"last":0.66700000000000004}]},{"osd":1,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70999999999999996,"15min":0.70899999999999996},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.90200000000000002}]},{"osd":3,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0509999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.79500000000000004}]},{"osd":4,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.96799999999999997,"15min":0.97799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.27,"5min":1.367,"15min":1.367},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.95599999999999996,"15min":0.96399999999999997},"min":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.84099999999999997}]},{"osd":5,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98999999999999999,"15min":0.98599999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.874},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.98099999999999998}]},{"osd":6,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0620000000000001,"15min":1.0740000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4239999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.075,"15min":1.075},"min":{"1min":0.85699999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.95599999999999996}]},{"osd":7,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1220000000000001,"15min":1.1240000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":1.028},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1000000000000001,"15min":1.1020000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.492,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.93000000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":259367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.51300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.44,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.68899999999999995,"15min":0.67000000000000004},"min":{"1min":0.53900000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.73099999999999998}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73499999999999999,"15min":0.72599999999999998},"min":{"1min":0.57999999999999996,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.872},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.42799999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.038,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.63900000000000001}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.95099999999999996,"15min":0.95199999999999996},"min":{"1min":0.67600000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.256,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.95499999999999996,"15min":0.94299999999999995},"min":{"1min":0.77300000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.2709999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":1.0900000000000001}]},{"osd":5,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0109999999999999,"15min":1.0089999999999999},"min":{"1min":0.72299999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.403,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.077,"15min":1.079},"min":{"1min":0.85499999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3879999999999999,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.77200000000000002}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0369999999999999,"15min":1.034},"min":{"1min":0.80000000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.4330000000000001,"5min":1.633,"15min":1.633},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.135,"15min":1.1419999999999999},"min":{"1min":0.80500000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.4750000000000001,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.98099999999999998}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.0209999999999999,"15min":1.0009999999999999},"min":{"1min":0.83099999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.371,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.081,"15min":1.0820000000000001},"min":{"1min":0.84999999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.306,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.95499999999999996}]}]},{"osd":7,"up_from":14,"seq":60129542182,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1224704,"data_stored":849699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.88300000000000001,"15min":0.89700000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.86399999999999999,"15min":0.86699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1379999999999999,"5min":1.359,"15min":1.359},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91200000000000003,"15min":0.91600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.23,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.83899999999999997}]},{"osd":2,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88,"15min":0.873},"min":{"1min":0.57499999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88,"15min":0.88500000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.133,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.76000000000000001}]},{"osd":3,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.91300000000000003,"15min":0.91100000000000003},"min":{"1min":0.60699999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.99299999999999999,"15min":0.99099999999999999},"min":{"1min":0.80100000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2470000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.55700000000000005}]},{"osd":4,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.90400000000000003,"15min":0.91200000000000003},"min":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.73999999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.1899999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.018}]},{"osd":5,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.028,"15min":1.028},"min":{"1min":0.64100000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.405,"5min":1.714,"15min":1.714},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.1950000000000001,"5min":1.325,"15min":1.325},"last":0.91800000000000004}]},{"osd":6,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.048,"15min":1.048},"min":{"1min":0.86599999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.4430000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.091,"5min":1.105,"15min":1.107},"min":{"1min":0.83999999999999997,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.359,"5min":1.496,"15min":1.496},"last":1.0760000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542182,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1224704,"data_stored":849631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.47499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":1.351},"last":0.44500000000000001}]},{"osd":2,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.51500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.238,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.546,"5min":1.869,"15min":1.869},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.84199999999999997,"15min":0.85599999999999998},"min":{"1min":0.55200000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.94199999999999995,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.626}]},{"osd":4,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97199999999999998,"15min":0.97299999999999998},"min":{"1min":0.71999999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.357,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.90800000000000003,"15min":0.89600000000000002},"min":{"1min":0.69899999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.82399999999999995}]},{"osd":5,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.052,"15min":1.0600000000000001},"min":{"1min":0.76800000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3819999999999999,"5min":1.474,"15min":1.474},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.92500000000000004,"15min":0.91400000000000003},"min":{"1min":0.72699999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.67400000000000004}]},{"osd":6,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.056,"15min":1.0720000000000001},"min":{"1min":0.76900000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.171,"5min":1.736,"15min":1.736},"last":0.751},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1180000000000001,"15min":1.121},"min":{"1min":0.89000000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4830000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.65900000000000003}]},{"osd":7,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.113,"15min":1.117},"min":{"1min":0.66600000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.397,"5min":1.72,"15min":1.72},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.097,"5min":1.109,"15min":1.111},"min":{"1min":0.86399999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.69099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:33:23.378 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T23:33:23.378 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:33:23.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.490+0000 7f50266be700 1 Processor -- start 2024-05-08T23:33:23.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.490+0000 7f50266be700 1 -- start start 2024-05-08T23:33:23.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50266be700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f50200af2e0 0x7f502015e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50266be700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f50200af6c0 0x7f5020071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50266be700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f502006d0a0 0x7f502006d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50266be700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50200cfeb0 con 0x7f50200af2e0 2024-05-08T23:33:23.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50266be700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5020072430 con 0x7f50200af6c0 2024-05-08T23:33:23.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50266be700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50200b4a40 con 0x7f502006d0a0 2024-05-08T23:33:23.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50256bc700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f50200af2e0 0x7f502015e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50256bc700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f50200af2e0 0x7f502015e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40214/0 (socket says 172.21.15.81:40214) 2024-05-08T23:33:23.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50256bc700 1 -- 172.21.15.81:0/907331154 learned_addr learned my addr 172.21.15.81:0/907331154 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:23.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f5024ebb700 1 --2- 172.21.15.81:0/907331154 >> v2:172.21.15.117:3300/0 conn(0x7f50200af6c0 0x7f5020071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f5025ebd700 1 --2- 172.21.15.81:0/907331154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f502006d0a0 0x7f502006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50256bc700 1 -- 172.21.15.81:0/907331154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f502006d0a0 msgr2=0x7f502006d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:23.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50256bc700 1 --2- 172.21.15.81:0/907331154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f502006d0a0 0x7f502006d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50256bc700 1 -- 172.21.15.81:0/907331154 >> v2:172.21.15.117:3300/0 conn(0x7f50200af6c0 msgr2=0x7f5020071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:23.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50256bc700 1 --2- 172.21.15.81:0/907331154 >> v2:172.21.15.117:3300/0 conn(0x7f50200af6c0 0x7f5020071ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50256bc700 1 -- 172.21.15.81:0/907331154 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50200b4d20 con 0x7f50200af2e0 2024-05-08T23:33:23.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50256bc700 1 --2- 172.21.15.81:0/907331154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200af2e0 0x7f502015e390 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f502006c480 tx=0x7f501c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3324b91d935852c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:23.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f5017fff700 1 -- 172.21.15.81:0/907331154 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c011830 con 0x7f50200af2e0 2024-05-08T23:33:23.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f5017fff700 1 -- 172.21.15.81:0/907331154 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501c011990 con 0x7f50200af2e0 2024-05-08T23:33:23.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f5017fff700 1 -- 172.21.15.81:0/907331154 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c011d20 con 0x7f50200af2e0 2024-05-08T23:33:23.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50266be700 1 -- 172.21.15.81:0/907331154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200af2e0 msgr2=0x7f502015e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50266be700 1 --2- 172.21.15.81:0/907331154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200af2e0 0x7f502015e390 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f502006c480 tx=0x7f501c009bf0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50266be700 1 -- 172.21.15.81:0/907331154 shutdown_connections 2024-05-08T23:33:23.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50266be700 1 --2- 172.21.15.81:0/907331154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f502006d0a0 0x7f502006d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50266be700 1 --2- 172.21.15.81:0/907331154 >> v2:172.21.15.117:3300/0 conn(0x7f50200af6c0 0x7f5020071ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50266be700 1 --2- 172.21.15.81:0/907331154 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200af2e0 0x7f502015e390 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50266be700 1 -- 172.21.15.81:0/907331154 >> 172.21.15.81:0/907331154 conn(0x7f502006b470 msgr2=0x7f502006b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:23.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50266be700 1 -- 172.21.15.81:0/907331154 shutdown_connections 2024-05-08T23:33:23.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.494+0000 7f50266be700 1 -- 172.21.15.81:0/907331154 wait complete. 2024-05-08T23:33:23.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.498+0000 7f50266be700 1 Processor -- start 2024-05-08T23:33:23.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.498+0000 7f50266be700 1 -- start start 2024-05-08T23:33:23.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f50266be700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f502006d0a0 0x7f5020152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f50266be700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200af2e0 0x7f50200cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f50266be700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200cdca0 0x7f50200b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f50266be700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50200cd880 con 0x7f50200cdca0 2024-05-08T23:33:23.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f50266be700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5020152590 con 0x7f502006d0a0 2024-05-08T23:33:23.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f50266be700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5020155fa0 con 0x7f50200af2e0 2024-05-08T23:33:23.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f5025ebd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200cdca0 0x7f50200b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f5025ebd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200cdca0 0x7f50200b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40216/0 (socket says 172.21.15.81:40216) 2024-05-08T23:33:23.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f5025ebd700 1 -- 172.21.15.81:0/736730587 learned_addr learned my addr 172.21.15.81:0/736730587 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:23.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f50256bc700 1 --2- 172.21.15.81:0/736730587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f502006d0a0 0x7f5020152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f5024ebb700 1 --2- 172.21.15.81:0/736730587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200af2e0 0x7f50200cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f5025ebd700 1 -- 172.21.15.81:0/736730587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200af2e0 msgr2=0x7f50200cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f5025ebd700 1 --2- 172.21.15.81:0/736730587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200af2e0 0x7f50200cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f5025ebd700 1 -- 172.21.15.81:0/736730587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f502006d0a0 msgr2=0x7f5020152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f5025ebd700 1 --2- 172.21.15.81:0/736730587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f502006d0a0 0x7f5020152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f5025ebd700 1 -- 172.21.15.81:0/736730587 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501c009570 con 0x7f50200cdca0 2024-05-08T23:33:23.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f5025ebd700 1 --2- 172.21.15.81:0/736730587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200cdca0 0x7f50200b2c00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f501800eed0 tx=0x7f501800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:23.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f5015ffb700 1 -- 172.21.15.81:0/736730587 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018010040 con 0x7f50200cdca0 2024-05-08T23:33:23.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f5015ffb700 1 -- 172.21.15.81:0/736730587 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501800bc80 con 0x7f50200cdca0 2024-05-08T23:33:23.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f50266be700 1 -- 172.21.15.81:0/736730587 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50200ccc00 con 0x7f50200cdca0 2024-05-08T23:33:23.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f5015ffb700 1 -- 172.21.15.81:0/736730587 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018009e40 con 0x7f50200cdca0 2024-05-08T23:33:23.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f50266be700 1 -- 172.21.15.81:0/736730587 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50200cd1c0 con 0x7f50200cdca0 2024-05-08T23:33:23.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f50266be700 1 -- 172.21.15.81:0/736730587 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50200672a0 con 0x7f50200cdca0 2024-05-08T23:33:23.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.502+0000 7f5015ffb700 1 -- 172.21.15.81:0/736730587 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5018014400 con 0x7f50200cdca0 2024-05-08T23:33:23.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.506+0000 7f5015ffb700 1 --2- 172.21.15.81:0/736730587 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5004066760 0x7f5004068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.506+0000 7f5015ffb700 1 -- 172.21.15.81:0/736730587 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f50180841a0 con 0x7f50200cdca0 2024-05-08T23:33:23.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.506+0000 7f50256bc700 1 --2- 172.21.15.81:0/736730587 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5004066760 0x7f5004068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.506+0000 7f50256bc700 1 --2- 172.21.15.81:0/736730587 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5004066760 0x7f5004068c20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f501c000c00 tx=0x7f501c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:23.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.514+0000 7f5015ffb700 1 -- 172.21.15.81:0/736730587 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5018051810 con 0x7f50200cdca0 2024-05-08T23:33:23.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.682+0000 7f50266be700 1 -- 172.21.15.81:0/736730587 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f50200cfa80 con 0x7f5004066760 2024-05-08T23:33:23.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.690+0000 7f5015ffb700 1 -- 172.21.15.81:0/736730587 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49780 (secure 0 0 0) 0x7f50200cfa80 con 0x7f5004066760 2024-05-08T23:33:23.692 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:23.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.694+0000 7f50266be700 1 -- 172.21.15.81:0/736730587 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5004066760 msgr2=0x7f5004068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.694+0000 7f50266be700 1 --2- 172.21.15.81:0/736730587 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5004066760 0x7f5004068c20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f501c000c00 tx=0x7f501c008030 comp rx=0 tx=0).stop 2024-05-08T23:33:23.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.694+0000 7f50266be700 1 -- 172.21.15.81:0/736730587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200cdca0 msgr2=0x7f50200b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.694+0000 7f50266be700 1 --2- 172.21.15.81:0/736730587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200cdca0 0x7f50200b2c00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f501800eed0 tx=0x7f501800b390 comp rx=0 tx=0).stop 2024-05-08T23:33:23.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.694+0000 7f50266be700 1 -- 172.21.15.81:0/736730587 shutdown_connections 2024-05-08T23:33:23.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.694+0000 7f50266be700 1 --2- 172.21.15.81:0/736730587 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f5004066760 0x7f5004068c20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.694+0000 7f50266be700 1 --2- 172.21.15.81:0/736730587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200cdca0 0x7f50200b2c00 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.694+0000 7f50266be700 1 --2- 172.21.15.81:0/736730587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200af2e0 0x7f50200cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.694+0000 7f50266be700 1 --2- 172.21.15.81:0/736730587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f502006d0a0 0x7f5020152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.694+0000 7f50266be700 1 -- 172.21.15.81:0/736730587 >> 172.21.15.81:0/736730587 conn(0x7f502006b470 msgr2=0x7f50200b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:23.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.694+0000 7f50266be700 1 -- 172.21.15.81:0/736730587 shutdown_connections 2024-05-08T23:33:23.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.694+0000 7f50266be700 1 -- 172.21.15.81:0/736730587 wait complete. 2024-05-08T23:33:23.697 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:33:23.712 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-05-08T23:33:23.380254+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230428,"kb_used_data":6068,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":754744292,"statfs":{"total":773094113280,"available":772858155008,"internally_reserved":0,"allocated":6213632,"data_stored":3260120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":229625400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14891,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001483"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":191,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T23:32:42.485798+0000","last_change":"2024-05-08T23:30:27.226484+0000","last_active":"2024-05-08T23:32:42.485798+0000","last_peered":"2024-05-08T23:32:42.485798+0000","last_clean":"2024-05-08T23:32:42.485798+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:32:42.485798+0000","last_undegraded":"2024-05-08T23:32:42.485798+0000","last_fullsized":"2024-05-08T23:32:42.485798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_clean_scrub_stamp":"2024-05-08T23:30:26.204118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:14.267287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":259859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.89400000000000002,"15min":0.90500000000000003},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1719999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.96899999999999997,"15min":0.97899999999999998},"min":{"1min":0.58299999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1910000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.75800000000000001}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.59099999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6579999999999999,"5min":2.004,"15min":2.004},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.92600000000000005,"15min":0.92400000000000004},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.327,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.0369999999999999}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88300000000000001,"15min":0.88500000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.95999999999999996,"15min":0.97099999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.218,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.56200000000000006}]},{"osd":3,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.89200000000000002,"15min":0.88400000000000001},"min":{"1min":0.65100000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.88},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96899999999999997,"15min":0.97799999999999998},"min":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2290000000000001,"5min":1.272,"15min":1.272},"last":0.84299999999999997}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.71499999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.1439999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0249999999999999,"15min":1.0269999999999999},"min":{"1min":0.67800000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.1919999999999999}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0469999999999999,"15min":1.056},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4450000000000001,"5min":2.109,"15min":2.109},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1240000000000001,"15min":1.1439999999999999},"min":{"1min":0.75800000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.996}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1970000000000001,"15min":1.2070000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5660000000000001,"5min":2.544,"15min":2.544},"last":1.266},{"interface":"front","average":{"1min":1.069,"5min":1.103,"15min":1.109},"min":{"1min":0.81200000000000006,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.2230000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509480,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":253126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2969999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.97099999999999997,"15min":0.98299999999999998},"min":{"1min":0.63700000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1859999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.82699999999999996}]},{"osd":1,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.90900000000000003,"15min":0.90400000000000003},"min":{"1min":0.67300000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88400000000000001,"15min":0.878},"min":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.042}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.91800000000000004,"15min":0.91700000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.385,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.93700000000000006,"15min":0.92500000000000004},"min":{"1min":0.70299999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.454,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.99399999999999999}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.97199999999999998,"15min":0.98399999999999999},"min":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.252,"5min":1.395,"15min":1.395},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.004,"5min":0.96099999999999997,"15min":0.95399999999999996},"min":{"1min":0.63700000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.101}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.995,"15min":1.0129999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.877},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.1339999999999999,"15min":1.1599999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.484,"5min":2.464,"15min":2.464},"last":1.155}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0800000000000001,"15min":1.0820000000000001},"min":{"1min":0.82099999999999995,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.133}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1930000000000001,"15min":1.216},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.21},{"interface":"front","average":{"1min":1.163,"5min":1.1759999999999999,"15min":1.1779999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.2589999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":614400,"data_stored":253708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.84099999999999997,"15min":0.85099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2030000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77900000000000003,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.085}]},{"osd":1,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.70999999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1459999999999999,"5min":1.268,"15min":1.268},"last":1.163},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.88800000000000001,"15min":0.90000000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.125,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.59899999999999998}]},{"osd":2,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.88800000000000001,"15min":0.89800000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89900000000000002,"15min":0.90700000000000003},"min":{"1min":0.65400000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0249999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.85299999999999998,"15min":0.84899999999999998},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1759999999999999,"5min":1.23,"15min":1.23},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2390000000000001,"5min":1.431,"15min":1.431},"last":1.1160000000000001}]},{"osd":5,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3420000000000001,"5min":1.413,"15min":1.413},"last":0.873},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95899999999999996,"15min":0.95099999999999996},"min":{"1min":0.73999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.278,"5min":1.504,"15min":1.504},"last":0.98299999999999998}]},{"osd":6,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0269999999999999,"15min":1.0309999999999999},"min":{"1min":0.73899999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1419999999999999,"15min":1.1559999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3859999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.83499999999999996}]},{"osd":7,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0800000000000001,"15min":1.0840000000000001},"min":{"1min":0.69899999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4379999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.03},{"interface":"front","average":{"1min":1.113,"5min":1.143,"15min":1.1479999999999999},"min":{"1min":0.94799999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4099999999999999,"5min":1.583,"15min":1.583},"last":1.246}]}]},{"osd":3,"up_from":15,"seq":64424509478,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":638976,"data_stored":269404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.49099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.46300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":1.032,"15min":1.032},"last":0.80400000000000005}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70599999999999996,"15min":0.71099999999999997},"min":{"1min":0.50700000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.89300000000000002,"5min":1,"15min":1},"last":0.76400000000000001}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0469999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80200000000000005,"15min":0.79900000000000004},"min":{"1min":0.56699999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0229999999999999,"5min":1.242,"15min":1.242},"last":0.88600000000000001}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.65600000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.129,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.96699999999999997,"15min":0.97399999999999998},"min":{"1min":0.78000000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.244,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.73999999999999999}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.014,"15min":1.022},"min":{"1min":0.80300000000000005,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.161,"5min":1.623,"15min":1.623},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.995,"15min":1.004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1359999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.96499999999999997}]},{"osd":6,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.997,"15min":1.0009999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.232,"5min":1.482,"15min":1.482},"last":1.3080000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.093,"15min":1.111},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3480000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.85299999999999998}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0800000000000001,"15min":1.0860000000000001},"min":{"1min":0.78400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.456,"5min":1.514,"15min":1.514},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0580000000000001,"15min":1.0600000000000001},"min":{"1min":0.75,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.2869999999999999,"5min":1.552,"15min":1.552},"last":1.2250000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":638976,"data_stored":265326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80400000000000005,"5min":1.099,"15min":1.099},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0660000000000001,"5min":1.129,"15min":1.129},"last":0.66700000000000004}]},{"osd":1,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70999999999999996,"15min":0.70899999999999996},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.90200000000000002}]},{"osd":3,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0509999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.79500000000000004}]},{"osd":4,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.96799999999999997,"15min":0.97799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.27,"5min":1.367,"15min":1.367},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.95599999999999996,"15min":0.96399999999999997},"min":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.84099999999999997}]},{"osd":5,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98999999999999999,"15min":0.98599999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.874},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.98099999999999998}]},{"osd":6,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0620000000000001,"15min":1.0740000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4239999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.075,"15min":1.075},"min":{"1min":0.85699999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.95599999999999996}]},{"osd":7,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1220000000000001,"15min":1.1240000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":1.028},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1000000000000001,"15min":1.1020000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.492,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.93000000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":259367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.51300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.44,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.68899999999999995,"15min":0.67000000000000004},"min":{"1min":0.53900000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.73099999999999998}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73499999999999999,"15min":0.72599999999999998},"min":{"1min":0.57999999999999996,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.872},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.42799999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.038,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.63900000000000001}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.95099999999999996,"15min":0.95199999999999996},"min":{"1min":0.67600000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.256,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.95499999999999996,"15min":0.94299999999999995},"min":{"1min":0.77300000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.2709999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":1.0900000000000001}]},{"osd":5,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0109999999999999,"15min":1.0089999999999999},"min":{"1min":0.72299999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.403,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.077,"15min":1.079},"min":{"1min":0.85499999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3879999999999999,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.77200000000000002}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0369999999999999,"15min":1.034},"min":{"1min":0.80000000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.4330000000000001,"5min":1.633,"15min":1.633},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.135,"15min":1.1419999999999999},"min":{"1min":0.80500000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.4750000000000001,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.98099999999999998}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.0209999999999999,"15min":1.0009999999999999},"min":{"1min":0.83099999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.371,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.081,"15min":1.0820000000000001},"min":{"1min":0.84999999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.306,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.95499999999999996}]}]},{"osd":7,"up_from":14,"seq":60129542182,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1224704,"data_stored":849699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.88300000000000001,"15min":0.89700000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.86399999999999999,"15min":0.86699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1379999999999999,"5min":1.359,"15min":1.359},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91200000000000003,"15min":0.91600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.23,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.83899999999999997}]},{"osd":2,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88,"15min":0.873},"min":{"1min":0.57499999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88,"15min":0.88500000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.133,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.76000000000000001}]},{"osd":3,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.91300000000000003,"15min":0.91100000000000003},"min":{"1min":0.60699999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.99299999999999999,"15min":0.99099999999999999},"min":{"1min":0.80100000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2470000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.55700000000000005}]},{"osd":4,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.90400000000000003,"15min":0.91200000000000003},"min":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.73999999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.1899999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.018}]},{"osd":5,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.028,"15min":1.028},"min":{"1min":0.64100000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.405,"5min":1.714,"15min":1.714},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.1950000000000001,"5min":1.325,"15min":1.325},"last":0.91800000000000004}]},{"osd":6,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.048,"15min":1.048},"min":{"1min":0.86599999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.4430000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.091,"5min":1.105,"15min":1.107},"min":{"1min":0.83999999999999997,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.359,"5min":1.496,"15min":1.496},"last":1.0760000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542182,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1224704,"data_stored":849631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.47499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":1.351},"last":0.44500000000000001}]},{"osd":2,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.51500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.238,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.546,"5min":1.869,"15min":1.869},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.84199999999999997,"15min":0.85599999999999998},"min":{"1min":0.55200000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.94199999999999995,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.626}]},{"osd":4,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97199999999999998,"15min":0.97299999999999998},"min":{"1min":0.71999999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.357,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.90800000000000003,"15min":0.89600000000000002},"min":{"1min":0.69899999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.82399999999999995}]},{"osd":5,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.052,"15min":1.0600000000000001},"min":{"1min":0.76800000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3819999999999999,"5min":1.474,"15min":1.474},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.92500000000000004,"15min":0.91400000000000003},"min":{"1min":0.72699999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.67400000000000004}]},{"osd":6,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.056,"15min":1.0720000000000001},"min":{"1min":0.76900000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.171,"5min":1.736,"15min":1.736},"last":0.751},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1180000000000001,"15min":1.121},"min":{"1min":0.89000000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4830000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.65900000000000003}]},{"osd":7,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.113,"15min":1.117},"min":{"1min":0.66600000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.397,"5min":1.72,"15min":1.72},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.097,"5min":1.109,"15min":1.111},"min":{"1min":0.86399999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.69099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:33:23.714 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:33:23.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa4aad6700 1 Processor -- start 2024-05-08T23:33:23.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa4aad6700 1 -- start start 2024-05-08T23:33:23.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa4aad6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa441596a0 0x7faa440b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa4aad6700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa440b3590 con 0x7faa4415a190 2024-05-08T23:33:23.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa4aad6700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa440b24f0 con 0x7faa440b4e50 2024-05-08T23:33:23.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa4aad6700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa440b27d0 con 0x7faa441596a0 2024-05-08T23:33:23.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa4a2d5700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7faa4415a190 0x7faa440709a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46196/0 (socket says 172.21.15.81:46196) 2024-05-08T23:33:23.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa4a2d5700 1 -- 172.21.15.81:0/1462600972 learned_addr learned my addr 172.21.15.81:0/1462600972 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:23.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa49ad4700 1 --2- 172.21.15.81:0/1462600972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa441596a0 0x7faa440b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa48ad2700 1 -- 172.21.15.81:0/1462600972 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2440450970 0 0) 0x7faa440b3590 con 0x7faa4415a190 2024-05-08T23:33:23.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa48ad2700 1 -- 172.21.15.81:0/1462600972 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa2c0035d0 con 0x7faa4415a190 2024-05-08T23:33:23.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa48ad2700 1 -- 172.21.15.81:0/1462600972 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459255999 0 0) 0x7faa440b24f0 con 0x7faa440b4e50 2024-05-08T23:33:23.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa48ad2700 1 -- 172.21.15.81:0/1462600972 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa440b3590 con 0x7faa440b4e50 2024-05-08T23:33:23.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa49ad4700 1 -- 172.21.15.81:0/1462600972 >> v1:172.21.15.117:6789/0 conn(0x7faa440b4e50 legacy=0x7faa44159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa49ad4700 1 -- 172.21.15.81:0/1462600972 >> v1:172.21.15.81:6789/0 conn(0x7faa4415a190 legacy=0x7faa440709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.826+0000 7faa49ad4700 1 -- 172.21.15.81:0/1462600972 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa440b2ab0 con 0x7faa441596a0 2024-05-08T23:33:23.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.830+0000 7faa49ad4700 1 --2- 172.21.15.81:0/1462600972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa441596a0 0x7faa440b4910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faa38009880 tx=0x7faa38009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e27a9ef1b6cb40c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:23.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.830+0000 7faa48ad2700 1 -- 172.21.15.81:0/1462600972 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38011830 con 0x7faa441596a0 2024-05-08T23:33:23.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.830+0000 7faa48ad2700 1 -- 172.21.15.81:0/1462600972 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa38011990 con 0x7faa441596a0 2024-05-08T23:33:23.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.830+0000 7faa48ad2700 1 -- 172.21.15.81:0/1462600972 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38011d00 con 0x7faa441596a0 2024-05-08T23:33:23.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.830+0000 7faa4aad6700 1 -- 172.21.15.81:0/1462600972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa441596a0 msgr2=0x7faa440b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.830+0000 7faa4aad6700 1 --2- 172.21.15.81:0/1462600972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa441596a0 0x7faa440b4910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faa38009880 tx=0x7faa38009b90 comp rx=0 tx=0).stop 2024-05-08T23:33:23.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.830+0000 7faa4aad6700 1 -- 172.21.15.81:0/1462600972 shutdown_connections 2024-05-08T23:33:23.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.830+0000 7faa4aad6700 1 --2- 172.21.15.81:0/1462600972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa441596a0 0x7faa440b4910 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.830+0000 7faa4aad6700 1 -- 172.21.15.81:0/1462600972 >> 172.21.15.81:0/1462600972 conn(0x7faa4406c330 msgr2=0x7faa4406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:23.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.830+0000 7faa4aad6700 1 -- 172.21.15.81:0/1462600972 shutdown_connections 2024-05-08T23:33:23.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.830+0000 7faa4aad6700 1 -- 172.21.15.81:0/1462600972 wait complete. 2024-05-08T23:33:23.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.830+0000 7faa4aad6700 1 Processor -- start 2024-05-08T23:33:23.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.830+0000 7faa4aad6700 1 -- start start 2024-05-08T23:33:23.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa4aad6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa440b4e50 0x7faa4414db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa4aad6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa441596a0 0x7faa440c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa4aad6700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa4415a190 0x7faa440cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa4aad6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa440ca3c0 con 0x7faa440b4e50 2024-05-08T23:33:23.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa4aad6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa440c9740 con 0x7faa441596a0 2024-05-08T23:33:23.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa4aad6700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa440c9a20 con 0x7faa4415a190 2024-05-08T23:33:23.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa49ad4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa440b4e50 0x7faa4414db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa49ad4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa440b4e50 0x7faa4414db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40220/0 (socket says 172.21.15.81:40220) 2024-05-08T23:33:23.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa49ad4700 1 -- 172.21.15.81:0/1960227727 learned_addr learned my addr 172.21.15.81:0/1960227727 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:23.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa492d3700 1 --2- 172.21.15.81:0/1960227727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa441596a0 0x7faa440c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa4a2d5700 1 --2- 172.21.15.81:0/1960227727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa4415a190 0x7faa440cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa49ad4700 1 -- 172.21.15.81:0/1960227727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa4415a190 msgr2=0x7faa440cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa49ad4700 1 --2- 172.21.15.81:0/1960227727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa4415a190 0x7faa440cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa49ad4700 1 -- 172.21.15.81:0/1960227727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa441596a0 msgr2=0x7faa440c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:23.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa49ad4700 1 --2- 172.21.15.81:0/1960227727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa441596a0 0x7faa440c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:23.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa49ad4700 1 -- 172.21.15.81:0/1960227727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa380094b0 con 0x7faa440b4e50 2024-05-08T23:33:23.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa49ad4700 1 --2- 172.21.15.81:0/1960227727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa440b4e50 0x7faa4414db10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7faa380090b0 tx=0x7faa38004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:23.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa427fc700 1 -- 172.21.15.81:0/1960227727 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38004510 con 0x7faa440b4e50 2024-05-08T23:33:23.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa427fc700 1 -- 172.21.15.81:0/1960227727 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa380036c0 con 0x7faa440b4e50 2024-05-08T23:33:23.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa4aad6700 1 -- 172.21.15.81:0/1960227727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa440c9d00 con 0x7faa440b4e50 2024-05-08T23:33:23.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa427fc700 1 -- 172.21.15.81:0/1960227727 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38003a70 con 0x7faa440b4e50 2024-05-08T23:33:23.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa4aad6700 1 -- 172.21.15.81:0/1960227727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa44156880 con 0x7faa440b4e50 2024-05-08T23:33:23.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.834+0000 7faa4aad6700 1 -- 172.21.15.81:0/1960227727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa44151b50 con 0x7faa440b4e50 2024-05-08T23:33:23.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.838+0000 7faa427fc700 1 -- 172.21.15.81:0/1960227727 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faa38003bd0 con 0x7faa440b4e50 2024-05-08T23:33:23.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.838+0000 7faa427fc700 1 --2- 172.21.15.81:0/1960227727 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa30066730 0x7faa30068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:23.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.838+0000 7faa492d3700 1 --2- 172.21.15.81:0/1960227727 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa30066730 0x7faa30068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:23.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.838+0000 7faa427fc700 1 -- 172.21.15.81:0/1960227727 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7faa38085240 con 0x7faa440b4e50 2024-05-08T23:33:23.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.838+0000 7faa492d3700 1 --2- 172.21.15.81:0/1960227727 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa30066730 0x7faa30068bf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faa4415c980 tx=0x7faa34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:23.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:23.842+0000 7faa427fc700 1 -- 172.21.15.81:0/1960227727 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faa38052940 con 0x7faa440b4e50 2024-05-08T23:33:24.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:24.014+0000 7faa4aad6700 1 -- 172.21.15.81:0/1960227727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa44156a10 con 0x7faa440b4e50 2024-05-08T23:33:24.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:24.018+0000 7faa427fc700 1 -- 172.21.15.81:0/1960227727 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+12520 (secure 0 0 0) 0x7faa38055f60 con 0x7faa440b4e50 2024-05-08T23:33:24.021 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:24.021 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":55,"fsid":"dee3de5e-bc15-439a-9578-b5583aefc890","created":"2024-05-08T23:30:14.270244+0000","modified":"2024-05-08T23:33:21.043897+0000","last_up_change":"2024-05-08T23:30:25.200740+0000","last_in_change":"2024-05-08T23:30:16.339862+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:30:25.389387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1d689f2b-d609-4b84-a66c-04b9ec10ba36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":26091},{"type":"v1","addr":"172.21.15.81:6801","nonce":26091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":26091},{"type":"v1","addr":"172.21.15.81:6803","nonce":26091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":26091},{"type":"v1","addr":"172.21.15.81:6810","nonce":26091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":26091},{"type":"v1","addr":"172.21.15.81:6806","nonce":26091}]},"public_addr":"172.21.15.81:6801/26091","cluster_addr":"172.21.15.81:6803/26091","heartbeat_back_addr":"172.21.15.81:6810/26091","heartbeat_front_addr":"172.21.15.81:6806/26091","state":["exists","up"]},{"osd":1,"uuid":"32ab34f3-4f16-44dd-9f62-91dc5e94cf83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":26093},{"type":"v1","addr":"172.21.15.81:6818","nonce":26093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":26093},{"type":"v1","addr":"172.21.15.81:6822","nonce":26093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6829","nonce":26093},{"type":"v1","addr":"172.21.15.81:6831","nonce":26093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":26093},{"type":"v1","addr":"172.21.15.81:6826","nonce":26093}]},"public_addr":"172.21.15.81:6818/26093","cluster_addr":"172.21.15.81:6822/26093","heartbeat_back_addr":"172.21.15.81:6831/26093","heartbeat_front_addr":"172.21.15.81:6826/26093","state":["exists","up"]},{"osd":2,"uuid":"ad0e9a48-6766-4d8e-9fcc-92ecbe9dab47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6817","nonce":26092},{"type":"v1","addr":"172.21.15.81:6819","nonce":26092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6821","nonce":26092},{"type":"v1","addr":"172.21.15.81:6823","nonce":26092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":26092},{"type":"v1","addr":"172.21.15.81:6830","nonce":26092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6825","nonce":26092},{"type":"v1","addr":"172.21.15.81:6827","nonce":26092}]},"public_addr":"172.21.15.81:6819/26092","cluster_addr":"172.21.15.81:6823/26092","heartbeat_back_addr":"172.21.15.81:6830/26092","heartbeat_front_addr":"172.21.15.81:6827/26092","state":["exists","up"]},{"osd":3,"uuid":"caedb186-c021-440e-aff2-889bdaf1f137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6805","nonce":26094},{"type":"v1","addr":"172.21.15.81:6807","nonce":26094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6809","nonce":26094},{"type":"v1","addr":"172.21.15.81:6811","nonce":26094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":26094},{"type":"v1","addr":"172.21.15.81:6815","nonce":26094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":26094},{"type":"v1","addr":"172.21.15.81:6813","nonce":26094}]},"public_addr":"172.21.15.81:6807/26094","cluster_addr":"172.21.15.81:6811/26094","heartbeat_back_addr":"172.21.15.81:6815/26094","heartbeat_front_addr":"172.21.15.81:6813/26094","state":["exists","up"]},{"osd":4,"uuid":"9aaee44f-bb9e-4014-af3a-a8b421885b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":26388},{"type":"v1","addr":"172.21.15.117:6809","nonce":26388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":26388},{"type":"v1","addr":"172.21.15.117:6811","nonce":26388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":26388},{"type":"v1","addr":"172.21.15.117:6815","nonce":26388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":26388},{"type":"v1","addr":"172.21.15.117:6813","nonce":26388}]},"public_addr":"172.21.15.117:6809/26388","cluster_addr":"172.21.15.117:6811/26388","heartbeat_back_addr":"172.21.15.117:6815/26388","heartbeat_front_addr":"172.21.15.117:6813/26388","state":["exists","up"]},{"osd":5,"uuid":"98cce22f-822c-43fc-a4de-c9f990847a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":26391},{"type":"v1","addr":"172.21.15.117:6817","nonce":26391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":26391},{"type":"v1","addr":"172.21.15.117:6819","nonce":26391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":26391},{"type":"v1","addr":"172.21.15.117:6823","nonce":26391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":26391},{"type":"v1","addr":"172.21.15.117:6821","nonce":26391}]},"public_addr":"172.21.15.117:6817/26391","cluster_addr":"172.21.15.117:6819/26391","heartbeat_back_addr":"172.21.15.117:6823/26391","heartbeat_front_addr":"172.21.15.117:6821/26391","state":["exists","up"]},{"osd":6,"uuid":"0b353abc-6eed-4974-8d15-f9d1e8e9c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":26390},{"type":"v1","addr":"172.21.15.117:6825","nonce":26390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":26390},{"type":"v1","addr":"172.21.15.117:6827","nonce":26390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":26390},{"type":"v1","addr":"172.21.15.117:6831","nonce":26390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":26390},{"type":"v1","addr":"172.21.15.117:6829","nonce":26390}]},"public_addr":"172.21.15.117:6825/26390","cluster_addr":"172.21.15.117:6827/26390","heartbeat_back_addr":"172.21.15.117:6831/26390","heartbeat_front_addr":"172.21.15.117:6829/26390","state":["exists","up"]},{"osd":7,"uuid":"c2c70905-4d88-4004-81c2-1475901fd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":26389},{"type":"v1","addr":"172.21.15.117:6801","nonce":26389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":26389},{"type":"v1","addr":"172.21.15.117:6803","nonce":26389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":26389},{"type":"v1","addr":"172.21.15.117:6807","nonce":26389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":26389},{"type":"v1","addr":"172.21.15.117:6805","nonce":26389}]},"public_addr":"172.21.15.117:6801/26389","cluster_addr":"172.21.15.117:6803/26389","heartbeat_back_addr":"172.21.15.117:6807/26389","heartbeat_front_addr":"172.21.15.117:6805/26389","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.184814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.609473+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.812215+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.614260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.656298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.627002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.859834+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:30:21.307252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6835/3401411259":"2024-05-09T23:33:18.023961+0000","172.21.15.117:6832/3401411259":"2024-05-09T23:33:18.023961+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:33:24.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:24.022+0000 7faa4aad6700 1 -- 172.21.15.81:0/1960227727 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa30066730 msgr2=0x7faa30068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:24.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:24.022+0000 7faa4aad6700 1 --2- 172.21.15.81:0/1960227727 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa30066730 0x7faa30068bf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faa4415c980 tx=0x7faa34009040 comp rx=0 tx=0).stop 2024-05-08T23:33:24.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:24.022+0000 7faa4aad6700 1 -- 172.21.15.81:0/1960227727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa440b4e50 msgr2=0x7faa4414db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:24.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:24.022+0000 7faa4aad6700 1 --2- 172.21.15.81:0/1960227727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa440b4e50 0x7faa4414db10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7faa380090b0 tx=0x7faa38004770 comp rx=0 tx=0).stop 2024-05-08T23:33:24.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:24.022+0000 7faa4aad6700 1 -- 172.21.15.81:0/1960227727 shutdown_connections 2024-05-08T23:33:24.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:24.022+0000 7faa4aad6700 1 --2- 172.21.15.81:0/1960227727 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7faa30066730 0x7faa30068bf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:24.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:24.022+0000 7faa4aad6700 1 --2- 172.21.15.81:0/1960227727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa4415a190 0x7faa440cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:24.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:24.022+0000 7faa4aad6700 1 --2- 172.21.15.81:0/1960227727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa441596a0 0x7faa440c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:24.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:24.022+0000 7faa4aad6700 1 --2- 172.21.15.81:0/1960227727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa440b4e50 0x7faa4414db10 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:24.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:24.022+0000 7faa4aad6700 1 -- 172.21.15.81:0/1960227727 >> 172.21.15.81:0/1960227727 conn(0x7faa4406c330 msgr2=0x7faa44158720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:24.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:24.022+0000 7faa4aad6700 1 -- 172.21.15.81:0/1960227727 shutdown_connections 2024-05-08T23:33:24.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:24.022+0000 7faa4aad6700 1 -- 172.21.15.81:0/1960227727 wait complete. 2024-05-08T23:33:25.042 INFO:tasks.ceph:Scrubbing osd.0 2024-05-08T23:33:25.042 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:33:25.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.166+0000 7f2caebc3700 1 Processor -- start 2024-05-08T23:33:25.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.166+0000 7f2caebc3700 1 -- start start 2024-05-08T23:33:25.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.166+0000 7f2caebc3700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2ca815b130 0x7f2ca80b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.166+0000 7f2caebc3700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2ca80b3f30 0x7f2ca815b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.166+0000 7f2caebc3700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca80b3020 con 0x7f2ca80cc4e0 2024-05-08T23:33:25.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.166+0000 7f2caebc3700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca80ccf30 con 0x7f2ca80b3f30 2024-05-08T23:33:25.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.166+0000 7f2caebc3700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca80b2000 con 0x7f2ca815b130 2024-05-08T23:33:25.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.166+0000 7f2cad160700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2ca80b3f30 0x7f2ca815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.166+0000 7f2ca7fff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2ca80cc4e0 0x7f2ca80b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46204/0 (socket says 172.21.15.81:46204) 2024-05-08T23:33:25.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2cad160700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2ca80b3f30 0x7f2ca815b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40230/0 (socket says 172.21.15.81:40230) 2024-05-08T23:33:25.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2ca7fff700 1 -- 172.21.15.81:0/3360449379 learned_addr learned my addr 172.21.15.81:0/3360449379 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:25.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2cac95f700 1 --2- 172.21.15.81:0/3360449379 >> v2:172.21.15.117:3300/0 conn(0x7f2ca815b130 0x7f2ca80b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2cad160700 1 -- 172.21.15.81:0/3360449379 >> v2:172.21.15.117:3300/0 conn(0x7f2ca815b130 msgr2=0x7f2ca80b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:25.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2cad160700 1 --2- 172.21.15.81:0/3360449379 >> v2:172.21.15.117:3300/0 conn(0x7f2ca815b130 0x7f2ca80b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2cad160700 1 -- 172.21.15.81:0/3360449379 >> v1:172.21.15.81:6789/0 conn(0x7f2ca80cc4e0 legacy=0x7f2ca80b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2cad160700 1 -- 172.21.15.81:0/3360449379 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80b22e0 con 0x7f2ca80b3f30 2024-05-08T23:33:25.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2cad160700 1 --2- 172.21.15.81:0/3360449379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca80b3f30 0x7f2ca815b510 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2ca806b050 tx=0x7f2ca000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd392088ee55dcf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:25.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2ca77fe700 1 -- 172.21.15.81:0/3360449379 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000e070 con 0x7f2ca80b3f30 2024-05-08T23:33:25.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2ca77fe700 1 -- 172.21.15.81:0/3360449379 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca000bbd0 con 0x7f2ca80b3f30 2024-05-08T23:33:25.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2ca77fe700 1 -- 172.21.15.81:0/3360449379 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0012430 con 0x7f2ca80b3f30 2024-05-08T23:33:25.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2caebc3700 1 -- 172.21.15.81:0/3360449379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca80b3f30 msgr2=0x7f2ca815b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2caebc3700 1 --2- 172.21.15.81:0/3360449379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca80b3f30 0x7f2ca815b510 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2ca806b050 tx=0x7f2ca000b330 comp rx=0 tx=0).stop 2024-05-08T23:33:25.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2caebc3700 1 -- 172.21.15.81:0/3360449379 shutdown_connections 2024-05-08T23:33:25.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2caebc3700 1 --2- 172.21.15.81:0/3360449379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca80b3f30 0x7f2ca815b510 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2caebc3700 1 --2- 172.21.15.81:0/3360449379 >> v2:172.21.15.117:3300/0 conn(0x7f2ca815b130 0x7f2ca80b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2caebc3700 1 -- 172.21.15.81:0/3360449379 >> 172.21.15.81:0/3360449379 conn(0x7f2ca806a360 msgr2=0x7f2ca806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:25.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2caebc3700 1 -- 172.21.15.81:0/3360449379 shutdown_connections 2024-05-08T23:33:25.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.170+0000 7f2caebc3700 1 -- 172.21.15.81:0/3360449379 wait complete. 2024-05-08T23:33:25.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2caebc3700 1 Processor -- start 2024-05-08T23:33:25.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2caebc3700 1 -- start start 2024-05-08T23:33:25.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2caebc3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca80b3f30 0x7f2ca8070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2caebc3700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ca80cc4e0 0x7f2ca8071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2caebc3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca815b130 0x7f2ca80b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2caebc3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80ac8e0 con 0x7f2ca80b3f30 2024-05-08T23:33:25.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2caebc3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca8072230 con 0x7f2ca815b130 2024-05-08T23:33:25.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2caebc3700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca8071180 con 0x7f2ca80cc4e0 2024-05-08T23:33:25.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2cac95f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca80b3f30 0x7f2ca8070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2cac95f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca80b3f30 0x7f2ca8070ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40244/0 (socket says 172.21.15.81:40244) 2024-05-08T23:33:25.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2cac95f700 1 -- 172.21.15.81:0/3066039249 learned_addr learned my addr 172.21.15.81:0/3066039249 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:25.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2cad160700 1 --2- 172.21.15.81:0/3066039249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca815b130 0x7f2ca80b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2ca7fff700 1 --2- 172.21.15.81:0/3066039249 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ca80cc4e0 0x7f2ca8071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2cac95f700 1 -- 172.21.15.81:0/3066039249 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ca80cc4e0 msgr2=0x7f2ca8071cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2cac95f700 1 --2- 172.21.15.81:0/3066039249 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ca80cc4e0 0x7f2ca8071cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2cac95f700 1 -- 172.21.15.81:0/3066039249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca815b130 msgr2=0x7f2ca80b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2cac95f700 1 --2- 172.21.15.81:0/3066039249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca815b130 0x7f2ca80b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2cac95f700 1 -- 172.21.15.81:0/3066039249 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca0009a70 con 0x7f2ca80b3f30 2024-05-08T23:33:25.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2cac95f700 1 --2- 172.21.15.81:0/3066039249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca80b3f30 0x7f2ca8070ba0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00d800 tx=0x7f2c9c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:25.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2ca57fa700 1 -- 172.21.15.81:0/3066039249 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0125f0 con 0x7f2ca80b3f30 2024-05-08T23:33:25.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2ca57fa700 1 -- 172.21.15.81:0/3066039249 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9c012750 con 0x7f2ca80b3f30 2024-05-08T23:33:25.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2caebc3700 1 -- 172.21.15.81:0/3066039249 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca8071460 con 0x7f2ca80b3f30 2024-05-08T23:33:25.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2ca57fa700 1 -- 172.21.15.81:0/3066039249 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c012ac0 con 0x7f2ca80b3f30 2024-05-08T23:33:25.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2caebc3700 1 -- 172.21.15.81:0/3066039249 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca80acda0 con 0x7f2ca80b3f30 2024-05-08T23:33:25.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.174+0000 7f2caebc3700 1 -- 172.21.15.81:0/3066039249 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2ca80cd210 con 0x7f2ca80b3f30 2024-05-08T23:33:25.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.178+0000 7f2ca57fa700 1 -- 172.21.15.81:0/3066039249 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2c9c012c20 con 0x7f2ca80b3f30 2024-05-08T23:33:25.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.178+0000 7f2ca57fa700 1 --2- 172.21.15.81:0/3066039249 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2c94066730 0x7f2c94068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.178+0000 7f2ca7fff700 1 --2- 172.21.15.81:0/3066039249 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2c94066730 0x7f2c94068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.178+0000 7f2ca57fa700 1 -- 172.21.15.81:0/3066039249 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2c9c0849f0 con 0x7f2ca80b3f30 2024-05-08T23:33:25.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.178+0000 7f2ca57fa700 1 --2- 172.21.15.81:0/3066039249 >> [v2:172.21.15.81:6800/26091,v1:172.21.15.81:6801/26091] conn(0x7f2c9406c590 0x7f2c9406e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.178+0000 7f2ca57fa700 1 -- 172.21.15.81:0/3066039249 --> [v2:172.21.15.81:6800/26091,v1:172.21.15.81:6801/26091] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c9406f060 con 0x7f2c9406c590 2024-05-08T23:33:25.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.178+0000 7f2ca57fa700 1 -- 172.21.15.81:0/3066039249 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c9c084d60 con 0x7f2ca80b3f30 2024-05-08T23:33:25.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.178+0000 7f2cad160700 1 --2- 172.21.15.81:0/3066039249 >> [v2:172.21.15.81:6800/26091,v1:172.21.15.81:6801/26091] conn(0x7f2c9406c590 0x7f2c9406e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.178+0000 7f2ca7fff700 1 --2- 172.21.15.81:0/3066039249 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2c94066730 0x7f2c94068bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2ca806c630 tx=0x7f2c98009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:25.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.178+0000 7f2cad160700 1 --2- 172.21.15.81:0/3066039249 >> [v2:172.21.15.81:6800/26091,v1:172.21.15.81:6801/26091] conn(0x7f2c9406c590 0x7f2c9406e9b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:25.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.182+0000 7f2ca57fa700 1 -- 172.21.15.81:0/3066039249 <== osd.0 v2:172.21.15.81:6800/26091 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f2c9406f060 con 0x7f2c9406c590 2024-05-08T23:33:25.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.198+0000 7f2caebc3700 1 -- 172.21.15.81:0/3066039249 --> [v2:172.21.15.81:6800/26091,v1:172.21.15.81:6801/26091] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2ca8071650 con 0x7f2c9406c590 2024-05-08T23:33:25.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.198+0000 7f2ca57fa700 1 -- 172.21.15.81:0/3066039249 <== osd.0 v2:172.21.15.81:6800/26091 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2ca8071650 con 0x7f2c9406c590 2024-05-08T23:33:25.203 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:33:25.203 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:33:25.203 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:33:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.202+0000 7f2caebc3700 1 -- 172.21.15.81:0/3066039249 >> [v2:172.21.15.81:6800/26091,v1:172.21.15.81:6801/26091] conn(0x7f2c9406c590 msgr2=0x7f2c9406e9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.202+0000 7f2caebc3700 1 --2- 172.21.15.81:0/3066039249 >> [v2:172.21.15.81:6800/26091,v1:172.21.15.81:6801/26091] conn(0x7f2c9406c590 0x7f2c9406e9b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.202+0000 7f2caebc3700 1 -- 172.21.15.81:0/3066039249 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2c94066730 msgr2=0x7f2c94068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.202+0000 7f2caebc3700 1 --2- 172.21.15.81:0/3066039249 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2c94066730 0x7f2c94068bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2ca806c630 tx=0x7f2c98009040 comp rx=0 tx=0).stop 2024-05-08T23:33:25.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.202+0000 7f2caebc3700 1 -- 172.21.15.81:0/3066039249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca80b3f30 msgr2=0x7f2ca8070ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.202+0000 7f2caebc3700 1 --2- 172.21.15.81:0/3066039249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca80b3f30 0x7f2ca8070ba0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00d800 tx=0x7f2c9c00db10 comp rx=0 tx=0).stop 2024-05-08T23:33:25.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.202+0000 7f2cad160700 1 -- 172.21.15.81:0/3066039249 reap_dead start 2024-05-08T23:33:25.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.202+0000 7f2caebc3700 1 -- 172.21.15.81:0/3066039249 shutdown_connections 2024-05-08T23:33:25.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.202+0000 7f2caebc3700 1 -- 172.21.15.81:0/3066039249 >> 172.21.15.81:0/3066039249 conn(0x7f2ca806a360 msgr2=0x7f2ca8158860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:25.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.202+0000 7f2caebc3700 1 -- 172.21.15.81:0/3066039249 shutdown_connections 2024-05-08T23:33:25.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.202+0000 7f2caebc3700 1 -- 172.21.15.81:0/3066039249 wait complete. 2024-05-08T23:33:25.223 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-08T23:33:25.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.342+0000 7ff87d8f4700 1 Processor -- start 2024-05-08T23:33:25.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.342+0000 7ff87d8f4700 1 -- start start 2024-05-08T23:33:25.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff87d8f4700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff8781596a0 0x7ff8780b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff87d8f4700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8780b3590 con 0x7ff8780b4e50 2024-05-08T23:33:25.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff87d8f4700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8780b24f0 con 0x7ff87815a190 2024-05-08T23:33:25.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff87d8f4700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8780b27d0 con 0x7ff8781596a0 2024-05-08T23:33:25.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff877fff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff8780b4e50 0x7ff878159a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46216/0 (socket says 172.21.15.81:46216) 2024-05-08T23:33:25.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff877fff700 1 -- 172.21.15.81:0/3869083988 learned_addr learned my addr 172.21.15.81:0/3869083988 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:25.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff87c8f2700 1 --2- 172.21.15.81:0/3869083988 >> v2:172.21.15.117:3300/0 conn(0x7ff8781596a0 0x7ff8780b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019652798 0 0) 0x7ff8780b3590 con 0x7ff8780b4e50 2024-05-08T23:33:25.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8600035d0 con 0x7ff8780b4e50 2024-05-08T23:33:25.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2733827887 0 0) 0x7ff8780b24f0 con 0x7ff87815a190 2024-05-08T23:33:25.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8780b3590 con 0x7ff87815a190 2024-05-08T23:33:25.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 250671416 0 0) 0x7ff8600035d0 con 0x7ff8780b4e50 2024-05-08T23:33:25.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 >> v2:172.21.15.117:3300/0 conn(0x7ff8781596a0 msgr2=0x7ff8780b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:25.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 --2- 172.21.15.81:0/3869083988 >> v2:172.21.15.117:3300/0 conn(0x7ff8781596a0 0x7ff8780b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 >> v1:172.21.15.117:6789/0 conn(0x7ff87815a190 legacy=0x7ff8780709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8780b2ab0 con 0x7ff8780b4e50 2024-05-08T23:33:25.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1562847242 0 0) 0x7ff868001670 con 0x7ff8780b4e50 2024-05-08T23:33:25.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 >> v1:172.21.15.81:6789/0 conn(0x7ff8780b4e50 legacy=0x7ff878159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 --2- 172.21.15.81:0/3869083988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff860003bb0 0x7ff860003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 --2- 172.21.15.81:0/3869083988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff860004870 0x7ff860004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 --2- 172.21.15.81:0/3869083988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff860005230 0x7ff86000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780b27d0 con 0x7ff860004870 2024-05-08T23:33:25.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86000e2c0 con 0x7ff860005230 2024-05-08T23:33:25.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86000e480 con 0x7ff860003bb0 2024-05-08T23:33:25.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff877fff700 1 --2- 172.21.15.81:0/3869083988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff860004870 0x7ff860004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff87d0f3700 1 --2- 172.21.15.81:0/3869083988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff860005230 0x7ff86000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff877fff700 1 -- 172.21.15.81:0/3869083988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff860003bb0 msgr2=0x7ff860003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:25.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff877fff700 1 --2- 172.21.15.81:0/3869083988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff860003bb0 0x7ff860003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff877fff700 1 -- 172.21.15.81:0/3869083988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff860005230 msgr2=0x7ff86000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:25.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff877fff700 1 --2- 172.21.15.81:0/3869083988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff860005230 0x7ff86000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff87d0f3700 1 -- 172.21.15.81:0/3869083988 reap_dead start 2024-05-08T23:33:25.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff877fff700 1 -- 172.21.15.81:0/3869083988 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8600035d0 con 0x7ff860004870 2024-05-08T23:33:25.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff877fff700 1 --2- 172.21.15.81:0/3869083988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff860004870 0x7ff860004cf0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff868007fd0 tx=0x7ff8600058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be9ab9ee45eb9b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:25.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86800cd80 con 0x7ff860004870 2024-05-08T23:33:25.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff868001af0 con 0x7ff860004870 2024-05-08T23:33:25.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff8777fe700 1 -- 172.21.15.81:0/3869083988 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868005bc0 con 0x7ff860004870 2024-05-08T23:33:25.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff87d8f4700 1 -- 172.21.15.81:0/3869083988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff860004870 msgr2=0x7ff860004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff87d8f4700 1 --2- 172.21.15.81:0/3869083988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff860004870 0x7ff860004cf0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff868007fd0 tx=0x7ff8600058a0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff87d8f4700 1 -- 172.21.15.81:0/3869083988 shutdown_connections 2024-05-08T23:33:25.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff87d8f4700 1 --2- 172.21.15.81:0/3869083988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff860004870 0x7ff860004cf0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff87d8f4700 1 -- 172.21.15.81:0/3869083988 >> 172.21.15.81:0/3869083988 conn(0x7ff87806c330 msgr2=0x7ff87806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:25.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff87d8f4700 1 -- 172.21.15.81:0/3869083988 shutdown_connections 2024-05-08T23:33:25.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.346+0000 7ff87d8f4700 1 -- 172.21.15.81:0/3869083988 wait complete. 2024-05-08T23:33:25.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87d8f4700 1 Processor -- start 2024-05-08T23:33:25.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87d8f4700 1 -- start start 2024-05-08T23:33:25.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87d8f4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff860004870 0x7ff8780af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87d8f4700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff878149820 0x7ff878150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87d8f4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff87814b750 0x7ff87814bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87d8f4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff878150b40 con 0x7ff860004870 2024-05-08T23:33:25.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87d8f4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780aff40 con 0x7ff87814b750 2024-05-08T23:33:25.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87d8f4700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780b0220 con 0x7ff878149820 2024-05-08T23:33:25.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87c8f2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff860004870 0x7ff8780af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87c8f2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff860004870 0x7ff8780af8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40250/0 (socket says 172.21.15.81:40250) 2024-05-08T23:33:25.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87c8f2700 1 -- 172.21.15.81:0/1710107494 learned_addr learned my addr 172.21.15.81:0/1710107494 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:25.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff877fff700 1 --2- 172.21.15.81:0/1710107494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff878149820 0x7ff878150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87d0f3700 1 --2- 172.21.15.81:0/1710107494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff87814b750 0x7ff87814bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87c8f2700 1 -- 172.21.15.81:0/1710107494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff878149820 msgr2=0x7ff878150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87c8f2700 1 --2- 172.21.15.81:0/1710107494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff878149820 0x7ff878150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87c8f2700 1 -- 172.21.15.81:0/1710107494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff87814b750 msgr2=0x7ff87814bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87c8f2700 1 --2- 172.21.15.81:0/1710107494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff87814b750 0x7ff87814bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87c8f2700 1 -- 172.21.15.81:0/1710107494 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8781e2ed0 con 0x7ff860004870 2024-05-08T23:33:25.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.350+0000 7ff87c8f2700 1 --2- 172.21.15.81:0/1710107494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff860004870 0x7ff8780af8f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00e810 tx=0x7ff86c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:25.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.354+0000 7ff8757fa700 1 -- 172.21.15.81:0/1710107494 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c011800 con 0x7ff860004870 2024-05-08T23:33:25.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.354+0000 7ff8757fa700 1 -- 172.21.15.81:0/1710107494 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c011980 con 0x7ff860004870 2024-05-08T23:33:25.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.354+0000 7ff87d8f4700 1 -- 172.21.15.81:0/1710107494 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8781e31b0 con 0x7ff860004870 2024-05-08T23:33:25.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.354+0000 7ff8757fa700 1 -- 172.21.15.81:0/1710107494 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c011d30 con 0x7ff860004870 2024-05-08T23:33:25.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.354+0000 7ff87d8f4700 1 -- 172.21.15.81:0/1710107494 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8780b0500 con 0x7ff860004870 2024-05-08T23:33:25.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.354+0000 7ff87d8f4700 1 -- 172.21.15.81:0/1710107494 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8780684b0 con 0x7ff860004870 2024-05-08T23:33:25.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.354+0000 7ff8757fa700 1 -- 172.21.15.81:0/1710107494 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff86c010350 con 0x7ff860004870 2024-05-08T23:33:25.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.354+0000 7ff8757fa700 1 --2- 172.21.15.81:0/1710107494 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff864066730 0x7ff864068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.358+0000 7ff8757fa700 1 -- 172.21.15.81:0/1710107494 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff86c083710 con 0x7ff860004870 2024-05-08T23:33:25.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.358+0000 7ff877fff700 1 --2- 172.21.15.81:0/1710107494 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff864066730 0x7ff864068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.358+0000 7ff877fff700 1 --2- 172.21.15.81:0/1710107494 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff864066730 0x7ff864068bf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff8680018b0 tx=0x7ff86800a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:25.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.362+0000 7ff8757fa700 1 -- 172.21.15.81:0/1710107494 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff86c050d80 con 0x7ff860004870 2024-05-08T23:33:25.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.534+0000 7ff87d8f4700 1 -- 172.21.15.81:0/1710107494 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8780ae740 con 0x7ff864066730 2024-05-08T23:33:25.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.534+0000 7ff8757fa700 1 -- 172.21.15.81:0/1710107494 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff8780ae740 con 0x7ff864066730 2024-05-08T23:33:25.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.538+0000 7ff87d8f4700 1 -- 172.21.15.81:0/1710107494 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff864066730 msgr2=0x7ff864068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.538+0000 7ff87d8f4700 1 --2- 172.21.15.81:0/1710107494 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff864066730 0x7ff864068bf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff8680018b0 tx=0x7ff86800a000 comp rx=0 tx=0).stop 2024-05-08T23:33:25.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.538+0000 7ff87d8f4700 1 -- 172.21.15.81:0/1710107494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff860004870 msgr2=0x7ff8780af8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.538+0000 7ff87d8f4700 1 --2- 172.21.15.81:0/1710107494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff860004870 0x7ff8780af8f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00e810 tx=0x7ff86c00eb20 comp rx=0 tx=0).stop 2024-05-08T23:33:25.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.538+0000 7ff87d8f4700 1 -- 172.21.15.81:0/1710107494 shutdown_connections 2024-05-08T23:33:25.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.538+0000 7ff87d8f4700 1 --2- 172.21.15.81:0/1710107494 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7ff864066730 0x7ff864068bf0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.538+0000 7ff87d8f4700 1 --2- 172.21.15.81:0/1710107494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff87814b750 0x7ff87814bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.538+0000 7ff87d8f4700 1 --2- 172.21.15.81:0/1710107494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff878149820 0x7ff878150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.538+0000 7ff87d8f4700 1 --2- 172.21.15.81:0/1710107494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff860004870 0x7ff8780af8f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.538+0000 7ff87d8f4700 1 -- 172.21.15.81:0/1710107494 >> 172.21.15.81:0/1710107494 conn(0x7ff87806c330 msgr2=0x7ff8780702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:25.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.538+0000 7ff87d8f4700 1 -- 172.21.15.81:0/1710107494 shutdown_connections 2024-05-08T23:33:25.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.538+0000 7ff87d8f4700 1 -- 172.21.15.81:0/1710107494 wait complete. 2024-05-08T23:33:25.541 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 0 to deep-scrub 2024-05-08T23:33:25.551 INFO:tasks.ceph:Scrubbing osd.1 2024-05-08T23:33:25.552 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:33:25.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.666+0000 7fb460978700 1 Processor -- start 2024-05-08T23:33:25.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.666+0000 7fb460978700 1 -- start start 2024-05-08T23:33:25.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb460978700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb45c0cb1c0 0x7fb45c1590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb460978700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45c15e200 con 0x7fb45c0b40d0 2024-05-08T23:33:25.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb460978700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45c15d460 con 0x7fb45c0b3cf0 2024-05-08T23:33:25.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb460978700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb45c15d740 con 0x7fb45c0cb1c0 2024-05-08T23:33:25.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb45affd700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb45c0cb1c0 0x7fb45c1590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb45bfff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb45c0b40d0 0x7fb45c159630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46222/0 (socket says 172.21.15.81:46222) 2024-05-08T23:33:25.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb45bfff700 1 -- 172.21.15.81:0/4215773297 learned_addr learned my addr 172.21.15.81:0/4215773297 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:25.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb45affd700 1 -- 172.21.15.81:0/4215773297 >> v1:172.21.15.117:6789/0 conn(0x7fb45c0b3cf0 legacy=0x7fb45c071c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb45affd700 1 -- 172.21.15.81:0/4215773297 >> v1:172.21.15.81:6789/0 conn(0x7fb45c0b40d0 legacy=0x7fb45c159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb45affd700 1 -- 172.21.15.81:0/4215773297 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c15da20 con 0x7fb45c0cb1c0 2024-05-08T23:33:25.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb45affd700 1 --2- 172.21.15.81:0/4215773297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb45c0cb1c0 0x7fb45c1590f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb444009850 tx=0x7fb444009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99ce23faf239635 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:25.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb45a7fc700 1 -- 172.21.15.81:0/4215773297 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444011830 con 0x7fb45c0cb1c0 2024-05-08T23:33:25.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb45a7fc700 1 -- 172.21.15.81:0/4215773297 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb444011990 con 0x7fb45c0cb1c0 2024-05-08T23:33:25.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb45a7fc700 1 -- 172.21.15.81:0/4215773297 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444011d20 con 0x7fb45c0cb1c0 2024-05-08T23:33:25.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb460978700 1 -- 172.21.15.81:0/4215773297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb45c0cb1c0 msgr2=0x7fb45c1590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb460978700 1 --2- 172.21.15.81:0/4215773297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb45c0cb1c0 0x7fb45c1590f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb444009850 tx=0x7fb444009b60 comp rx=0 tx=0).stop 2024-05-08T23:33:25.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb460978700 1 -- 172.21.15.81:0/4215773297 shutdown_connections 2024-05-08T23:33:25.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb460978700 1 --2- 172.21.15.81:0/4215773297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb45c0cb1c0 0x7fb45c1590f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb460978700 1 -- 172.21.15.81:0/4215773297 >> 172.21.15.81:0/4215773297 conn(0x7fb45c06d650 msgr2=0x7fb45c06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:25.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb460978700 1 -- 172.21.15.81:0/4215773297 shutdown_connections 2024-05-08T23:33:25.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.670+0000 7fb460978700 1 -- 172.21.15.81:0/4215773297 wait complete. 2024-05-08T23:33:25.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb460978700 1 Processor -- start 2024-05-08T23:33:25.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb460978700 1 -- start start 2024-05-08T23:33:25.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb460978700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb45c0b3cf0 0x7fb45c14c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb460978700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb45c0b40d0 0x7fb45c14da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb460978700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb45c0cb1c0 0x7fb45c13dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb460978700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c1576f0 con 0x7fb45c0b3cf0 2024-05-08T23:33:25.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb460978700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c14c5d0 con 0x7fb45c0b40d0 2024-05-08T23:33:25.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb460978700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c14e280 con 0x7fb45c0cb1c0 2024-05-08T23:33:25.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb45b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb45c0b3cf0 0x7fb45c14c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb45b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb45c0b3cf0 0x7fb45c14c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40264/0 (socket says 172.21.15.81:40264) 2024-05-08T23:33:25.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb45b7fe700 1 -- 172.21.15.81:0/244998985 learned_addr learned my addr 172.21.15.81:0/244998985 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:25.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb45affd700 1 --2- 172.21.15.81:0/244998985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb45c0b40d0 0x7fb45c14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb45b7fe700 1 -- 172.21.15.81:0/244998985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb45c0cb1c0 msgr2=0x7fb45c13dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb45bfff700 1 --2- 172.21.15.81:0/244998985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb45c0cb1c0 0x7fb45c13dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb45b7fe700 1 --2- 172.21.15.81:0/244998985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb45c0cb1c0 0x7fb45c13dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb45b7fe700 1 -- 172.21.15.81:0/244998985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb45c0b40d0 msgr2=0x7fb45c14da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb45b7fe700 1 --2- 172.21.15.81:0/244998985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb45c0b40d0 0x7fb45c14da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb45b7fe700 1 -- 172.21.15.81:0/244998985 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4440094b0 con 0x7fb45c0b3cf0 2024-05-08T23:33:25.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb45affd700 1 --2- 172.21.15.81:0/244998985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb45c0b40d0 0x7fb45c14da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:33:25.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb45b7fe700 1 --2- 172.21.15.81:0/244998985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb45c0b3cf0 0x7fb45c14c090 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00c500 tx=0x7fb44c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:25.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb453fff700 1 -- 172.21.15.81:0/244998985 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0052d0 con 0x7fb45c0b3cf0 2024-05-08T23:33:25.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb460978700 1 -- 172.21.15.81:0/244998985 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c156dc0 con 0x7fb45c0b3cf0 2024-05-08T23:33:25.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb453fff700 1 -- 172.21.15.81:0/244998985 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44c005430 con 0x7fb45c0b3cf0 2024-05-08T23:33:25.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb460978700 1 -- 172.21.15.81:0/244998985 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c157380 con 0x7fb45c0b3cf0 2024-05-08T23:33:25.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.674+0000 7fb453fff700 1 -- 172.21.15.81:0/244998985 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0055a0 con 0x7fb45c0b3cf0 2024-05-08T23:33:25.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.678+0000 7fb460978700 1 -- 172.21.15.81:0/244998985 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb45c15ad50 con 0x7fb45c0b3cf0 2024-05-08T23:33:25.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.678+0000 7fb453fff700 1 -- 172.21.15.81:0/244998985 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb44c006a80 con 0x7fb45c0b3cf0 2024-05-08T23:33:25.681 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.678+0000 7fb453fff700 1 --2- 172.21.15.81:0/244998985 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb440066730 0x7fb440068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.678+0000 7fb453fff700 1 -- 172.21.15.81:0/244998985 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb44c0831b0 con 0x7fb45c0b3cf0 2024-05-08T23:33:25.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.678+0000 7fb45affd700 1 --2- 172.21.15.81:0/244998985 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb440066730 0x7fb440068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.678+0000 7fb453fff700 1 --2- 172.21.15.81:0/244998985 >> [v2:172.21.15.81:6816/26093,v1:172.21.15.81:6818/26093] conn(0x7fb44006c590 0x7fb44006e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.678+0000 7fb453fff700 1 -- 172.21.15.81:0/244998985 --> [v2:172.21.15.81:6816/26093,v1:172.21.15.81:6818/26093] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb44006f060 con 0x7fb44006c590 2024-05-08T23:33:25.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.678+0000 7fb453fff700 1 -- 172.21.15.81:0/244998985 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb44c085870 con 0x7fb45c0b3cf0 2024-05-08T23:33:25.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.678+0000 7fb45bfff700 1 --2- 172.21.15.81:0/244998985 >> [v2:172.21.15.81:6816/26093,v1:172.21.15.81:6818/26093] conn(0x7fb44006c590 0x7fb44006e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.678+0000 7fb45affd700 1 --2- 172.21.15.81:0/244998985 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb440066730 0x7fb440068bf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb4440090b0 tx=0x7fb444008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:25.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.678+0000 7fb45bfff700 1 --2- 172.21.15.81:0/244998985 >> [v2:172.21.15.81:6816/26093,v1:172.21.15.81:6818/26093] conn(0x7fb44006c590 0x7fb44006e9b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:25.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.682+0000 7fb453fff700 1 -- 172.21.15.81:0/244998985 <== osd.1 v2:172.21.15.81:6816/26093 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb44006f060 con 0x7fb44006c590 2024-05-08T23:33:25.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.698+0000 7fb460978700 1 -- 172.21.15.81:0/244998985 --> [v2:172.21.15.81:6816/26093,v1:172.21.15.81:6818/26093] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb45c157830 con 0x7fb44006c590 2024-05-08T23:33:25.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.698+0000 7fb453fff700 1 -- 172.21.15.81:0/244998985 <== osd.1 v2:172.21.15.81:6816/26093 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb454001cc0 con 0x7fb44006c590 2024-05-08T23:33:25.702 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:33:25.702 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:33:25.702 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:33:25.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.698+0000 7fb460978700 1 -- 172.21.15.81:0/244998985 >> [v2:172.21.15.81:6816/26093,v1:172.21.15.81:6818/26093] conn(0x7fb44006c590 msgr2=0x7fb44006e9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.698+0000 7fb460978700 1 --2- 172.21.15.81:0/244998985 >> [v2:172.21.15.81:6816/26093,v1:172.21.15.81:6818/26093] conn(0x7fb44006c590 0x7fb44006e9b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.698+0000 7fb460978700 1 -- 172.21.15.81:0/244998985 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb440066730 msgr2=0x7fb440068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.698+0000 7fb460978700 1 --2- 172.21.15.81:0/244998985 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fb440066730 0x7fb440068bf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb4440090b0 tx=0x7fb444008030 comp rx=0 tx=0).stop 2024-05-08T23:33:25.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.698+0000 7fb460978700 1 -- 172.21.15.81:0/244998985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb45c0b3cf0 msgr2=0x7fb45c14c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.698+0000 7fb460978700 1 --2- 172.21.15.81:0/244998985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb45c0b3cf0 0x7fb45c14c090 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00c500 tx=0x7fb44c00c8c0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.698+0000 7fb45bfff700 1 -- 172.21.15.81:0/244998985 reap_dead start 2024-05-08T23:33:25.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.698+0000 7fb460978700 1 -- 172.21.15.81:0/244998985 shutdown_connections 2024-05-08T23:33:25.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.698+0000 7fb460978700 1 -- 172.21.15.81:0/244998985 >> 172.21.15.81:0/244998985 conn(0x7fb45c06d650 msgr2=0x7fb45c0cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:25.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.698+0000 7fb460978700 1 -- 172.21.15.81:0/244998985 shutdown_connections 2024-05-08T23:33:25.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.698+0000 7fb460978700 1 -- 172.21.15.81:0/244998985 wait complete. 2024-05-08T23:33:25.714 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-08T23:33:25.827 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.822+0000 7fd111433700 1 Processor -- start 2024-05-08T23:33:25.827 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.822+0000 7fd111433700 1 -- start start 2024-05-08T23:33:25.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd111433700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd10c071ab0 0x7fd10c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd111433700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd10c071e90 0x7fd10c15a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd111433700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10c0cea50 con 0x7fd10c15aaa0 2024-05-08T23:33:25.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd111433700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd10c0ce080 con 0x7fd10c071ab0 2024-05-08T23:33:25.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd111433700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd10c0cd7c0 con 0x7fd10c071e90 2024-05-08T23:33:25.828 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd10bfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd10c071ab0 0x7fd10c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd10bfff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd10c071ab0 0x7fd10c0ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40274/0 (socket says 172.21.15.81:40274) 2024-05-08T23:33:25.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd10bfff700 1 -- 172.21.15.81:0/2813476258 learned_addr learned my addr 172.21.15.81:0/2813476258 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:25.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd10b7fe700 1 --2- 172.21.15.81:0/2813476258 >> v2:172.21.15.117:3300/0 conn(0x7fd10c071e90 0x7fd10c15a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd10bfff700 1 -- 172.21.15.81:0/2813476258 >> v2:172.21.15.117:3300/0 conn(0x7fd10c071e90 msgr2=0x7fd10c15a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd10bfff700 1 --2- 172.21.15.81:0/2813476258 >> v2:172.21.15.117:3300/0 conn(0x7fd10c071e90 0x7fd10c15a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd10bfff700 1 -- 172.21.15.81:0/2813476258 >> v1:172.21.15.117:6789/0 conn(0x7fd10c15aaa0 legacy=0x7fd10c13d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd10bfff700 1 -- 172.21.15.81:0/2813476258 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c0cda30 con 0x7fd10c071ab0 2024-05-08T23:33:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd10bfff700 1 --2- 172.21.15.81:0/2813476258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10c071ab0 0x7fd10c0ce510 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd10c06bb20 tx=0x7fd0f4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1513b1862b893a01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd10affd700 1 -- 172.21.15.81:0/2813476258 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4011830 con 0x7fd10c071ab0 2024-05-08T23:33:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd10affd700 1 -- 172.21.15.81:0/2813476258 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f4011990 con 0x7fd10c071ab0 2024-05-08T23:33:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd10affd700 1 -- 172.21.15.81:0/2813476258 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4011d20 con 0x7fd10c071ab0 2024-05-08T23:33:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd111433700 1 -- 172.21.15.81:0/2813476258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10c071ab0 msgr2=0x7fd10c0ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd111433700 1 --2- 172.21.15.81:0/2813476258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10c071ab0 0x7fd10c0ce510 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd10c06bb20 tx=0x7fd0f4009bf0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd111433700 1 -- 172.21.15.81:0/2813476258 shutdown_connections 2024-05-08T23:33:25.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd111433700 1 --2- 172.21.15.81:0/2813476258 >> v2:172.21.15.117:3300/0 conn(0x7fd10c071e90 0x7fd10c15a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd111433700 1 --2- 172.21.15.81:0/2813476258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10c071ab0 0x7fd10c0ce510 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd111433700 1 -- 172.21.15.81:0/2813476258 >> 172.21.15.81:0/2813476258 conn(0x7fd10c06c330 msgr2=0x7fd10c06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:25.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd111433700 1 -- 172.21.15.81:0/2813476258 shutdown_connections 2024-05-08T23:33:25.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.826+0000 7fd111433700 1 -- 172.21.15.81:0/2813476258 wait complete. 2024-05-08T23:33:25.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd111433700 1 Processor -- start 2024-05-08T23:33:25.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd111433700 1 -- start start 2024-05-08T23:33:25.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd111433700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c071ab0 0x7fd10c1524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd111433700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd10c071e90 0x7fd10c1405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd111433700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10c15aaa0 0x7fd10c13c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd111433700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c140ff0 con 0x7fd10c15aaa0 2024-05-08T23:33:25.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd111433700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c1412d0 con 0x7fd10c071e90 2024-05-08T23:33:25.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd111433700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c1415b0 con 0x7fd10c071ab0 2024-05-08T23:33:25.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd110c32700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10c15aaa0 0x7fd10c13c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd110c32700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10c15aaa0 0x7fd10c13c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40284/0 (socket says 172.21.15.81:40284) 2024-05-08T23:33:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd110c32700 1 -- 172.21.15.81:0/2319468312 learned_addr learned my addr 172.21.15.81:0/2319468312 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd10bfff700 1 --2- 172.21.15.81:0/2319468312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c071ab0 0x7fd10c1524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd10b7fe700 1 --2- 172.21.15.81:0/2319468312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd10c071e90 0x7fd10c1405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd110c32700 1 -- 172.21.15.81:0/2319468312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c071ab0 msgr2=0x7fd10c1524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd110c32700 1 --2- 172.21.15.81:0/2319468312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c071ab0 0x7fd10c1524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd110c32700 1 -- 172.21.15.81:0/2319468312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd10c071e90 msgr2=0x7fd10c1405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd110c32700 1 --2- 172.21.15.81:0/2319468312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd10c071e90 0x7fd10c1405a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd110c32700 1 -- 172.21.15.81:0/2319468312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f4009570 con 0x7fd10c15aaa0 2024-05-08T23:33:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd10bfff700 1 --2- 172.21.15.81:0/2319468312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c071ab0 0x7fd10c1524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd110c32700 1 --2- 172.21.15.81:0/2319468312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10c15aaa0 0x7fd10c13c340 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd10400cc50 tx=0x7fd10400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd108ff9700 1 -- 172.21.15.81:0/2319468312 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10400ad80 con 0x7fd10c15aaa0 2024-05-08T23:33:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd108ff9700 1 -- 172.21.15.81:0/2319468312 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10400ce50 con 0x7fd10c15aaa0 2024-05-08T23:33:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd111433700 1 -- 172.21.15.81:0/2319468312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10c141890 con 0x7fd10c15aaa0 2024-05-08T23:33:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd108ff9700 1 -- 172.21.15.81:0/2319468312 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104003c60 con 0x7fd10c15aaa0 2024-05-08T23:33:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.830+0000 7fd111433700 1 -- 172.21.15.81:0/2319468312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10c142090 con 0x7fd10c15aaa0 2024-05-08T23:33:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.834+0000 7fd111433700 1 -- 172.21.15.81:0/2319468312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10c141db0 con 0x7fd10c15aaa0 2024-05-08T23:33:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.834+0000 7fd108ff9700 1 -- 172.21.15.81:0/2319468312 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd1040050d0 con 0x7fd10c15aaa0 2024-05-08T23:33:25.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.834+0000 7fd108ff9700 1 --2- 172.21.15.81:0/2319468312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd0ec066730 0x7fd0ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:25.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.834+0000 7fd108ff9700 1 -- 172.21.15.81:0/2319468312 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd104083f10 con 0x7fd10c15aaa0 2024-05-08T23:33:25.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.834+0000 7fd10bfff700 1 --2- 172.21.15.81:0/2319468312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd0ec066730 0x7fd0ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:25.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.834+0000 7fd10bfff700 1 --2- 172.21.15.81:0/2319468312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd0ec066730 0x7fd0ec068bf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4000c00 tx=0x7fd0f4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:25.842 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:25.838+0000 7fd108ff9700 1 -- 172.21.15.81:0/2319468312 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd10c141db0 con 0x7fd10c15aaa0 2024-05-08T23:33:26.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.010+0000 7fd111433700 1 -- 172.21.15.81:0/2319468312 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd10c15af30 con 0x7fd0ec066730 2024-05-08T23:33:26.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.010+0000 7fd108ff9700 1 -- 172.21.15.81:0/2319468312 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd0f4011c00 con 0x7fd0ec066730 2024-05-08T23:33:26.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.014+0000 7fd111433700 1 -- 172.21.15.81:0/2319468312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd0ec066730 msgr2=0x7fd0ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.014+0000 7fd111433700 1 --2- 172.21.15.81:0/2319468312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd0ec066730 0x7fd0ec068bf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4000c00 tx=0x7fd0f4008030 comp rx=0 tx=0).stop 2024-05-08T23:33:26.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.014+0000 7fd111433700 1 -- 172.21.15.81:0/2319468312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10c15aaa0 msgr2=0x7fd10c13c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.014+0000 7fd111433700 1 --2- 172.21.15.81:0/2319468312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10c15aaa0 0x7fd10c13c340 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd10400cc50 tx=0x7fd10400a340 comp rx=0 tx=0).stop 2024-05-08T23:33:26.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.014+0000 7fd111433700 1 -- 172.21.15.81:0/2319468312 shutdown_connections 2024-05-08T23:33:26.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.014+0000 7fd111433700 1 --2- 172.21.15.81:0/2319468312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd0ec066730 0x7fd0ec068bf0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.014+0000 7fd111433700 1 --2- 172.21.15.81:0/2319468312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10c15aaa0 0x7fd10c13c340 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.014+0000 7fd111433700 1 --2- 172.21.15.81:0/2319468312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd10c071e90 0x7fd10c1405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.014+0000 7fd111433700 1 --2- 172.21.15.81:0/2319468312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c071ab0 0x7fd10c1524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.014+0000 7fd111433700 1 -- 172.21.15.81:0/2319468312 >> 172.21.15.81:0/2319468312 conn(0x7fd10c06c330 msgr2=0x7fd10c13fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:26.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.014+0000 7fd111433700 1 -- 172.21.15.81:0/2319468312 shutdown_connections 2024-05-08T23:33:26.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.014+0000 7fd111433700 1 -- 172.21.15.81:0/2319468312 wait complete. 2024-05-08T23:33:26.018 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 1 to deep-scrub 2024-05-08T23:33:26.028 INFO:tasks.ceph:Scrubbing osd.2 2024-05-08T23:33:26.028 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:33:26.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe09d49c700 1 Processor -- start 2024-05-08T23:33:26.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe09d49c700 1 -- start start 2024-05-08T23:33:26.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe09d49c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09815d300 0x7fe0980d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe09d49c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe0980d1c00 0x7fe0980b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe09d49c700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0980b3030 con 0x7fe0980d02b0 2024-05-08T23:33:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe09d49c700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0980d0710 con 0x7fe0980d1c00 2024-05-08T23:33:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe09d49c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980d1210 con 0x7fe09815d300 2024-05-08T23:33:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe09cc9b700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe0980d1c00 0x7fe0980b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe0977fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe0980d02b0 0x7fe0980d14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49030/0 (socket says 172.21.15.81:49030) 2024-05-08T23:33:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe0977fe700 1 -- 172.21.15.81:0/3681513041 learned_addr learned my addr 172.21.15.81:0/3681513041 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe097fff700 1 --2- 172.21.15.81:0/3681513041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09815d300 0x7fe0980d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe09cc9b700 1 -- 172.21.15.81:0/3681513041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09815d300 msgr2=0x7fe0980d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe09cc9b700 1 --2- 172.21.15.81:0/3681513041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09815d300 0x7fe0980d0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe096ffd700 1 -- 172.21.15.81:0/3681513041 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203682975 0 0) 0x7fe0980b3030 con 0x7fe0980d02b0 2024-05-08T23:33:26.151 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe09cc9b700 1 -- 172.21.15.81:0/3681513041 >> v1:172.21.15.81:6789/0 conn(0x7fe0980d02b0 legacy=0x7fe0980d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe09cc9b700 1 -- 172.21.15.81:0/3681513041 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0980b29e0 con 0x7fe0980d1c00 2024-05-08T23:33:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe097fff700 1 --2- 172.21.15.81:0/3681513041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09815d300 0x7fe0980d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe09cc9b700 1 --2- 172.21.15.81:0/3681513041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0980d1c00 0x7fe0980b34c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe090009dc0 tx=0x7fe09000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7bf204c6ad48dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe096ffd700 1 -- 172.21.15.81:0/3681513041 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09000e070 con 0x7fe0980d1c00 2024-05-08T23:33:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe096ffd700 1 -- 172.21.15.81:0/3681513041 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09000bb90 con 0x7fe0980d1c00 2024-05-08T23:33:26.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.146+0000 7fe096ffd700 1 -- 172.21.15.81:0/3681513041 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe090012430 con 0x7fe0980d1c00 2024-05-08T23:33:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 -- 172.21.15.81:0/3681513041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0980d1c00 msgr2=0x7fe0980b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 --2- 172.21.15.81:0/3681513041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0980d1c00 0x7fe0980b34c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe090009dc0 tx=0x7fe09000b330 comp rx=0 tx=0).stop 2024-05-08T23:33:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 -- 172.21.15.81:0/3681513041 shutdown_connections 2024-05-08T23:33:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 --2- 172.21.15.81:0/3681513041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0980d1c00 0x7fe0980b34c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 --2- 172.21.15.81:0/3681513041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09815d300 0x7fe0980d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 -- 172.21.15.81:0/3681513041 >> 172.21.15.81:0/3681513041 conn(0x7fe09809c3b0 msgr2=0x7fe0980d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 -- 172.21.15.81:0/3681513041 shutdown_connections 2024-05-08T23:33:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 -- 172.21.15.81:0/3681513041 wait complete. 2024-05-08T23:33:26.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 Processor -- start 2024-05-08T23:33:26.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 -- start start 2024-05-08T23:33:26.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0980d02b0 0x7fe0980c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0980d1c00 0x7fe0980c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe09815d300 0x7fe0981f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0981f4190 con 0x7fe0980d1c00 2024-05-08T23:33:26.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980b75d0 con 0x7fe09815d300 2024-05-08T23:33:26.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe09d49c700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980c9110 con 0x7fe0980d02b0 2024-05-08T23:33:26.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.150+0000 7fe0977fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0980d1c00 0x7fe0980c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe0977fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0980d1c00 0x7fe0980c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34348/0 (socket says 172.21.15.81:34348) 2024-05-08T23:33:26.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe097fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0980d02b0 0x7fe0980c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe0977fe700 1 -- 172.21.15.81:0/522280061 learned_addr learned my addr 172.21.15.81:0/522280061 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:26.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe09cc9b700 1 --2- 172.21.15.81:0/522280061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe09815d300 0x7fe0981f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe0977fe700 1 -- 172.21.15.81:0/522280061 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0980d02b0 msgr2=0x7fe0980c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe0977fe700 1 --2- 172.21.15.81:0/522280061 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0980d02b0 0x7fe0980c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe0977fe700 1 -- 172.21.15.81:0/522280061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe09815d300 msgr2=0x7fe0981f3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe0977fe700 1 --2- 172.21.15.81:0/522280061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe09815d300 0x7fe0981f3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe0977fe700 1 -- 172.21.15.81:0/522280061 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe090009a70 con 0x7fe0980d1c00 2024-05-08T23:33:26.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe0977fe700 1 --2- 172.21.15.81:0/522280061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0980d1c00 0x7fe0980c6ad0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00c580 tx=0x7fe08c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:26.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe094ff9700 1 -- 172.21.15.81:0/522280061 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c004ca0 con 0x7fe0980d1c00 2024-05-08T23:33:26.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe094ff9700 1 -- 172.21.15.81:0/522280061 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08c004e20 con 0x7fe0980d1c00 2024-05-08T23:33:26.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe09d49c700 1 -- 172.21.15.81:0/522280061 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0980c93f0 con 0x7fe0980d1c00 2024-05-08T23:33:26.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe094ff9700 1 -- 172.21.15.81:0/522280061 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c007310 con 0x7fe0980d1c00 2024-05-08T23:33:26.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe09d49c700 1 -- 172.21.15.81:0/522280061 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0980b7b30 con 0x7fe0980d1c00 2024-05-08T23:33:26.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe09d49c700 1 -- 172.21.15.81:0/522280061 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe0980b3830 con 0x7fe0980d1c00 2024-05-08T23:33:26.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.154+0000 7fe094ff9700 1 -- 172.21.15.81:0/522280061 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe08c007470 con 0x7fe0980d1c00 2024-05-08T23:33:26.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.158+0000 7fe094ff9700 1 --2- 172.21.15.81:0/522280061 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe084066730 0x7fe084068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.158+0000 7fe094ff9700 1 -- 172.21.15.81:0/522280061 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe08c083ad0 con 0x7fe0980d1c00 2024-05-08T23:33:26.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.158+0000 7fe097fff700 1 --2- 172.21.15.81:0/522280061 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe084066730 0x7fe084068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.158+0000 7fe094ff9700 1 --2- 172.21.15.81:0/522280061 >> [v2:172.21.15.81:6817/26092,v1:172.21.15.81:6819/26092] conn(0x7fe08406c590 0x7fe08406e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.158+0000 7fe094ff9700 1 -- 172.21.15.81:0/522280061 --> [v2:172.21.15.81:6817/26092,v1:172.21.15.81:6819/26092] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe08406f060 con 0x7fe08406c590 2024-05-08T23:33:26.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.158+0000 7fe094ff9700 1 -- 172.21.15.81:0/522280061 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe08c0882c0 con 0x7fe0980d1c00 2024-05-08T23:33:26.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.158+0000 7fe09cc9b700 1 --2- 172.21.15.81:0/522280061 >> [v2:172.21.15.81:6817/26092,v1:172.21.15.81:6819/26092] conn(0x7fe08406c590 0x7fe08406e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.158+0000 7fe097fff700 1 --2- 172.21.15.81:0/522280061 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe084066730 0x7fe084068bf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe098067360 tx=0x7fe088008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:26.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.158+0000 7fe09cc9b700 1 --2- 172.21.15.81:0/522280061 >> [v2:172.21.15.81:6817/26092,v1:172.21.15.81:6819/26092] conn(0x7fe08406c590 0x7fe08406e9b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:26.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.158+0000 7fe094ff9700 1 -- 172.21.15.81:0/522280061 <== osd.2 v2:172.21.15.81:6817/26092 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe08406f060 con 0x7fe08406c590 2024-05-08T23:33:26.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.174+0000 7fe09d49c700 1 -- 172.21.15.81:0/522280061 --> [v2:172.21.15.81:6817/26092,v1:172.21.15.81:6819/26092] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe0980674e0 con 0x7fe08406c590 2024-05-08T23:33:26.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.174+0000 7fe094ff9700 1 -- 172.21.15.81:0/522280061 <== osd.2 v2:172.21.15.81:6817/26092 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe0980674e0 con 0x7fe08406c590 2024-05-08T23:33:26.178 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:33:26.178 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:33:26.178 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:33:26.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.174+0000 7fe09d49c700 1 -- 172.21.15.81:0/522280061 >> [v2:172.21.15.81:6817/26092,v1:172.21.15.81:6819/26092] conn(0x7fe08406c590 msgr2=0x7fe08406e9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.174+0000 7fe09d49c700 1 --2- 172.21.15.81:0/522280061 >> [v2:172.21.15.81:6817/26092,v1:172.21.15.81:6819/26092] conn(0x7fe08406c590 0x7fe08406e9b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.174+0000 7fe09d49c700 1 -- 172.21.15.81:0/522280061 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe084066730 msgr2=0x7fe084068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.174+0000 7fe09d49c700 1 --2- 172.21.15.81:0/522280061 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fe084066730 0x7fe084068bf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe098067360 tx=0x7fe088008040 comp rx=0 tx=0).stop 2024-05-08T23:33:26.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.174+0000 7fe09d49c700 1 -- 172.21.15.81:0/522280061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0980d1c00 msgr2=0x7fe0980c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.174+0000 7fe09d49c700 1 --2- 172.21.15.81:0/522280061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0980d1c00 0x7fe0980c6ad0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00c580 tx=0x7fe08c00c890 comp rx=0 tx=0).stop 2024-05-08T23:33:26.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.174+0000 7fe09cc9b700 1 -- 172.21.15.81:0/522280061 reap_dead start 2024-05-08T23:33:26.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.174+0000 7fe09d49c700 1 -- 172.21.15.81:0/522280061 shutdown_connections 2024-05-08T23:33:26.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.174+0000 7fe09d49c700 1 -- 172.21.15.81:0/522280061 >> 172.21.15.81:0/522280061 conn(0x7fe09809c3b0 msgr2=0x7fe0980b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:26.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.174+0000 7fe09d49c700 1 -- 172.21.15.81:0/522280061 shutdown_connections 2024-05-08T23:33:26.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.174+0000 7fe09d49c700 1 -- 172.21.15.81:0/522280061 wait complete. 2024-05-08T23:33:26.191 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-08T23:33:26.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.302+0000 7effc5098700 1 Processor -- start 2024-05-08T23:33:26.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.302+0000 7effc5098700 1 -- start start 2024-05-08T23:33:26.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc5098700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc015d500 0x7effc00b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc5098700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7effc0153f00 0x7effc006a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc5098700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc006b630 con 0x7effc006bac0 2024-05-08T23:33:26.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc5098700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc006a8a0 con 0x7effc0153f00 2024-05-08T23:33:26.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc5098700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc006aaa0 con 0x7effc015d500 2024-05-08T23:33:26.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effb7fff700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7effc006bac0 0x7effc01537f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49044/0 (socket says 172.21.15.81:49044) 2024-05-08T23:33:26.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effb7fff700 1 -- 172.21.15.81:0/3155519686 learned_addr learned my addr 172.21.15.81:0/3155519686 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:26.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc4897700 1 --2- 172.21.15.81:0/3155519686 >> v2:172.21.15.117:3300/0 conn(0x7effc0153f00 0x7effc006a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effbffff700 1 --2- 172.21.15.81:0/3155519686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc015d500 0x7effc00b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effbf7fe700 1 -- 172.21.15.81:0/3155519686 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369727505 0 0) 0x7effc006b630 con 0x7effc006bac0 2024-05-08T23:33:26.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effbf7fe700 1 -- 172.21.15.81:0/3155519686 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effac0035d0 con 0x7effc006bac0 2024-05-08T23:33:26.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc4897700 1 -- 172.21.15.81:0/3155519686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc015d500 msgr2=0x7effc00b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:26.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc4897700 1 --2- 172.21.15.81:0/3155519686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc015d500 0x7effc00b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc4897700 1 -- 172.21.15.81:0/3155519686 >> v1:172.21.15.81:6789/0 conn(0x7effc006bac0 legacy=0x7effc01537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc4897700 1 -- 172.21.15.81:0/3155519686 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc006ad40 con 0x7effc0153f00 2024-05-08T23:33:26.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc4897700 1 --2- 172.21.15.81:0/3155519686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc0153f00 0x7effc006a360 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7effb8009d90 tx=0x7effb800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd499e7d23657c41 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:26.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effbf7fe700 1 -- 172.21.15.81:0/3155519686 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb800e070 con 0x7effc0153f00 2024-05-08T23:33:26.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effbf7fe700 1 -- 172.21.15.81:0/3155519686 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb800bb20 con 0x7effc0153f00 2024-05-08T23:33:26.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effbf7fe700 1 -- 172.21.15.81:0/3155519686 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8012430 con 0x7effc0153f00 2024-05-08T23:33:26.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc5098700 1 -- 172.21.15.81:0/3155519686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc0153f00 msgr2=0x7effc006a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc5098700 1 --2- 172.21.15.81:0/3155519686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc0153f00 0x7effc006a360 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7effb8009d90 tx=0x7effb800b330 comp rx=0 tx=0).stop 2024-05-08T23:33:26.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc5098700 1 -- 172.21.15.81:0/3155519686 shutdown_connections 2024-05-08T23:33:26.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc5098700 1 --2- 172.21.15.81:0/3155519686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc0153f00 0x7effc006a360 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc5098700 1 --2- 172.21.15.81:0/3155519686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc015d500 0x7effc00b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc5098700 1 -- 172.21.15.81:0/3155519686 >> 172.21.15.81:0/3155519686 conn(0x7effc00cf570 msgr2=0x7effc00cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:26.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc5098700 1 -- 172.21.15.81:0/3155519686 shutdown_connections 2024-05-08T23:33:26.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.306+0000 7effc5098700 1 -- 172.21.15.81:0/3155519686 wait complete. 2024-05-08T23:33:26.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effc5098700 1 Processor -- start 2024-05-08T23:33:26.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effc5098700 1 -- start start 2024-05-08T23:33:26.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effc5098700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc006bac0 0x7effc00cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effc5098700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effc0153f00 0x7effc014e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effc5098700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc015d500 0x7effc015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effc5098700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc0132920 con 0x7effc0153f00 2024-05-08T23:33:26.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effc5098700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc0132c00 con 0x7effc006bac0 2024-05-08T23:33:26.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effc5098700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc0132ee0 con 0x7effc015d500 2024-05-08T23:33:26.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effb7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effc0153f00 0x7effc014e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effb7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effc0153f00 0x7effc014e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34350/0 (socket says 172.21.15.81:34350) 2024-05-08T23:33:26.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effb7fff700 1 -- 172.21.15.81:0/3267588436 learned_addr learned my addr 172.21.15.81:0/3267588436 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effc4897700 1 --2- 172.21.15.81:0/3267588436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc015d500 0x7effc015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effbffff700 1 --2- 172.21.15.81:0/3267588436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc006bac0 0x7effc00cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effb7fff700 1 -- 172.21.15.81:0/3267588436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc015d500 msgr2=0x7effc015ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effb7fff700 1 --2- 172.21.15.81:0/3267588436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc015d500 0x7effc015ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effb7fff700 1 -- 172.21.15.81:0/3267588436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc006bac0 msgr2=0x7effc00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effb7fff700 1 --2- 172.21.15.81:0/3267588436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc006bac0 0x7effc00cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effb7fff700 1 -- 172.21.15.81:0/3267588436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb8009a70 con 0x7effc0153f00 2024-05-08T23:33:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effbffff700 1 --2- 172.21.15.81:0/3267588436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc006bac0 0x7effc00cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effb7fff700 1 --2- 172.21.15.81:0/3267588436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effc0153f00 0x7effc014e2c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7effa800c4d0 tx=0x7effa800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effbd7fa700 1 -- 172.21.15.81:0/3267588436 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8005d90 con 0x7effc0153f00 2024-05-08T23:33:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effbd7fa700 1 -- 172.21.15.81:0/3267588436 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa80070d0 con 0x7effc0153f00 2024-05-08T23:33:26.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effc5098700 1 -- 172.21.15.81:0/3267588436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc01331c0 con 0x7effc0153f00 2024-05-08T23:33:26.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effbd7fa700 1 -- 172.21.15.81:0/3267588436 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8007480 con 0x7effc0153f00 2024-05-08T23:33:26.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.310+0000 7effc5098700 1 -- 172.21.15.81:0/3267588436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc01471c0 con 0x7effc0153f00 2024-05-08T23:33:26.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.314+0000 7effc5098700 1 -- 172.21.15.81:0/3267588436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc0073d40 con 0x7effc0153f00 2024-05-08T23:33:26.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.314+0000 7effbd7fa700 1 -- 172.21.15.81:0/3267588436 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7effa80075e0 con 0x7effc0153f00 2024-05-08T23:33:26.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.314+0000 7effbd7fa700 1 --2- 172.21.15.81:0/3267588436 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7effa0066730 0x7effa0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.314+0000 7effbffff700 1 --2- 172.21.15.81:0/3267588436 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7effa0066730 0x7effa0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.314+0000 7effbd7fa700 1 -- 172.21.15.81:0/3267588436 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7effa80833e0 con 0x7effc0153f00 2024-05-08T23:33:26.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.314+0000 7effbffff700 1 --2- 172.21.15.81:0/3267588436 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7effa0066730 0x7effa0068bf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7effb0009ba0 tx=0x7effb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:26.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.322+0000 7effbd7fa700 1 -- 172.21.15.81:0/3267588436 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7effa8050a50 con 0x7effc0153f00 2024-05-08T23:33:26.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.494+0000 7effc5098700 1 -- 172.21.15.81:0/3267588436 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7effc0132d70 con 0x7effa0066730 2024-05-08T23:33:26.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.494+0000 7effbd7fa700 1 -- 172.21.15.81:0/3267588436 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7effc0132d70 con 0x7effa0066730 2024-05-08T23:33:26.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.494+0000 7effc5098700 1 -- 172.21.15.81:0/3267588436 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7effa0066730 msgr2=0x7effa0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.494+0000 7effc5098700 1 --2- 172.21.15.81:0/3267588436 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7effa0066730 0x7effa0068bf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7effb0009ba0 tx=0x7effb0008040 comp rx=0 tx=0).stop 2024-05-08T23:33:26.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.494+0000 7effc5098700 1 -- 172.21.15.81:0/3267588436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effc0153f00 msgr2=0x7effc014e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.494+0000 7effc5098700 1 --2- 172.21.15.81:0/3267588436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effc0153f00 0x7effc014e2c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7effa800c4d0 tx=0x7effa800c890 comp rx=0 tx=0).stop 2024-05-08T23:33:26.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.494+0000 7effc5098700 1 -- 172.21.15.81:0/3267588436 shutdown_connections 2024-05-08T23:33:26.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.494+0000 7effc5098700 1 --2- 172.21.15.81:0/3267588436 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7effa0066730 0x7effa0068bf0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.494+0000 7effc5098700 1 --2- 172.21.15.81:0/3267588436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc015d500 0x7effc015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.494+0000 7effc5098700 1 --2- 172.21.15.81:0/3267588436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effc0153f00 0x7effc014e2c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.494+0000 7effc5098700 1 --2- 172.21.15.81:0/3267588436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc006bac0 0x7effc00cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.494+0000 7effc5098700 1 -- 172.21.15.81:0/3267588436 >> 172.21.15.81:0/3267588436 conn(0x7effc00cf570 msgr2=0x7effc00ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:26.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.494+0000 7effc5098700 1 -- 172.21.15.81:0/3267588436 shutdown_connections 2024-05-08T23:33:26.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.494+0000 7effc5098700 1 -- 172.21.15.81:0/3267588436 wait complete. 2024-05-08T23:33:26.500 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 2 to deep-scrub 2024-05-08T23:33:26.511 INFO:tasks.ceph:Scrubbing osd.3 2024-05-08T23:33:26.511 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:33:26.631 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.626+0000 7fefe6326700 1 Processor -- start 2024-05-08T23:33:26.631 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.626+0000 7fefe6326700 1 -- start start 2024-05-08T23:33:26.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefe6326700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe015d510 0x7fefe00ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefe6326700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fefe00b5320 0x7fefe00b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefe6326700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe00ce710 con 0x7fefe00cdc90 2024-05-08T23:33:26.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefe6326700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe00ce9c0 con 0x7fefe00b5320 2024-05-08T23:33:26.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefe6326700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe00d2f50 con 0x7fefe015d510 2024-05-08T23:33:26.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefdf7fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fefe00cdc90 0x7fefe00b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49058/0 (socket says 172.21.15.81:49058) 2024-05-08T23:33:26.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefdf7fe700 1 -- 172.21.15.81:0/3733384988 learned_addr learned my addr 172.21.15.81:0/3733384988 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:26.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefdffff700 1 --2- 172.21.15.81:0/3733384988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe015d510 0x7fefe00ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefe48c3700 1 --2- 172.21.15.81:0/3733384988 >> v2:172.21.15.117:3300/0 conn(0x7fefe00b5320 0x7fefe00b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefdeffd700 1 -- 172.21.15.81:0/3733384988 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527733455 0 0) 0x7fefe00ce710 con 0x7fefe00cdc90 2024-05-08T23:33:26.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefdeffd700 1 -- 172.21.15.81:0/3733384988 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefc80035d0 con 0x7fefe00cdc90 2024-05-08T23:33:26.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefdffff700 1 -- 172.21.15.81:0/3733384988 >> v2:172.21.15.117:3300/0 conn(0x7fefe00b5320 msgr2=0x7fefe00b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:26.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefdffff700 1 --2- 172.21.15.81:0/3733384988 >> v2:172.21.15.117:3300/0 conn(0x7fefe00b5320 0x7fefe00b63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefdffff700 1 -- 172.21.15.81:0/3733384988 >> v1:172.21.15.81:6789/0 conn(0x7fefe00cdc90 legacy=0x7fefe00b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.635 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefdffff700 1 -- 172.21.15.81:0/3733384988 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe00d2190 con 0x7fefe015d510 2024-05-08T23:33:26.635 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefe48c3700 1 --2- 172.21.15.81:0/3733384988 >> v2:172.21.15.117:3300/0 conn(0x7fefe00b5320 0x7fefe00b63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:26.635 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefdffff700 1 --2- 172.21.15.81:0/3733384988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe015d510 0x7fefe00ce1d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009850 tx=0x7fefd4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6a489f8fe2d1583 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:26.636 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefdeffd700 1 -- 172.21.15.81:0/3733384988 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4011830 con 0x7fefe015d510 2024-05-08T23:33:26.636 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefdeffd700 1 -- 172.21.15.81:0/3733384988 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd4011990 con 0x7fefe015d510 2024-05-08T23:33:26.636 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefdeffd700 1 -- 172.21.15.81:0/3733384988 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4011d40 con 0x7fefe015d510 2024-05-08T23:33:26.636 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefe6326700 1 -- 172.21.15.81:0/3733384988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe015d510 msgr2=0x7fefe00ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.636 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.630+0000 7fefe6326700 1 --2- 172.21.15.81:0/3733384988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe015d510 0x7fefe00ce1d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009850 tx=0x7fefd4009b60 comp rx=0 tx=0).stop 2024-05-08T23:33:26.636 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefe6326700 1 -- 172.21.15.81:0/3733384988 shutdown_connections 2024-05-08T23:33:26.636 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefe6326700 1 --2- 172.21.15.81:0/3733384988 >> v2:172.21.15.117:3300/0 conn(0x7fefe00b5320 0x7fefe00b63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.636 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefe6326700 1 --2- 172.21.15.81:0/3733384988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe015d510 0x7fefe00ce1d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.636 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefe6326700 1 -- 172.21.15.81:0/3733384988 >> 172.21.15.81:0/3733384988 conn(0x7fefe0073cf0 msgr2=0x7fefe00cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:26.636 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefe6326700 1 -- 172.21.15.81:0/3733384988 shutdown_connections 2024-05-08T23:33:26.636 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefe6326700 1 -- 172.21.15.81:0/3733384988 wait complete. 2024-05-08T23:33:26.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefe6326700 1 Processor -- start 2024-05-08T23:33:26.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefe6326700 1 -- start start 2024-05-08T23:33:26.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefe6326700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe00b5320 0x7fefe015fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefe6326700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefe00cdc90 0x7fefe0153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefe6326700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe015d510 0x7fefe0153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefe6326700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe015fa60 con 0x7fefe00cdc90 2024-05-08T23:33:26.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefe6326700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0160430 con 0x7fefe015d510 2024-05-08T23:33:26.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefe6326700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe015ee50 con 0x7fefe00b5320 2024-05-08T23:33:26.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefdf7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefe00cdc90 0x7fefe0153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefdf7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefe00cdc90 0x7fefe0153040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34356/0 (socket says 172.21.15.81:34356) 2024-05-08T23:33:26.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefdf7fe700 1 -- 172.21.15.81:0/3010257602 learned_addr learned my addr 172.21.15.81:0/3010257602 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:26.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefdffff700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe00b5320 0x7fefe015fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.634+0000 7fefe48c3700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe015d510 0x7fefe0153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.638+0000 7fefdffff700 1 -- 172.21.15.81:0/3010257602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe015d510 msgr2=0x7fefe0153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.638+0000 7fefdffff700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe015d510 0x7fefe0153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.638+0000 7fefdffff700 1 -- 172.21.15.81:0/3010257602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefe00cdc90 msgr2=0x7fefe0153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.638+0000 7fefdffff700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefe00cdc90 0x7fefe0153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.638+0000 7fefdffff700 1 -- 172.21.15.81:0/3010257602 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd8009a70 con 0x7fefe00b5320 2024-05-08T23:33:26.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.638+0000 7fefe48c3700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe015d510 0x7fefe0153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:26.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.638+0000 7fefdf7fe700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefe00cdc90 0x7fefe0153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:26.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.638+0000 7fefdffff700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe00b5320 0x7fefe015fef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fefd4016000 tx=0x7fefd4004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:26.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.638+0000 7fefdcff9700 1 -- 172.21.15.81:0/3010257602 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4004440 con 0x7fefe00b5320 2024-05-08T23:33:26.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.638+0000 7fefdcff9700 1 -- 172.21.15.81:0/3010257602 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd40045a0 con 0x7fefe00b5320 2024-05-08T23:33:26.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.638+0000 7fefdcff9700 1 -- 172.21.15.81:0/3010257602 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd40038e0 con 0x7fefe00b5320 2024-05-08T23:33:26.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.638+0000 7fefe6326700 1 -- 172.21.15.81:0/3010257602 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd40094b0 con 0x7fefe00b5320 2024-05-08T23:33:26.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.638+0000 7fefe6326700 1 -- 172.21.15.81:0/3010257602 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe015f410 con 0x7fefe00b5320 2024-05-08T23:33:26.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.638+0000 7fefe6326700 1 -- 172.21.15.81:0/3010257602 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fefe00d2a30 con 0x7fefe00b5320 2024-05-08T23:33:26.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.638+0000 7fefdcff9700 1 -- 172.21.15.81:0/3010257602 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fefd4003a40 con 0x7fefe00b5320 2024-05-08T23:33:26.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.642+0000 7fefdcff9700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fefcc066760 0x7fefcc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.642+0000 7fefdf7fe700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fefcc066760 0x7fefcc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.642+0000 7fefdcff9700 1 -- 172.21.15.81:0/3010257602 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fefd4085700 con 0x7fefe00b5320 2024-05-08T23:33:26.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.642+0000 7fefdcff9700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.81:6805/26094,v1:172.21.15.81:6807/26094] conn(0x7fefcc06c5c0 0x7fefcc06e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.642+0000 7fefdcff9700 1 -- 172.21.15.81:0/3010257602 --> [v2:172.21.15.81:6805/26094,v1:172.21.15.81:6807/26094] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fefcc06f090 con 0x7fefcc06c5c0 2024-05-08T23:33:26.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.642+0000 7fefdcff9700 1 -- 172.21.15.81:0/3010257602 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefd4085b50 con 0x7fefe00b5320 2024-05-08T23:33:26.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.642+0000 7fefe48c3700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.81:6805/26094,v1:172.21.15.81:6807/26094] conn(0x7fefcc06c5c0 0x7fefcc06e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.642+0000 7fefdf7fe700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fefcc066760 0x7fefcc068c20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fefe01432b0 tx=0x7fefd0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:26.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.642+0000 7fefe48c3700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.81:6805/26094,v1:172.21.15.81:6807/26094] conn(0x7fefcc06c5c0 0x7fefcc06e9e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:26.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.642+0000 7fefdcff9700 1 -- 172.21.15.81:0/3010257602 <== osd.3 v2:172.21.15.81:6805/26094 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fefcc06f090 con 0x7fefcc06c5c0 2024-05-08T23:33:26.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.662+0000 7fefe6326700 1 -- 172.21.15.81:0/3010257602 --> [v2:172.21.15.81:6805/26094,v1:172.21.15.81:6807/26094] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fefe00d2750 con 0x7fefcc06c5c0 2024-05-08T23:33:26.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.662+0000 7fefdcff9700 1 -- 172.21.15.81:0/3010257602 <== osd.3 v2:172.21.15.81:6805/26094 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fefd800f040 con 0x7fefcc06c5c0 2024-05-08T23:33:26.664 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:33:26.665 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:33:26.665 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:33:26.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.662+0000 7fefe6326700 1 -- 172.21.15.81:0/3010257602 >> [v2:172.21.15.81:6805/26094,v1:172.21.15.81:6807/26094] conn(0x7fefcc06c5c0 msgr2=0x7fefcc06e9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.662+0000 7fefe6326700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.81:6805/26094,v1:172.21.15.81:6807/26094] conn(0x7fefcc06c5c0 0x7fefcc06e9e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.662+0000 7fefe6326700 1 -- 172.21.15.81:0/3010257602 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fefcc066760 msgr2=0x7fefcc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.662+0000 7fefe6326700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fefcc066760 0x7fefcc068c20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fefe01432b0 tx=0x7fefd0009040 comp rx=0 tx=0).stop 2024-05-08T23:33:26.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.662+0000 7fefe6326700 1 -- 172.21.15.81:0/3010257602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe00b5320 msgr2=0x7fefe015fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.662+0000 7fefe6326700 1 --2- 172.21.15.81:0/3010257602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe00b5320 0x7fefe015fef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fefd4016000 tx=0x7fefd4004770 comp rx=0 tx=0).stop 2024-05-08T23:33:26.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.662+0000 7fefe48c3700 1 -- 172.21.15.81:0/3010257602 reap_dead start 2024-05-08T23:33:26.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.662+0000 7fefe6326700 1 -- 172.21.15.81:0/3010257602 shutdown_connections 2024-05-08T23:33:26.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.662+0000 7fefe6326700 1 -- 172.21.15.81:0/3010257602 >> 172.21.15.81:0/3010257602 conn(0x7fefe0073cf0 msgr2=0x7fefe00b7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:26.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.662+0000 7fefe6326700 1 -- 172.21.15.81:0/3010257602 shutdown_connections 2024-05-08T23:33:26.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.662+0000 7fefe6326700 1 -- 172.21.15.81:0/3010257602 wait complete. 2024-05-08T23:33:26.681 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-08T23:33:26.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.798+0000 7f666ffff700 1 Processor -- start 2024-05-08T23:33:26.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.798+0000 7f666ffff700 1 -- start start 2024-05-08T23:33:26.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.798+0000 7f666ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66700ce510 0x7f667015a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.798+0000 7f666ffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f667015aaa0 0x7f667013d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.798+0000 7f666ffff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66700ce080 con 0x7f6670071ab0 2024-05-08T23:33:26.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.798+0000 7f666ffff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66700cd7c0 con 0x7f667015aaa0 2024-05-08T23:33:26.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.798+0000 7f666ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66700cda30 con 0x7f66700ce510 2024-05-08T23:33:26.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.798+0000 7f666effd700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f6670071ab0 0x7f6670071e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49064/0 (socket says 172.21.15.81:49064) 2024-05-08T23:33:26.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.798+0000 7f666effd700 1 -- 172.21.15.81:0/3581136111 learned_addr learned my addr 172.21.15.81:0/3581136111 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:26.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f66667fc700 1 --2- 172.21.15.81:0/3581136111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66700ce510 0x7f667015a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f666f7fe700 1 --2- 172.21.15.81:0/3581136111 >> v2:172.21.15.117:3300/0 conn(0x7f667015aaa0 0x7f667013d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f66667fc700 1 -- 172.21.15.81:0/3581136111 >> v2:172.21.15.117:3300/0 conn(0x7f667015aaa0 msgr2=0x7f667013d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:26.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f666e7fc700 1 -- 172.21.15.81:0/3581136111 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688806216 0 0) 0x7f66700ce080 con 0x7f6670071ab0 2024-05-08T23:33:26.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f66667fc700 1 --2- 172.21.15.81:0/3581136111 >> v2:172.21.15.117:3300/0 conn(0x7f667015aaa0 0x7f667013d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f66667fc700 1 -- 172.21.15.81:0/3581136111 >> v1:172.21.15.81:6789/0 conn(0x7f6670071ab0 legacy=0x7f6670071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f66667fc700 1 -- 172.21.15.81:0/3581136111 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66700cdd10 con 0x7f66700ce510 2024-05-08T23:33:26.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f666f7fe700 1 --2- 172.21.15.81:0/3581136111 >> v2:172.21.15.117:3300/0 conn(0x7f667015aaa0 0x7f667013d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:26.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f66667fc700 1 --2- 172.21.15.81:0/3581136111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66700ce510 0x7f667015a560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6658009850 tx=0x7f6658009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41c7390ff5cab70c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:26.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f666e7fc700 1 -- 172.21.15.81:0/3581136111 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658011830 con 0x7f66700ce510 2024-05-08T23:33:26.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f666e7fc700 1 -- 172.21.15.81:0/3581136111 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6658011990 con 0x7f66700ce510 2024-05-08T23:33:26.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f666e7fc700 1 -- 172.21.15.81:0/3581136111 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658011d40 con 0x7f66700ce510 2024-05-08T23:33:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f666ffff700 1 -- 172.21.15.81:0/3581136111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66700ce510 msgr2=0x7f667015a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f666ffff700 1 --2- 172.21.15.81:0/3581136111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66700ce510 0x7f667015a560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6658009850 tx=0x7f6658009b60 comp rx=0 tx=0).stop 2024-05-08T23:33:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f666ffff700 1 -- 172.21.15.81:0/3581136111 shutdown_connections 2024-05-08T23:33:26.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f666ffff700 1 --2- 172.21.15.81:0/3581136111 >> v2:172.21.15.117:3300/0 conn(0x7f667015aaa0 0x7f667013d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f666ffff700 1 --2- 172.21.15.81:0/3581136111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66700ce510 0x7f667015a560 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f666ffff700 1 -- 172.21.15.81:0/3581136111 >> 172.21.15.81:0/3581136111 conn(0x7f667006c330 msgr2=0x7f667006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:26.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f666ffff700 1 -- 172.21.15.81:0/3581136111 shutdown_connections 2024-05-08T23:33:26.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.802+0000 7f666ffff700 1 -- 172.21.15.81:0/3581136111 wait complete. 2024-05-08T23:33:26.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.806+0000 7f666ffff700 1 Processor -- start 2024-05-08T23:33:26.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.806+0000 7f666ffff700 1 -- start start 2024-05-08T23:33:26.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.806+0000 7f666ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6670071ab0 0x7f667013f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.806+0000 7f666ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66700ce510 0x7f66701526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.806+0000 7f666ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f667015aaa0 0x7f6670140680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.806+0000 7f666ffff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667013d400 con 0x7f667015aaa0 2024-05-08T23:33:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.806+0000 7f666ffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667013d610 con 0x7f66700ce510 2024-05-08T23:33:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.806+0000 7f666ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667013d8f0 con 0x7f6670071ab0 2024-05-08T23:33:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.806+0000 7f666f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f667015aaa0 0x7f6670140680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.806+0000 7f666f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f667015aaa0 0x7f6670140680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34366/0 (socket says 172.21.15.81:34366) 2024-05-08T23:33:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.806+0000 7f666f7fe700 1 -- 172.21.15.81:0/3623204394 learned_addr learned my addr 172.21.15.81:0/3623204394 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.806+0000 7f666effd700 1 --2- 172.21.15.81:0/3623204394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6670071ab0 0x7f667013f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.806+0000 7f66667fc700 1 --2- 172.21.15.81:0/3623204394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66700ce510 0x7f66701526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.810+0000 7f666f7fe700 1 -- 172.21.15.81:0/3623204394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6670071ab0 msgr2=0x7f667013f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.810+0000 7f666f7fe700 1 --2- 172.21.15.81:0/3623204394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6670071ab0 0x7f667013f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.810+0000 7f666f7fe700 1 -- 172.21.15.81:0/3623204394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66700ce510 msgr2=0x7f66701526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.810+0000 7f666f7fe700 1 --2- 172.21.15.81:0/3623204394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66700ce510 0x7f66701526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.810+0000 7f666f7fe700 1 -- 172.21.15.81:0/3623204394 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6668009a70 con 0x7f667015aaa0 2024-05-08T23:33:26.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.810+0000 7f666f7fe700 1 --2- 172.21.15.81:0/3623204394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f667015aaa0 0x7f6670140680 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f666800efc0 tx=0x7f666800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:26.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.810+0000 7f666cff9700 1 -- 172.21.15.81:0/3623204394 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666800ca90 con 0x7f667015aaa0 2024-05-08T23:33:26.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.810+0000 7f666cff9700 1 -- 172.21.15.81:0/3623204394 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666800f040 con 0x7f667015aaa0 2024-05-08T23:33:26.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.810+0000 7f666ffff700 1 -- 172.21.15.81:0/3623204394 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66580094b0 con 0x7f667015aaa0 2024-05-08T23:33:26.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.810+0000 7f666cff9700 1 -- 172.21.15.81:0/3623204394 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666800ce30 con 0x7f667015aaa0 2024-05-08T23:33:26.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.810+0000 7f666ffff700 1 -- 172.21.15.81:0/3623204394 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667013deb0 con 0x7f667015aaa0 2024-05-08T23:33:26.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.810+0000 7f666ffff700 1 -- 172.21.15.81:0/3623204394 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66700684b0 con 0x7f667015aaa0 2024-05-08T23:33:26.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.810+0000 7f666cff9700 1 -- 172.21.15.81:0/3623204394 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6668014480 con 0x7f667015aaa0 2024-05-08T23:33:26.815 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.814+0000 7f666cff9700 1 --2- 172.21.15.81:0/3623204394 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6650066730 0x7f6650068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:26.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.814+0000 7f666cff9700 1 -- 172.21.15.81:0/3623204394 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6668084620 con 0x7f667015aaa0 2024-05-08T23:33:26.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.814+0000 7f666effd700 1 --2- 172.21.15.81:0/3623204394 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6650066730 0x7f6650068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:26.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.814+0000 7f666effd700 1 --2- 172.21.15.81:0/3623204394 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6650066730 0x7f6650068bf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6660000f60 tx=0x7f6660007e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:26.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.818+0000 7f666cff9700 1 -- 172.21.15.81:0/3623204394 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6668051b60 con 0x7f667015aaa0 2024-05-08T23:33:26.990 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.986+0000 7f666ffff700 1 -- 172.21.15.81:0/3623204394 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f667013da60 con 0x7f6650066730 2024-05-08T23:33:26.991 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.986+0000 7f666cff9700 1 -- 172.21.15.81:0/3623204394 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f667013da60 con 0x7f6650066730 2024-05-08T23:33:26.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.990+0000 7f666ffff700 1 -- 172.21.15.81:0/3623204394 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6650066730 msgr2=0x7f6650068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.990+0000 7f666ffff700 1 --2- 172.21.15.81:0/3623204394 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6650066730 0x7f6650068bf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6660000f60 tx=0x7f6660007e00 comp rx=0 tx=0).stop 2024-05-08T23:33:26.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.990+0000 7f666ffff700 1 -- 172.21.15.81:0/3623204394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f667015aaa0 msgr2=0x7f6670140680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:26.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.990+0000 7f666ffff700 1 --2- 172.21.15.81:0/3623204394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f667015aaa0 0x7f6670140680 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f666800efc0 tx=0x7f666800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.990+0000 7f666ffff700 1 -- 172.21.15.81:0/3623204394 shutdown_connections 2024-05-08T23:33:26.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.990+0000 7f666ffff700 1 --2- 172.21.15.81:0/3623204394 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6650066730 0x7f6650068bf0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.990+0000 7f666ffff700 1 --2- 172.21.15.81:0/3623204394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f667015aaa0 0x7f6670140680 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.990+0000 7f666ffff700 1 --2- 172.21.15.81:0/3623204394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66700ce510 0x7f66701526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.990+0000 7f666ffff700 1 --2- 172.21.15.81:0/3623204394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6670071ab0 0x7f667013f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:26.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.990+0000 7f666ffff700 1 -- 172.21.15.81:0/3623204394 >> 172.21.15.81:0/3623204394 conn(0x7f667006c330 msgr2=0x7f667006d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:26.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.990+0000 7f666ffff700 1 -- 172.21.15.81:0/3623204394 shutdown_connections 2024-05-08T23:33:26.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:26.990+0000 7f666ffff700 1 -- 172.21.15.81:0/3623204394 wait complete. 2024-05-08T23:33:26.997 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 3 to deep-scrub 2024-05-08T23:33:27.011 INFO:tasks.ceph:Scrubbing osd.4 2024-05-08T23:33:27.011 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:33:27.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.134+0000 7f261b0f1700 1 Processor -- start 2024-05-08T23:33:27.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.134+0000 7f261b0f1700 1 -- start start 2024-05-08T23:33:27.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.134+0000 7f261b0f1700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26140ceae0 0x7f261415af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.134+0000 7f261b0f1700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f261415dab0 0x7f261406c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.134+0000 7f261b0f1700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26140cde80 con 0x7f26140ce700 2024-05-08T23:33:27.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.134+0000 7f261b0f1700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261415ea30 con 0x7f261415dab0 2024-05-08T23:33:27.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.134+0000 7f261b0f1700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261415def0 con 0x7f26140ceae0 2024-05-08T23:33:27.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.134+0000 7f261a8f0700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f261415dab0 0x7f261406c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.134+0000 7f261a8f0700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f261415dab0 0x7f261406c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34376/0 (socket says 172.21.15.81:34376) 2024-05-08T23:33:27.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.134+0000 7f261a8f0700 1 -- 172.21.15.81:0/389507202 learned_addr learned my addr 172.21.15.81:0/389507202 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:27.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f26198ee700 1 --2- 172.21.15.81:0/389507202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26140ceae0 0x7f261415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f261a8f0700 1 -- 172.21.15.81:0/389507202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26140ceae0 msgr2=0x7f261415af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:27.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f261a8f0700 1 --2- 172.21.15.81:0/389507202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26140ceae0 0x7f261415af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f261a8f0700 1 -- 172.21.15.81:0/389507202 >> v1:172.21.15.117:6789/0 conn(0x7f26140ce700 legacy=0x7f26140cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f261a8f0700 1 -- 172.21.15.81:0/389507202 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261415e190 con 0x7f261415dab0 2024-05-08T23:33:27.142 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f261a8f0700 1 --2- 172.21.15.81:0/389507202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f261415dab0 0x7f261406c810 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f260c009e50 tx=0x7f260c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153eddcd4fada40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:27.142 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f26190ed700 1 -- 172.21.15.81:0/389507202 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c00e070 con 0x7f261415dab0 2024-05-08T23:33:27.142 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f26190ed700 1 -- 172.21.15.81:0/389507202 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260c00b910 con 0x7f261415dab0 2024-05-08T23:33:27.142 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f26190ed700 1 -- 172.21.15.81:0/389507202 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c00bcc0 con 0x7f261415dab0 2024-05-08T23:33:27.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f261b0f1700 1 -- 172.21.15.81:0/389507202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f261415dab0 msgr2=0x7f261406c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f261b0f1700 1 --2- 172.21.15.81:0/389507202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f261415dab0 0x7f261406c810 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f260c009e50 tx=0x7f260c00b330 comp rx=0 tx=0).stop 2024-05-08T23:33:27.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f261b0f1700 1 -- 172.21.15.81:0/389507202 shutdown_connections 2024-05-08T23:33:27.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f261b0f1700 1 --2- 172.21.15.81:0/389507202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f261415dab0 0x7f261406c810 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f261b0f1700 1 --2- 172.21.15.81:0/389507202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26140ceae0 0x7f261415af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f261b0f1700 1 -- 172.21.15.81:0/389507202 >> 172.21.15.81:0/389507202 conn(0x7f261406d650 msgr2=0x7f261406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:27.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f261b0f1700 1 -- 172.21.15.81:0/389507202 shutdown_connections 2024-05-08T23:33:27.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f261b0f1700 1 -- 172.21.15.81:0/389507202 wait complete. 2024-05-08T23:33:27.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f261b0f1700 1 Processor -- start 2024-05-08T23:33:27.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.138+0000 7f261b0f1700 1 -- start start 2024-05-08T23:33:27.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f261b0f1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26140ce700 0x7f261413beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f261b0f1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26140ceae0 0x7f26141410d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f261b0f1700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f261415dab0 0x7f2614141610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f261b0f1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26141592c0 con 0x7f26140ceae0 2024-05-08T23:33:27.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f261b0f1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2614154af0 con 0x7f26140ce700 2024-05-08T23:33:27.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f261b0f1700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26141594d0 con 0x7f261415dab0 2024-05-08T23:33:27.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f26198ee700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26140ceae0 0x7f26141410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f26198ee700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26140ceae0 0x7f26141410d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34390/0 (socket says 172.21.15.81:34390) 2024-05-08T23:33:27.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f26198ee700 1 -- 172.21.15.81:0/3068574859 learned_addr learned my addr 172.21.15.81:0/3068574859 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:27.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f261a8f0700 1 --2- 172.21.15.81:0/3068574859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f261415dab0 0x7f2614141610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f261a0ef700 1 --2- 172.21.15.81:0/3068574859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26140ce700 0x7f261413beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f26198ee700 1 -- 172.21.15.81:0/3068574859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f261415dab0 msgr2=0x7f2614141610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f26198ee700 1 --2- 172.21.15.81:0/3068574859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f261415dab0 0x7f2614141610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f26198ee700 1 -- 172.21.15.81:0/3068574859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26140ce700 msgr2=0x7f261413beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f26198ee700 1 --2- 172.21.15.81:0/3068574859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26140ce700 0x7f261413beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f26198ee700 1 -- 172.21.15.81:0/3068574859 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c009b30 con 0x7f26140ceae0 2024-05-08T23:33:27.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f261a0ef700 1 --2- 172.21.15.81:0/3068574859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26140ce700 0x7f261413beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:27.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f26198ee700 1 --2- 172.21.15.81:0/3068574859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26140ceae0 0x7f26141410d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f260800e9d0 tx=0x7f260800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:27.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f2612ffd700 1 -- 172.21.15.81:0/3068574859 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608009720 con 0x7f26140ceae0 2024-05-08T23:33:27.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f2612ffd700 1 -- 172.21.15.81:0/3068574859 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260800c850 con 0x7f26140ceae0 2024-05-08T23:33:27.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f261b0f1700 1 -- 172.21.15.81:0/3068574859 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26141597b0 con 0x7f26140ceae0 2024-05-08T23:33:27.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f2612ffd700 1 -- 172.21.15.81:0/3068574859 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260800cbc0 con 0x7f26140ceae0 2024-05-08T23:33:27.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f261b0f1700 1 -- 172.21.15.81:0/3068574859 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2614154fe0 con 0x7f26140ceae0 2024-05-08T23:33:27.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.142+0000 7f261b0f1700 1 -- 172.21.15.81:0/3068574859 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f26140b2740 con 0x7f26140ceae0 2024-05-08T23:33:27.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.146+0000 7f2612ffd700 1 -- 172.21.15.81:0/3068574859 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f260800cd20 con 0x7f26140ceae0 2024-05-08T23:33:27.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.146+0000 7f2612ffd700 1 --2- 172.21.15.81:0/3068574859 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2600066730 0x7f2600068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.146+0000 7f2612ffd700 1 -- 172.21.15.81:0/3068574859 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2608083e90 con 0x7f26140ceae0 2024-05-08T23:33:27.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.146+0000 7f261a0ef700 1 --2- 172.21.15.81:0/3068574859 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2600066730 0x7f2600068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.146+0000 7f2612ffd700 1 --2- 172.21.15.81:0/3068574859 >> [v2:172.21.15.117:6808/26388,v1:172.21.15.117:6809/26388] conn(0x7f260006c590 0x7f260006e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.146+0000 7f2612ffd700 1 -- 172.21.15.81:0/3068574859 --> [v2:172.21.15.117:6808/26388,v1:172.21.15.117:6809/26388] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f260006f060 con 0x7f260006c590 2024-05-08T23:33:27.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.146+0000 7f2612ffd700 1 -- 172.21.15.81:0/3068574859 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2608086550 con 0x7f26140ceae0 2024-05-08T23:33:27.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.146+0000 7f261a8f0700 1 --2- 172.21.15.81:0/3068574859 >> [v2:172.21.15.117:6808/26388,v1:172.21.15.117:6809/26388] conn(0x7f260006c590 0x7f260006e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.146+0000 7f261a0ef700 1 --2- 172.21.15.81:0/3068574859 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2600066730 0x7f2600068bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f26141546b0 tx=0x7f2604009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:27.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.146+0000 7f261a8f0700 1 --2- 172.21.15.81:0/3068574859 >> [v2:172.21.15.117:6808/26388,v1:172.21.15.117:6809/26388] conn(0x7f260006c590 0x7f260006e9b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:27.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.146+0000 7f2612ffd700 1 -- 172.21.15.81:0/3068574859 <== osd.4 v2:172.21.15.117:6808/26388 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f260006f060 con 0x7f260006c590 2024-05-08T23:33:27.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.166+0000 7f261b0f1700 1 -- 172.21.15.81:0/3068574859 --> [v2:172.21.15.117:6808/26388,v1:172.21.15.117:6809/26388] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f261415e750 con 0x7f260006c590 2024-05-08T23:33:27.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.166+0000 7f2612ffd700 1 -- 172.21.15.81:0/3068574859 <== osd.4 v2:172.21.15.117:6808/26388 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f260c0092e0 con 0x7f260006c590 2024-05-08T23:33:27.169 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:33:27.169 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:33:27.169 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:33:27.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.166+0000 7f261b0f1700 1 -- 172.21.15.81:0/3068574859 >> [v2:172.21.15.117:6808/26388,v1:172.21.15.117:6809/26388] conn(0x7f260006c590 msgr2=0x7f260006e9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.166+0000 7f261b0f1700 1 --2- 172.21.15.81:0/3068574859 >> [v2:172.21.15.117:6808/26388,v1:172.21.15.117:6809/26388] conn(0x7f260006c590 0x7f260006e9b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.166+0000 7f261b0f1700 1 -- 172.21.15.81:0/3068574859 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2600066730 msgr2=0x7f2600068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.166+0000 7f261b0f1700 1 --2- 172.21.15.81:0/3068574859 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f2600066730 0x7f2600068bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f26141546b0 tx=0x7f2604009040 comp rx=0 tx=0).stop 2024-05-08T23:33:27.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.166+0000 7f261b0f1700 1 -- 172.21.15.81:0/3068574859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26140ceae0 msgr2=0x7f26141410d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.166+0000 7f261b0f1700 1 --2- 172.21.15.81:0/3068574859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26140ceae0 0x7f26141410d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f260800e9d0 tx=0x7f260800ece0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.166+0000 7f261a8f0700 1 -- 172.21.15.81:0/3068574859 reap_dead start 2024-05-08T23:33:27.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.166+0000 7f261b0f1700 1 -- 172.21.15.81:0/3068574859 shutdown_connections 2024-05-08T23:33:27.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.166+0000 7f261b0f1700 1 -- 172.21.15.81:0/3068574859 >> 172.21.15.81:0/3068574859 conn(0x7f261406d650 msgr2=0x7f261415b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:27.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.166+0000 7f261b0f1700 1 -- 172.21.15.81:0/3068574859 shutdown_connections 2024-05-08T23:33:27.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.166+0000 7f261b0f1700 1 -- 172.21.15.81:0/3068574859 wait complete. 2024-05-08T23:33:27.182 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-08T23:33:27.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.298+0000 7f6ec1ed4700 1 Processor -- start 2024-05-08T23:33:27.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.298+0000 7f6ec1ed4700 1 -- start start 2024-05-08T23:33:27.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.298+0000 7f6ec1ed4700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6ebc0b3c20 0x7f6ebc1624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.298+0000 7f6ec1ed4700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ebc071e70 0x7f6ebc0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.298+0000 7f6ec1ed4700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ebc158960 con 0x7f6ebc165070 2024-05-08T23:33:27.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.298+0000 7f6ec1ed4700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ebc0b1f60 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.298+0000 7f6ec1ed4700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc071120 con 0x7f6ebc071e70 2024-05-08T23:33:27.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec0ed2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6ebc0b3c20 0x7f6ebc1624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec0ed2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6ebc0b3c20 0x7f6ebc1624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34398/0 (socket says 172.21.15.81:34398) 2024-05-08T23:33:27.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec0ed2700 1 -- 172.21.15.81:0/2400163948 learned_addr learned my addr 172.21.15.81:0/2400163948 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:27.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ebbfff700 1 --2- 172.21.15.81:0/2400163948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ebc071e70 0x7f6ebc0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec0ed2700 1 -- 172.21.15.81:0/2400163948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ebc071e70 msgr2=0x7f6ebc0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:27.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec0ed2700 1 --2- 172.21.15.81:0/2400163948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ebc071e70 0x7f6ebc0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec0ed2700 1 -- 172.21.15.81:0/2400163948 >> v1:172.21.15.117:6789/0 conn(0x7f6ebc165070 legacy=0x7f6ebc165500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec0ed2700 1 -- 172.21.15.81:0/2400163948 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc0b4540 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ebbfff700 1 --2- 172.21.15.81:0/2400163948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ebc071e70 0x7f6ebc0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:27.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec0ed2700 1 --2- 172.21.15.81:0/2400163948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ebc0b3c20 0x7f6ebc1624b0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009910 tx=0x7f6eac009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84844911ffcbeb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:27.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ebb7fe700 1 -- 172.21.15.81:0/2400163948 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00ba60 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ebb7fe700 1 -- 172.21.15.81:0/2400163948 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac00bbc0 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ebb7fe700 1 -- 172.21.15.81:0/2400163948 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac009e60 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec1ed4700 1 -- 172.21.15.81:0/2400163948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ebc0b3c20 msgr2=0x7f6ebc1624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec1ed4700 1 --2- 172.21.15.81:0/2400163948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ebc0b3c20 0x7f6ebc1624b0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009910 tx=0x7f6eac009c20 comp rx=0 tx=0).stop 2024-05-08T23:33:27.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec1ed4700 1 -- 172.21.15.81:0/2400163948 shutdown_connections 2024-05-08T23:33:27.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec1ed4700 1 --2- 172.21.15.81:0/2400163948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ebc071e70 0x7f6ebc0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec1ed4700 1 --2- 172.21.15.81:0/2400163948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ebc0b3c20 0x7f6ebc1624b0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec1ed4700 1 -- 172.21.15.81:0/2400163948 >> 172.21.15.81:0/2400163948 conn(0x7f6ebc06c330 msgr2=0x7f6ebc06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:27.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec1ed4700 1 -- 172.21.15.81:0/2400163948 shutdown_connections 2024-05-08T23:33:27.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.302+0000 7f6ec1ed4700 1 -- 172.21.15.81:0/2400163948 wait complete. 2024-05-08T23:33:27.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ec1ed4700 1 Processor -- start 2024-05-08T23:33:27.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ec1ed4700 1 -- start start 2024-05-08T23:33:27.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ec1ed4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ebc071e70 0x7f6ebc146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ec1ed4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ebc0b3c20 0x7f6ebc0b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ec1ed4700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ebc165070 0x7f6ebc0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ec1ed4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1f7090 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ec1ed4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc147220 con 0x7f6ebc071e70 2024-05-08T23:33:27.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ec1ed4700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1f72a0 con 0x7f6ebc165070 2024-05-08T23:33:27.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ebbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ebc0b3c20 0x7f6ebc0b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ec16d3700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ebc165070 0x7f6ebc0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ebbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ebc0b3c20 0x7f6ebc0b2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34402/0 (socket says 172.21.15.81:34402) 2024-05-08T23:33:27.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ebbfff700 1 -- 172.21.15.81:0/4161360387 learned_addr learned my addr 172.21.15.81:0/4161360387 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:27.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ec0ed2700 1 --2- 172.21.15.81:0/4161360387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ebc071e70 0x7f6ebc146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ebbfff700 1 -- 172.21.15.81:0/4161360387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ebc165070 msgr2=0x7f6ebc0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ebbfff700 1 --2- 172.21.15.81:0/4161360387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ebc165070 0x7f6ebc0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ebbfff700 1 -- 172.21.15.81:0/4161360387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ebc071e70 msgr2=0x7f6ebc146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ebbfff700 1 --2- 172.21.15.81:0/4161360387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ebc071e70 0x7f6ebc146ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ebbfff700 1 -- 172.21.15.81:0/4161360387 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac009570 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ebbfff700 1 --2- 172.21.15.81:0/4161360387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ebc0b3c20 0x7f6ebc0b2d80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400e810 tx=0x7f6ea400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:27.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6eb97fa700 1 -- 172.21.15.81:0/4161360387 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4004230 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6eb97fa700 1 -- 172.21.15.81:0/4161360387 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea4004390 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ec1ed4700 1 -- 172.21.15.81:0/4161360387 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc1f7580 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6eb97fa700 1 -- 172.21.15.81:0/4161360387 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea40036c0 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.306+0000 7f6ec1ed4700 1 -- 172.21.15.81:0/4161360387 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc1f7b40 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.310+0000 7f6ec1ed4700 1 -- 172.21.15.81:0/4161360387 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ebc0684b0 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.310+0000 7f6eb97fa700 1 -- 172.21.15.81:0/4161360387 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6ea4003840 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.310+0000 7f6eb97fa700 1 --2- 172.21.15.81:0/4161360387 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6ea0066730 0x7f6ea0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.310+0000 7f6eb97fa700 1 -- 172.21.15.81:0/4161360387 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6ea40837e0 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.310+0000 7f6ec0ed2700 1 --2- 172.21.15.81:0/4161360387 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6ea0066730 0x7f6ea0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.310+0000 7f6ec0ed2700 1 --2- 172.21.15.81:0/4161360387 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6ea0066730 0x7f6ea0068bf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6eac000c00 tx=0x7f6eac008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:27.322 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.318+0000 7f6eb97fa700 1 -- 172.21.15.81:0/4161360387 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6ea4050dd0 con 0x7f6ebc0b3c20 2024-05-08T23:33:27.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.486+0000 7f6ec1ed4700 1 -- 172.21.15.81:0/4161360387 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ebc0b2450 con 0x7f6ea0066730 2024-05-08T23:33:27.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.486+0000 7f6eb97fa700 1 -- 172.21.15.81:0/4161360387 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6eac00b8e0 con 0x7f6ea0066730 2024-05-08T23:33:27.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.490+0000 7f6ec1ed4700 1 -- 172.21.15.81:0/4161360387 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6ea0066730 msgr2=0x7f6ea0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.495 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.490+0000 7f6ec1ed4700 1 --2- 172.21.15.81:0/4161360387 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6ea0066730 0x7f6ea0068bf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6eac000c00 tx=0x7f6eac008030 comp rx=0 tx=0).stop 2024-05-08T23:33:27.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.490+0000 7f6ec1ed4700 1 -- 172.21.15.81:0/4161360387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ebc0b3c20 msgr2=0x7f6ebc0b2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.490+0000 7f6ec1ed4700 1 --2- 172.21.15.81:0/4161360387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ebc0b3c20 0x7f6ebc0b2d80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400e810 tx=0x7f6ea400eb20 comp rx=0 tx=0).stop 2024-05-08T23:33:27.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.494+0000 7f6ec1ed4700 1 -- 172.21.15.81:0/4161360387 shutdown_connections 2024-05-08T23:33:27.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.494+0000 7f6ec1ed4700 1 --2- 172.21.15.81:0/4161360387 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f6ea0066730 0x7f6ea0068bf0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.494+0000 7f6ec1ed4700 1 --2- 172.21.15.81:0/4161360387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ebc165070 0x7f6ebc0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.494+0000 7f6ec1ed4700 1 --2- 172.21.15.81:0/4161360387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ebc0b3c20 0x7f6ebc0b2d80 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.494+0000 7f6ec1ed4700 1 --2- 172.21.15.81:0/4161360387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ebc071e70 0x7f6ebc146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.494+0000 7f6ec1ed4700 1 -- 172.21.15.81:0/4161360387 >> 172.21.15.81:0/4161360387 conn(0x7f6ebc06c330 msgr2=0x7f6ebc0ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:27.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.494+0000 7f6ec1ed4700 1 -- 172.21.15.81:0/4161360387 shutdown_connections 2024-05-08T23:33:27.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.494+0000 7f6ec1ed4700 1 -- 172.21.15.81:0/4161360387 wait complete. 2024-05-08T23:33:27.497 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 4 to deep-scrub 2024-05-08T23:33:27.510 INFO:tasks.ceph:Scrubbing osd.5 2024-05-08T23:33:27.510 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:33:27.630 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.626+0000 7fc538dc2700 1 Processor -- start 2024-05-08T23:33:27.630 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.626+0000 7fc538dc2700 1 -- start start 2024-05-08T23:33:27.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.626+0000 7fc538dc2700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc5340b3c20 0x7fc5340ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.626+0000 7fc538dc2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5340b1f60 0x7fc53406c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.626+0000 7fc538dc2700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5340ad1e0 con 0x7fc53415e3f0 2024-05-08T23:33:27.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.626+0000 7fc538dc2700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5340ad4c0 con 0x7fc5340b3c20 2024-05-08T23:33:27.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.626+0000 7fc538dc2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340ad7a0 con 0x7fc5340b1f60 2024-05-08T23:33:27.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc53259c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc5340b3c20 0x7fc5340ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc53259c700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc5340b3c20 0x7fc5340ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34408/0 (socket says 172.21.15.81:34408) 2024-05-08T23:33:27.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc53259c700 1 -- 172.21.15.81:0/3441218406 learned_addr learned my addr 172.21.15.81:0/3441218406 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:27.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc531d9b700 1 --2- 172.21.15.81:0/3441218406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5340b1f60 0x7fc53406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc53259c700 1 -- 172.21.15.81:0/3441218406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5340b1f60 msgr2=0x7fc53406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:27.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc53259c700 1 --2- 172.21.15.81:0/3441218406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5340b1f60 0x7fc53406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc53259c700 1 -- 172.21.15.81:0/3441218406 >> v1:172.21.15.117:6789/0 conn(0x7fc53415e3f0 legacy=0x7fc534071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc53259c700 1 -- 172.21.15.81:0/3441218406 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5340ada80 con 0x7fc5340b3c20 2024-05-08T23:33:27.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc53259c700 1 --2- 172.21.15.81:0/3441218406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5340b3c20 0x7fc5340ac8e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009910 tx=0x7fc52c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5294e7e8128336 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:27.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc53159a700 1 -- 172.21.15.81:0/3441218406 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c011830 con 0x7fc5340b3c20 2024-05-08T23:33:27.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc53159a700 1 -- 172.21.15.81:0/3441218406 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c011990 con 0x7fc5340b3c20 2024-05-08T23:33:27.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc53159a700 1 -- 172.21.15.81:0/3441218406 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c011d40 con 0x7fc5340b3c20 2024-05-08T23:33:27.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc538dc2700 1 -- 172.21.15.81:0/3441218406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5340b3c20 msgr2=0x7fc5340ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc538dc2700 1 --2- 172.21.15.81:0/3441218406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5340b3c20 0x7fc5340ac8e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009910 tx=0x7fc52c009c20 comp rx=0 tx=0).stop 2024-05-08T23:33:27.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc538dc2700 1 -- 172.21.15.81:0/3441218406 shutdown_connections 2024-05-08T23:33:27.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc538dc2700 1 --2- 172.21.15.81:0/3441218406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5340b1f60 0x7fc53406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.635 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc538dc2700 1 --2- 172.21.15.81:0/3441218406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5340b3c20 0x7fc5340ac8e0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.635 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc538dc2700 1 -- 172.21.15.81:0/3441218406 >> 172.21.15.81:0/3441218406 conn(0x7fc53406a360 msgr2=0x7fc53406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:27.635 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc538dc2700 1 -- 172.21.15.81:0/3441218406 shutdown_connections 2024-05-08T23:33:27.635 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc538dc2700 1 -- 172.21.15.81:0/3441218406 wait complete. 2024-05-08T23:33:27.635 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc538dc2700 1 Processor -- start 2024-05-08T23:33:27.635 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.630+0000 7fc538dc2700 1 -- start start 2024-05-08T23:33:27.636 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc538dc2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5340b1f60 0x7fc53413fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc538dc2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5340b3c20 0x7fc534158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc538dc2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc53415e3f0 0x7fc53415bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc538dc2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53415fcf0 con 0x7fc5340b1f60 2024-05-08T23:33:27.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc538dc2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5341402a0 con 0x7fc5340b3c20 2024-05-08T23:33:27.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc538dc2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc534141100 con 0x7fc53415e3f0 2024-05-08T23:33:27.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc53259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5340b1f60 0x7fc53413fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc53259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5340b1f60 0x7fc53413fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34414/0 (socket says 172.21.15.81:34414) 2024-05-08T23:33:27.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc53259c700 1 -- 172.21.15.81:0/685956773 learned_addr learned my addr 172.21.15.81:0/685956773 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:27.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc532d9d700 1 --2- 172.21.15.81:0/685956773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc53415e3f0 0x7fc53415bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc531d9b700 1 --2- 172.21.15.81:0/685956773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5340b3c20 0x7fc534158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc53259c700 1 -- 172.21.15.81:0/685956773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc53415e3f0 msgr2=0x7fc53415bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc53259c700 1 --2- 172.21.15.81:0/685956773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc53415e3f0 0x7fc53415bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc53259c700 1 -- 172.21.15.81:0/685956773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5340b3c20 msgr2=0x7fc534158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc53259c700 1 --2- 172.21.15.81:0/685956773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5340b3c20 0x7fc534158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc53259c700 1 -- 172.21.15.81:0/685956773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c009570 con 0x7fc5340b1f60 2024-05-08T23:33:27.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc532d9d700 1 --2- 172.21.15.81:0/685956773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc53415e3f0 0x7fc53415bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:27.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc53259c700 1 --2- 172.21.15.81:0/685956773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5340b1f60 0x7fc53413fb50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc52c000c00 tx=0x7fc52c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:27.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc52affd700 1 -- 172.21.15.81:0/685956773 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c004510 con 0x7fc5340b1f60 2024-05-08T23:33:27.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc52affd700 1 -- 172.21.15.81:0/685956773 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c0036c0 con 0x7fc5340b1f60 2024-05-08T23:33:27.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc538dc2700 1 -- 172.21.15.81:0/685956773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc534140090 con 0x7fc5340b1f60 2024-05-08T23:33:27.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc52affd700 1 -- 172.21.15.81:0/685956773 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c003a70 con 0x7fc5340b1f60 2024-05-08T23:33:27.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.634+0000 7fc538dc2700 1 -- 172.21.15.81:0/685956773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5341407d0 con 0x7fc5340b1f60 2024-05-08T23:33:27.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.638+0000 7fc538dc2700 1 -- 172.21.15.81:0/685956773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc53406c870 con 0x7fc5340b1f60 2024-05-08T23:33:27.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.638+0000 7fc52affd700 1 -- 172.21.15.81:0/685956773 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc52c003bd0 con 0x7fc5340b1f60 2024-05-08T23:33:27.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.638+0000 7fc52affd700 1 --2- 172.21.15.81:0/685956773 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc514066730 0x7fc514068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.641 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.638+0000 7fc531d9b700 1 --2- 172.21.15.81:0/685956773 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc514066730 0x7fc514068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.638+0000 7fc52affd700 1 -- 172.21.15.81:0/685956773 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc52c021030 con 0x7fc5340b1f60 2024-05-08T23:33:27.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.638+0000 7fc52affd700 1 --2- 172.21.15.81:0/685956773 >> [v2:172.21.15.117:6816/26391,v1:172.21.15.117:6817/26391] conn(0x7fc51406c590 0x7fc51406e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.638+0000 7fc52affd700 1 -- 172.21.15.81:0/685956773 --> [v2:172.21.15.117:6816/26391,v1:172.21.15.117:6817/26391] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc51406f060 con 0x7fc51406c590 2024-05-08T23:33:27.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.638+0000 7fc52affd700 1 -- 172.21.15.81:0/685956773 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc52c0878f0 con 0x7fc5340b1f60 2024-05-08T23:33:27.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.638+0000 7fc532d9d700 1 --2- 172.21.15.81:0/685956773 >> [v2:172.21.15.117:6816/26391,v1:172.21.15.117:6817/26391] conn(0x7fc51406c590 0x7fc51406e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.638+0000 7fc531d9b700 1 --2- 172.21.15.81:0/685956773 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc514066730 0x7fc514068bf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc534070c40 tx=0x7fc51c009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:27.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.642+0000 7fc532d9d700 1 --2- 172.21.15.81:0/685956773 >> [v2:172.21.15.117:6816/26391,v1:172.21.15.117:6817/26391] conn(0x7fc51406c590 0x7fc51406e9b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:27.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.642+0000 7fc52affd700 1 -- 172.21.15.81:0/685956773 <== osd.5 v2:172.21.15.117:6816/26391 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc51406f060 con 0x7fc51406c590 2024-05-08T23:33:27.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.662+0000 7fc538dc2700 1 -- 172.21.15.81:0/685956773 --> [v2:172.21.15.117:6816/26391,v1:172.21.15.117:6817/26391] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc534068630 con 0x7fc51406c590 2024-05-08T23:33:27.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.662+0000 7fc52affd700 1 -- 172.21.15.81:0/685956773 <== osd.5 v2:172.21.15.117:6816/26391 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc534068630 con 0x7fc51406c590 2024-05-08T23:33:27.664 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:33:27.664 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:33:27.664 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:33:27.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.662+0000 7fc538dc2700 1 -- 172.21.15.81:0/685956773 >> [v2:172.21.15.117:6816/26391,v1:172.21.15.117:6817/26391] conn(0x7fc51406c590 msgr2=0x7fc51406e9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.662+0000 7fc538dc2700 1 --2- 172.21.15.81:0/685956773 >> [v2:172.21.15.117:6816/26391,v1:172.21.15.117:6817/26391] conn(0x7fc51406c590 0x7fc51406e9b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.662+0000 7fc538dc2700 1 -- 172.21.15.81:0/685956773 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc514066730 msgr2=0x7fc514068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.662+0000 7fc538dc2700 1 --2- 172.21.15.81:0/685956773 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fc514066730 0x7fc514068bf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc534070c40 tx=0x7fc51c009240 comp rx=0 tx=0).stop 2024-05-08T23:33:27.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.662+0000 7fc538dc2700 1 -- 172.21.15.81:0/685956773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5340b1f60 msgr2=0x7fc53413fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.662+0000 7fc538dc2700 1 --2- 172.21.15.81:0/685956773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5340b1f60 0x7fc53413fb50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc52c000c00 tx=0x7fc52c004770 comp rx=0 tx=0).stop 2024-05-08T23:33:27.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.662+0000 7fc532d9d700 1 -- 172.21.15.81:0/685956773 reap_dead start 2024-05-08T23:33:27.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.662+0000 7fc538dc2700 1 -- 172.21.15.81:0/685956773 shutdown_connections 2024-05-08T23:33:27.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.662+0000 7fc538dc2700 1 -- 172.21.15.81:0/685956773 >> 172.21.15.81:0/685956773 conn(0x7fc53406a360 msgr2=0x7fc5340aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:27.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.662+0000 7fc538dc2700 1 -- 172.21.15.81:0/685956773 shutdown_connections 2024-05-08T23:33:27.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.662+0000 7fc538dc2700 1 -- 172.21.15.81:0/685956773 wait complete. 2024-05-08T23:33:27.677 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-08T23:33:27.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.790+0000 7fdefd3ae700 1 Processor -- start 2024-05-08T23:33:27.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.790+0000 7fdefd3ae700 1 -- start start 2024-05-08T23:33:27.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefd3ae700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdef81606b0 0x7fdef80cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefd3ae700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdef806c1b0 0x7fdef80b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefd3ae700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef80d0420 con 0x7fdef806b470 2024-05-08T23:33:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefd3ae700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef80d0180 con 0x7fdef806c1b0 2024-05-08T23:33:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefd3ae700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef80b04a0 con 0x7fdef81606b0 2024-05-08T23:33:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdef77fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fdef806b470 0x7fdef806baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49072/0 (socket says 172.21.15.81:49072) 2024-05-08T23:33:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdef77fe700 1 -- 172.21.15.81:0/2912545819 learned_addr learned my addr 172.21.15.81:0/2912545819 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdef7fff700 1 --2- 172.21.15.81:0/2912545819 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdef81606b0 0x7fdef80cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefcbad700 1 --2- 172.21.15.81:0/2912545819 >> v2:172.21.15.117:3300/0 conn(0x7fdef806c1b0 0x7fdef80b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdef6ffd700 1 -- 172.21.15.81:0/2912545819 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060308831 0 0) 0x7fdef80d0420 con 0x7fdef806b470 2024-05-08T23:33:27.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdef6ffd700 1 -- 172.21.15.81:0/2912545819 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdee00035d0 con 0x7fdef806b470 2024-05-08T23:33:27.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefcbad700 1 -- 172.21.15.81:0/2912545819 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdef81606b0 msgr2=0x7fdef80cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:27.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefcbad700 1 --2- 172.21.15.81:0/2912545819 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdef81606b0 0x7fdef80cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefcbad700 1 -- 172.21.15.81:0/2912545819 >> v1:172.21.15.81:6789/0 conn(0x7fdef806b470 legacy=0x7fdef806baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefcbad700 1 -- 172.21.15.81:0/2912545819 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef80b3850 con 0x7fdef806c1b0 2024-05-08T23:33:27.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefcbad700 1 --2- 172.21.15.81:0/2912545819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef806c1b0 0x7fdef80b0f40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009d90 tx=0x7fdef000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33dddbf085ae5d85 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:27.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdef6ffd700 1 -- 172.21.15.81:0/2912545819 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef000e070 con 0x7fdef806c1b0 2024-05-08T23:33:27.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdef6ffd700 1 -- 172.21.15.81:0/2912545819 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef000bb20 con 0x7fdef806c1b0 2024-05-08T23:33:27.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdef6ffd700 1 -- 172.21.15.81:0/2912545819 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef000be90 con 0x7fdef806c1b0 2024-05-08T23:33:27.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefd3ae700 1 -- 172.21.15.81:0/2912545819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef806c1b0 msgr2=0x7fdef80b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefd3ae700 1 --2- 172.21.15.81:0/2912545819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef806c1b0 0x7fdef80b0f40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009d90 tx=0x7fdef000b330 comp rx=0 tx=0).stop 2024-05-08T23:33:27.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefd3ae700 1 -- 172.21.15.81:0/2912545819 shutdown_connections 2024-05-08T23:33:27.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefd3ae700 1 --2- 172.21.15.81:0/2912545819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef806c1b0 0x7fdef80b0f40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefd3ae700 1 --2- 172.21.15.81:0/2912545819 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdef81606b0 0x7fdef80cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefd3ae700 1 -- 172.21.15.81:0/2912545819 >> 172.21.15.81:0/2912545819 conn(0x7fdef80b2970 msgr2=0x7fdef80b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:27.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefd3ae700 1 -- 172.21.15.81:0/2912545819 shutdown_connections 2024-05-08T23:33:27.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.794+0000 7fdefd3ae700 1 -- 172.21.15.81:0/2912545819 wait complete. 2024-05-08T23:33:27.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefd3ae700 1 Processor -- start 2024-05-08T23:33:27.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefd3ae700 1 -- start start 2024-05-08T23:33:27.806 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefd3ae700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef806b470 0x7fdef809d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefd3ae700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdef806c1b0 0x7fdef809b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefd3ae700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdef81606b0 0x7fdef809be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefd3ae700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef815b540 con 0x7fdef81606b0 2024-05-08T23:33:27.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefd3ae700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef809ea60 con 0x7fdef806b470 2024-05-08T23:33:27.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefd3ae700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef809c4a0 con 0x7fdef806c1b0 2024-05-08T23:33:27.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefcbad700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdef81606b0 0x7fdef809be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdef7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef806b470 0x7fdef809d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefcbad700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdef81606b0 0x7fdef809be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34424/0 (socket says 172.21.15.81:34424) 2024-05-08T23:33:27.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefcbad700 1 -- 172.21.15.81:0/3616479873 learned_addr learned my addr 172.21.15.81:0/3616479873 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:27.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdef77fe700 1 --2- 172.21.15.81:0/3616479873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdef806c1b0 0x7fdef809b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefcbad700 1 -- 172.21.15.81:0/3616479873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdef806c1b0 msgr2=0x7fdef809b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefcbad700 1 --2- 172.21.15.81:0/3616479873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdef806c1b0 0x7fdef809b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefcbad700 1 -- 172.21.15.81:0/3616479873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef806b470 msgr2=0x7fdef809d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefcbad700 1 --2- 172.21.15.81:0/3616479873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef806b470 0x7fdef809d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.798+0000 7fdefcbad700 1 -- 172.21.15.81:0/3616479873 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef0009a70 con 0x7fdef81606b0 2024-05-08T23:33:27.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.802+0000 7fdefcbad700 1 --2- 172.21.15.81:0/3616479873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdef81606b0 0x7fdef809be00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdef0014040 tx=0x7fdef0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:27.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.802+0000 7fdef4ff9700 1 -- 172.21.15.81:0/3616479873 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0004800 con 0x7fdef81606b0 2024-05-08T23:33:27.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.802+0000 7fdef4ff9700 1 -- 172.21.15.81:0/3616479873 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef0020070 con 0x7fdef81606b0 2024-05-08T23:33:27.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.802+0000 7fdefd3ae700 1 -- 172.21.15.81:0/3616479873 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef809c780 con 0x7fdef81606b0 2024-05-08T23:33:27.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.802+0000 7fdef4ff9700 1 -- 172.21.15.81:0/3616479873 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef00276c0 con 0x7fdef81606b0 2024-05-08T23:33:27.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.802+0000 7fdefd3ae700 1 -- 172.21.15.81:0/3616479873 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef809cd40 con 0x7fdef81606b0 2024-05-08T23:33:27.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.802+0000 7fdefd3ae700 1 -- 172.21.15.81:0/3616479873 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef80b0190 con 0x7fdef81606b0 2024-05-08T23:33:27.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.802+0000 7fdef4ff9700 1 -- 172.21.15.81:0/3616479873 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdef0012430 con 0x7fdef81606b0 2024-05-08T23:33:27.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.806+0000 7fdef4ff9700 1 --2- 172.21.15.81:0/3616479873 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdee4066760 0x7fdee4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:27.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.806+0000 7fdef7fff700 1 --2- 172.21.15.81:0/3616479873 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdee4066760 0x7fdee4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:27.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.806+0000 7fdef4ff9700 1 -- 172.21.15.81:0/3616479873 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdef0022070 con 0x7fdef81606b0 2024-05-08T23:33:27.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.806+0000 7fdef7fff700 1 --2- 172.21.15.81:0/3616479873 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdee4066760 0x7fdee4068c20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdef815fea0 tx=0x7fdee8009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:27.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.810+0000 7fdef4ff9700 1 -- 172.21.15.81:0/3616479873 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdef0052fd0 con 0x7fdef81606b0 2024-05-08T23:33:27.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.978+0000 7fdefd3ae700 1 -- 172.21.15.81:0/3616479873 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fdef8160a90 con 0x7fdee4066760 2024-05-08T23:33:27.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.978+0000 7fdef4ff9700 1 -- 172.21.15.81:0/3616479873 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdee8011860 con 0x7fdee4066760 2024-05-08T23:33:27.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.982+0000 7fdefd3ae700 1 -- 172.21.15.81:0/3616479873 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdee4066760 msgr2=0x7fdee4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.982+0000 7fdefd3ae700 1 --2- 172.21.15.81:0/3616479873 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdee4066760 0x7fdee4068c20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdef815fea0 tx=0x7fdee8009240 comp rx=0 tx=0).stop 2024-05-08T23:33:27.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.982+0000 7fdefd3ae700 1 -- 172.21.15.81:0/3616479873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdef81606b0 msgr2=0x7fdef809be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:27.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.982+0000 7fdefd3ae700 1 --2- 172.21.15.81:0/3616479873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdef81606b0 0x7fdef809be00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdef0014040 tx=0x7fdef0004af0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.982+0000 7fdefd3ae700 1 -- 172.21.15.81:0/3616479873 shutdown_connections 2024-05-08T23:33:27.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.982+0000 7fdefd3ae700 1 --2- 172.21.15.81:0/3616479873 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fdee4066760 0x7fdee4068c20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.982+0000 7fdefd3ae700 1 --2- 172.21.15.81:0/3616479873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdef81606b0 0x7fdef809be00 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.982+0000 7fdefd3ae700 1 --2- 172.21.15.81:0/3616479873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdef806c1b0 0x7fdef809b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.982+0000 7fdefd3ae700 1 --2- 172.21.15.81:0/3616479873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef806b470 0x7fdef809d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:27.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.982+0000 7fdefd3ae700 1 -- 172.21.15.81:0/3616479873 >> 172.21.15.81:0/3616479873 conn(0x7fdef80b2970 msgr2=0x7fdef806f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:27.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.982+0000 7fdefd3ae700 1 -- 172.21.15.81:0/3616479873 shutdown_connections 2024-05-08T23:33:27.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:27.982+0000 7fdefd3ae700 1 -- 172.21.15.81:0/3616479873 wait complete. 2024-05-08T23:33:27.986 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 5 to deep-scrub 2024-05-08T23:33:27.997 INFO:tasks.ceph:Scrubbing osd.6 2024-05-08T23:33:27.997 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:33:28.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.114+0000 7fd9caf84700 1 Processor -- start 2024-05-08T23:33:28.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.114+0000 7fd9caf84700 1 -- start start 2024-05-08T23:33:28.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.114+0000 7fd9caf84700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9c415d380 0x7fd9c40d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.114+0000 7fd9caf84700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd9c40b8f30 0x7fd9c40d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.114+0000 7fd9caf84700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd9c40b4c10 0x7fd9c40b5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.114+0000 7fd9caf84700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c40d27a0 con 0x7fd9c40b8f30 2024-05-08T23:33:28.120 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.114+0000 7fd9caf84700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c40b55c0 con 0x7fd9c40b4c10 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.114+0000 7fd9caf84700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c40d1b20 con 0x7fd9c415d380 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.114+0000 7fd9c3fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd9c40b8f30 0x7fd9c40d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9c3fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd9c40b8f30 0x7fd9c40d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34436/0 (socket says 172.21.15.81:34436) 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9c3fff700 1 -- 172.21.15.81:0/2705600340 learned_addr learned my addr 172.21.15.81:0/2705600340 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9c8d20700 1 --2- 172.21.15.81:0/2705600340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9c415d380 0x7fd9c40d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9c9521700 1 --2- 172.21.15.81:0/2705600340 >> v2:172.21.15.117:3300/0 conn(0x7fd9c40b4c10 0x7fd9c40b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9c3fff700 1 -- 172.21.15.81:0/2705600340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9c415d380 msgr2=0x7fd9c40d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9c3fff700 1 --2- 172.21.15.81:0/2705600340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9c415d380 0x7fd9c40d2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9c3fff700 1 -- 172.21.15.81:0/2705600340 >> v2:172.21.15.117:3300/0 conn(0x7fd9c40b4c10 msgr2=0x7fd9c40b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9c3fff700 1 --2- 172.21.15.81:0/2705600340 >> v2:172.21.15.117:3300/0 conn(0x7fd9c40b4c10 0x7fd9c40b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9c3fff700 1 -- 172.21.15.81:0/2705600340 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c40d1e00 con 0x7fd9c40b8f30 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9c9521700 1 --2- 172.21.15.81:0/2705600340 >> v2:172.21.15.117:3300/0 conn(0x7fd9c40b4c10 0x7fd9c40b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9c3fff700 1 --2- 172.21.15.81:0/2705600340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c40b8f30 0x7fd9c40d11b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40c9fb0 tx=0x7fd9b4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f0aa8b1eda2b2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9c37fe700 1 -- 172.21.15.81:0/2705600340 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b400ba20 con 0x7fd9c40b8f30 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9c37fe700 1 -- 172.21.15.81:0/2705600340 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b400bb80 con 0x7fd9c40b8f30 2024-05-08T23:33:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9c37fe700 1 -- 172.21.15.81:0/2705600340 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4009e60 con 0x7fd9c40b8f30 2024-05-08T23:33:28.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9caf84700 1 -- 172.21.15.81:0/2705600340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c40b8f30 msgr2=0x7fd9c40d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.122 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9caf84700 1 --2- 172.21.15.81:0/2705600340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c40b8f30 0x7fd9c40d11b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40c9fb0 tx=0x7fd9b4009c20 comp rx=0 tx=0).stop 2024-05-08T23:33:28.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9caf84700 1 -- 172.21.15.81:0/2705600340 shutdown_connections 2024-05-08T23:33:28.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9caf84700 1 --2- 172.21.15.81:0/2705600340 >> v2:172.21.15.117:3300/0 conn(0x7fd9c40b4c10 0x7fd9c40b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9caf84700 1 --2- 172.21.15.81:0/2705600340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c40b8f30 0x7fd9c40d11b0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9caf84700 1 --2- 172.21.15.81:0/2705600340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9c415d380 0x7fd9c40d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9caf84700 1 -- 172.21.15.81:0/2705600340 >> 172.21.15.81:0/2705600340 conn(0x7fd9c4073cf0 msgr2=0x7fd9c40cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:28.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9caf84700 1 -- 172.21.15.81:0/2705600340 shutdown_connections 2024-05-08T23:33:28.123 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.118+0000 7fd9caf84700 1 -- 172.21.15.81:0/2705600340 wait complete. 2024-05-08T23:33:28.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9caf84700 1 Processor -- start 2024-05-08T23:33:28.124 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9caf84700 1 -- start start 2024-05-08T23:33:28.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9caf84700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c40b4c10 0x7fd9c40cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9caf84700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9c40b8f30 0x7fd9c40cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9caf84700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9c415d380 0x7fd9c4155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9caf84700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c40ce580 con 0x7fd9c40b4c10 2024-05-08T23:33:28.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9caf84700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c40cdce0 con 0x7fd9c415d380 2024-05-08T23:33:28.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9caf84700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c40cdfc0 con 0x7fd9c40b8f30 2024-05-08T23:33:28.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9c8d20700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c40b4c10 0x7fd9c40cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9c8d20700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c40b4c10 0x7fd9c40cd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34442/0 (socket says 172.21.15.81:34442) 2024-05-08T23:33:28.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9c8d20700 1 -- 172.21.15.81:0/2845486500 learned_addr learned my addr 172.21.15.81:0/2845486500 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:28.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9c9521700 1 --2- 172.21.15.81:0/2845486500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9c415d380 0x7fd9c4155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9c3fff700 1 --2- 172.21.15.81:0/2845486500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9c40b8f30 0x7fd9c40cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9c8d20700 1 -- 172.21.15.81:0/2845486500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9c40b8f30 msgr2=0x7fd9c40cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9c8d20700 1 --2- 172.21.15.81:0/2845486500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9c40b8f30 0x7fd9c40cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9c8d20700 1 -- 172.21.15.81:0/2845486500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9c415d380 msgr2=0x7fd9c4155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9c8d20700 1 --2- 172.21.15.81:0/2845486500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9c415d380 0x7fd9c4155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9c8d20700 1 -- 172.21.15.81:0/2845486500 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b4009570 con 0x7fd9c40b4c10 2024-05-08T23:33:28.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9c3fff700 1 --2- 172.21.15.81:0/2845486500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9c40b8f30 0x7fd9c40cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:28.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9c8d20700 1 --2- 172.21.15.81:0/2845486500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c40b4c10 0x7fd9c40cd650 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800d800 tx=0x7fd9b800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9c17fa700 1 -- 172.21.15.81:0/2845486500 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b80125f0 con 0x7fd9c40b4c10 2024-05-08T23:33:28.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9c17fa700 1 -- 172.21.15.81:0/2845486500 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b8012750 con 0x7fd9c40b4c10 2024-05-08T23:33:28.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9caf84700 1 -- 172.21.15.81:0/2845486500 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c40ce2a0 con 0x7fd9c40b4c10 2024-05-08T23:33:28.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9c17fa700 1 -- 172.21.15.81:0/2845486500 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8012ac0 con 0x7fd9c40b4c10 2024-05-08T23:33:28.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.122+0000 7fd9caf84700 1 -- 172.21.15.81:0/2845486500 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c40cb620 con 0x7fd9c40b4c10 2024-05-08T23:33:28.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.126+0000 7fd9caf84700 1 -- 172.21.15.81:0/2845486500 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9c415d760 con 0x7fd9c40b4c10 2024-05-08T23:33:28.129 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.126+0000 7fd9c17fa700 1 -- 172.21.15.81:0/2845486500 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd9b8012c20 con 0x7fd9c40b4c10 2024-05-08T23:33:28.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.126+0000 7fd9c17fa700 1 --2- 172.21.15.81:0/2845486500 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd9b0066730 0x7fd9b0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.126+0000 7fd9c17fa700 1 -- 172.21.15.81:0/2845486500 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd9b8083ea0 con 0x7fd9c40b4c10 2024-05-08T23:33:28.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.126+0000 7fd9c3fff700 1 --2- 172.21.15.81:0/2845486500 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd9b0066730 0x7fd9b0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.126+0000 7fd9c17fa700 1 --2- 172.21.15.81:0/2845486500 >> [v2:172.21.15.117:6824/26390,v1:172.21.15.117:6825/26390] conn(0x7fd9b006c590 0x7fd9b006e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.126+0000 7fd9c17fa700 1 -- 172.21.15.81:0/2845486500 --> [v2:172.21.15.117:6824/26390,v1:172.21.15.117:6825/26390] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9b006f060 con 0x7fd9b006c590 2024-05-08T23:33:28.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.126+0000 7fd9c17fa700 1 -- 172.21.15.81:0/2845486500 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9b8086560 con 0x7fd9c40b4c10 2024-05-08T23:33:28.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.126+0000 7fd9c9521700 1 --2- 172.21.15.81:0/2845486500 >> [v2:172.21.15.117:6824/26390,v1:172.21.15.117:6825/26390] conn(0x7fd9b006c590 0x7fd9b006e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.130+0000 7fd9c3fff700 1 --2- 172.21.15.81:0/2845486500 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd9b0066730 0x7fd9b0068bf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4000c00 tx=0x7fd9b4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.130+0000 7fd9c9521700 1 --2- 172.21.15.81:0/2845486500 >> [v2:172.21.15.117:6824/26390,v1:172.21.15.117:6825/26390] conn(0x7fd9b006c590 0x7fd9b006e9b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.130+0000 7fd9c17fa700 1 -- 172.21.15.81:0/2845486500 <== osd.6 v2:172.21.15.117:6824/26390 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd9b006f060 con 0x7fd9b006c590 2024-05-08T23:33:28.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.150+0000 7fd9caf84700 1 -- 172.21.15.81:0/2845486500 --> [v2:172.21.15.117:6824/26390,v1:172.21.15.117:6825/26390] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd9c40d23c0 con 0x7fd9b006c590 2024-05-08T23:33:28.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.150+0000 7fd9c17fa700 1 -- 172.21.15.81:0/2845486500 <== osd.6 v2:172.21.15.117:6824/26390 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd9bc00f040 con 0x7fd9b006c590 2024-05-08T23:33:28.154 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:33:28.154 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:33:28.155 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:33:28.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.150+0000 7fd9caf84700 1 -- 172.21.15.81:0/2845486500 >> [v2:172.21.15.117:6824/26390,v1:172.21.15.117:6825/26390] conn(0x7fd9b006c590 msgr2=0x7fd9b006e9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.150+0000 7fd9caf84700 1 --2- 172.21.15.81:0/2845486500 >> [v2:172.21.15.117:6824/26390,v1:172.21.15.117:6825/26390] conn(0x7fd9b006c590 0x7fd9b006e9b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.150+0000 7fd9caf84700 1 -- 172.21.15.81:0/2845486500 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd9b0066730 msgr2=0x7fd9b0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.150+0000 7fd9caf84700 1 --2- 172.21.15.81:0/2845486500 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fd9b0066730 0x7fd9b0068bf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4000c00 tx=0x7fd9b4008030 comp rx=0 tx=0).stop 2024-05-08T23:33:28.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.150+0000 7fd9caf84700 1 -- 172.21.15.81:0/2845486500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c40b4c10 msgr2=0x7fd9c40cd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.150+0000 7fd9caf84700 1 --2- 172.21.15.81:0/2845486500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c40b4c10 0x7fd9c40cd650 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800d800 tx=0x7fd9b800db10 comp rx=0 tx=0).stop 2024-05-08T23:33:28.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.150+0000 7fd9c9521700 1 -- 172.21.15.81:0/2845486500 reap_dead start 2024-05-08T23:33:28.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.154+0000 7fd9caf84700 1 -- 172.21.15.81:0/2845486500 shutdown_connections 2024-05-08T23:33:28.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.154+0000 7fd9caf84700 1 -- 172.21.15.81:0/2845486500 >> 172.21.15.81:0/2845486500 conn(0x7fd9c4073cf0 msgr2=0x7fd9c4160290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:28.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.154+0000 7fd9caf84700 1 -- 172.21.15.81:0/2845486500 shutdown_connections 2024-05-08T23:33:28.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.154+0000 7fd9caf84700 1 -- 172.21.15.81:0/2845486500 wait complete. 2024-05-08T23:33:28.168 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-08T23:33:28.289 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.286+0000 7f0a0ffff700 1 Processor -- start 2024-05-08T23:33:28.289 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.286+0000 7f0a0ffff700 1 -- start start 2024-05-08T23:33:28.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.286+0000 7f0a0ffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0a101596a0 0x7f0a100b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.286+0000 7f0a0ffff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a100b3590 con 0x7f0a1015a190 2024-05-08T23:33:28.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.286+0000 7f0a0ffff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a100b24f0 con 0x7f0a100b4e50 2024-05-08T23:33:28.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0ffff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a100b27d0 con 0x7f0a101596a0 2024-05-08T23:33:28.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0f7fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f0a1015a190 0x7f0a100709a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49088/0 (socket says 172.21.15.81:49088) 2024-05-08T23:33:28.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0f7fe700 1 -- 172.21.15.81:0/3557819895 learned_addr learned my addr 172.21.15.81:0/3557819895 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:28.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0effd700 1 --2- 172.21.15.81:0/3557819895 >> v2:172.21.15.117:3300/0 conn(0x7f0a101596a0 0x7f0a100b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0e7fc700 1 -- 172.21.15.81:0/3557819895 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506029200 0 0) 0x7f0a100b3590 con 0x7f0a1015a190 2024-05-08T23:33:28.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0e7fc700 1 -- 172.21.15.81:0/3557819895 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09fc0035d0 con 0x7f0a1015a190 2024-05-08T23:33:28.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0e7fc700 1 -- 172.21.15.81:0/3557819895 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1381121344 0 0) 0x7f0a100b24f0 con 0x7f0a100b4e50 2024-05-08T23:33:28.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0e7fc700 1 -- 172.21.15.81:0/3557819895 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a100b3590 con 0x7f0a100b4e50 2024-05-08T23:33:28.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0effd700 1 -- 172.21.15.81:0/3557819895 >> v1:172.21.15.117:6789/0 conn(0x7f0a100b4e50 legacy=0x7f0a10159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0effd700 1 -- 172.21.15.81:0/3557819895 >> v1:172.21.15.81:6789/0 conn(0x7f0a1015a190 legacy=0x7f0a100709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0effd700 1 -- 172.21.15.81:0/3557819895 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a100b2ab0 con 0x7f0a101596a0 2024-05-08T23:33:28.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0effd700 1 --2- 172.21.15.81:0/3557819895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a101596a0 0x7f0a100b4910 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009850 tx=0x7f0a00009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68cb62655e261594 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0e7fc700 1 -- 172.21.15.81:0/3557819895 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00011830 con 0x7f0a101596a0 2024-05-08T23:33:28.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0e7fc700 1 -- 172.21.15.81:0/3557819895 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a00011990 con 0x7f0a101596a0 2024-05-08T23:33:28.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0e7fc700 1 -- 172.21.15.81:0/3557819895 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00011d40 con 0x7f0a101596a0 2024-05-08T23:33:28.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0ffff700 1 -- 172.21.15.81:0/3557819895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a101596a0 msgr2=0x7f0a100b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0ffff700 1 --2- 172.21.15.81:0/3557819895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a101596a0 0x7f0a100b4910 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009850 tx=0x7f0a00009b60 comp rx=0 tx=0).stop 2024-05-08T23:33:28.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0ffff700 1 -- 172.21.15.81:0/3557819895 shutdown_connections 2024-05-08T23:33:28.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0ffff700 1 --2- 172.21.15.81:0/3557819895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a101596a0 0x7f0a100b4910 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0ffff700 1 -- 172.21.15.81:0/3557819895 >> 172.21.15.81:0/3557819895 conn(0x7f0a1006c330 msgr2=0x7f0a1006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:28.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0ffff700 1 -- 172.21.15.81:0/3557819895 shutdown_connections 2024-05-08T23:33:28.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.290+0000 7f0a0ffff700 1 -- 172.21.15.81:0/3557819895 wait complete. 2024-05-08T23:33:28.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0ffff700 1 Processor -- start 2024-05-08T23:33:28.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0ffff700 1 -- start start 2024-05-08T23:33:28.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a100b4e50 0x7f0a1014fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a101596a0 0x7f0a1006d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a1015a190 0x7f0a1006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0ffff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a10156d00 con 0x7f0a100b4e50 2024-05-08T23:33:28.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0ffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a101f6450 con 0x7f0a1015a190 2024-05-08T23:33:28.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a101f6730 con 0x7f0a101596a0 2024-05-08T23:33:28.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a100b4e50 0x7f0a1014fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a100b4e50 0x7f0a1014fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34446/0 (socket says 172.21.15.81:34446) 2024-05-08T23:33:28.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0effd700 1 -- 172.21.15.81:0/1736808237 learned_addr learned my addr 172.21.15.81:0/1736808237 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:28.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0f7fe700 1 --2- 172.21.15.81:0/1736808237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a1015a190 0x7f0a1006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a07fff700 1 --2- 172.21.15.81:0/1736808237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a101596a0 0x7f0a1006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0effd700 1 -- 172.21.15.81:0/1736808237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a101596a0 msgr2=0x7f0a1006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0effd700 1 --2- 172.21.15.81:0/1736808237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a101596a0 0x7f0a1006d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0effd700 1 -- 172.21.15.81:0/1736808237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a1015a190 msgr2=0x7f0a1006d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0effd700 1 --2- 172.21.15.81:0/1736808237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a1015a190 0x7f0a1006d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.294+0000 7f0a0effd700 1 -- 172.21.15.81:0/1736808237 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a000094b0 con 0x7f0a100b4e50 2024-05-08T23:33:28.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.298+0000 7f0a0effd700 1 --2- 172.21.15.81:0/1736808237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a100b4e50 0x7f0a1014fe60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0a100725c0 tx=0x7f0a00004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.298+0000 7f0a077fe700 1 -- 172.21.15.81:0/1736808237 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0001a990 con 0x7f0a100b4e50 2024-05-08T23:33:28.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.298+0000 7f0a077fe700 1 -- 172.21.15.81:0/1736808237 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0001f070 con 0x7f0a100b4e50 2024-05-08T23:33:28.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.298+0000 7f0a077fe700 1 -- 172.21.15.81:0/1736808237 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0001ad30 con 0x7f0a100b4e50 2024-05-08T23:33:28.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.298+0000 7f0a0ffff700 1 -- 172.21.15.81:0/1736808237 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a101f6a10 con 0x7f0a100b4e50 2024-05-08T23:33:28.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.298+0000 7f0a0ffff700 1 -- 172.21.15.81:0/1736808237 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a10152f30 con 0x7f0a100b4e50 2024-05-08T23:33:28.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.298+0000 7f0a077fe700 1 -- 172.21.15.81:0/1736808237 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0a0001b430 con 0x7f0a100b4e50 2024-05-08T23:33:28.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.298+0000 7f0a0ffff700 1 -- 172.21.15.81:0/1736808237 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a100684b0 con 0x7f0a100b4e50 2024-05-08T23:33:28.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.298+0000 7f0a077fe700 1 --2- 172.21.15.81:0/1736808237 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f09f0066730 0x7f09f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.298+0000 7f0a07fff700 1 --2- 172.21.15.81:0/1736808237 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f09f0066730 0x7f09f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.298+0000 7f0a077fe700 1 -- 172.21.15.81:0/1736808237 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0a00084f10 con 0x7f0a100b4e50 2024-05-08T23:33:28.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.298+0000 7f0a07fff700 1 --2- 172.21.15.81:0/1736808237 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f09f0066730 0x7f09f0068bf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0a100cb6a0 tx=0x7f09f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.306+0000 7f0a077fe700 1 -- 172.21.15.81:0/1736808237 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0a000524f0 con 0x7f0a100b4e50 2024-05-08T23:33:28.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.470+0000 7f0a0ffff700 1 -- 172.21.15.81:0/1736808237 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a101f62d0 con 0x7f09f0066730 2024-05-08T23:33:28.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.474+0000 7f0a077fe700 1 -- 172.21.15.81:0/1736808237 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f09f8001590 con 0x7f09f0066730 2024-05-08T23:33:28.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.474+0000 7f0a0ffff700 1 -- 172.21.15.81:0/1736808237 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f09f0066730 msgr2=0x7f09f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.474+0000 7f0a0ffff700 1 --2- 172.21.15.81:0/1736808237 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f09f0066730 0x7f09f0068bf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0a100cb6a0 tx=0x7f09f8009040 comp rx=0 tx=0).stop 2024-05-08T23:33:28.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.474+0000 7f0a0ffff700 1 -- 172.21.15.81:0/1736808237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a100b4e50 msgr2=0x7f0a1014fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.474+0000 7f0a0ffff700 1 --2- 172.21.15.81:0/1736808237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a100b4e50 0x7f0a1014fe60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0a100725c0 tx=0x7f0a00004770 comp rx=0 tx=0).stop 2024-05-08T23:33:28.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.474+0000 7f0a0ffff700 1 -- 172.21.15.81:0/1736808237 shutdown_connections 2024-05-08T23:33:28.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.474+0000 7f0a0ffff700 1 --2- 172.21.15.81:0/1736808237 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f09f0066730 0x7f09f0068bf0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.474+0000 7f0a0ffff700 1 --2- 172.21.15.81:0/1736808237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a1015a190 0x7f0a1006d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.474+0000 7f0a0ffff700 1 --2- 172.21.15.81:0/1736808237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a101596a0 0x7f0a1006d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.474+0000 7f0a0ffff700 1 --2- 172.21.15.81:0/1736808237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a100b4e50 0x7f0a1014fe60 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.474+0000 7f0a0ffff700 1 -- 172.21.15.81:0/1736808237 >> 172.21.15.81:0/1736808237 conn(0x7f0a1006c330 msgr2=0x7f0a10154a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:28.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.474+0000 7f0a0ffff700 1 -- 172.21.15.81:0/1736808237 shutdown_connections 2024-05-08T23:33:28.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.474+0000 7f0a0ffff700 1 -- 172.21.15.81:0/1736808237 wait complete. 2024-05-08T23:33:28.479 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 6 to deep-scrub 2024-05-08T23:33:28.490 INFO:tasks.ceph:Scrubbing osd.7 2024-05-08T23:33:28.490 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:33:28.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.602+0000 7f86f295f700 1 Processor -- start 2024-05-08T23:33:28.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.602+0000 7f86f295f700 1 -- start start 2024-05-08T23:33:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f295f700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f86ec0d1c00 0x7f86ec0d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f295f700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86ec0b3030 con 0x7f86ec0d0c30 2024-05-08T23:33:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f295f700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86ec0d1210 con 0x7f86ec15d300 2024-05-08T23:33:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f295f700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86ec0b29e0 con 0x7f86ec0d1c00 2024-05-08T23:33:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f215e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f86ec0d1c00 0x7f86ec0d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f215e700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f86ec0d1c00 0x7f86ec0d02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34456/0 (socket says 172.21.15.81:34456) 2024-05-08T23:33:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f115c700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f86ec0d0c30 0x7f86ec0b34c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49094/0 (socket says 172.21.15.81:49094) 2024-05-08T23:33:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f215e700 1 -- 172.21.15.81:0/4040401695 learned_addr learned my addr 172.21.15.81:0/4040401695 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f215e700 1 -- 172.21.15.81:0/4040401695 >> v1:172.21.15.117:6789/0 conn(0x7f86ec15d300 legacy=0x7f86ec0d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f215e700 1 -- 172.21.15.81:0/4040401695 >> v1:172.21.15.81:6789/0 conn(0x7f86ec0d0c30 legacy=0x7f86ec0b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f215e700 1 -- 172.21.15.81:0/4040401695 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec0b2cc0 con 0x7f86ec0d1c00 2024-05-08T23:33:28.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f215e700 1 --2- 172.21.15.81:0/4040401695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86ec0d1c00 0x7f86ec0d02b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009e50 tx=0x7f86e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95fd2811e6eefde server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f095b700 1 -- 172.21.15.81:0/4040401695 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e400e070 con 0x7f86ec0d1c00 2024-05-08T23:33:28.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f095b700 1 -- 172.21.15.81:0/4040401695 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e400b9c0 con 0x7f86ec0d1c00 2024-05-08T23:33:28.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f095b700 1 -- 172.21.15.81:0/4040401695 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e400bd70 con 0x7f86ec0d1c00 2024-05-08T23:33:28.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f295f700 1 -- 172.21.15.81:0/4040401695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86ec0d1c00 msgr2=0x7f86ec0d02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f295f700 1 --2- 172.21.15.81:0/4040401695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86ec0d1c00 0x7f86ec0d02b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009e50 tx=0x7f86e400b330 comp rx=0 tx=0).stop 2024-05-08T23:33:28.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f295f700 1 -- 172.21.15.81:0/4040401695 shutdown_connections 2024-05-08T23:33:28.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f295f700 1 --2- 172.21.15.81:0/4040401695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86ec0d1c00 0x7f86ec0d02b0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f295f700 1 -- 172.21.15.81:0/4040401695 >> 172.21.15.81:0/4040401695 conn(0x7f86ec09c3b0 msgr2=0x7f86ec0d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:28.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f295f700 1 -- 172.21.15.81:0/4040401695 shutdown_connections 2024-05-08T23:33:28.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.606+0000 7f86f295f700 1 -- 172.21.15.81:0/4040401695 wait complete. 2024-05-08T23:33:28.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f295f700 1 Processor -- start 2024-05-08T23:33:28.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f295f700 1 -- start start 2024-05-08T23:33:28.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f295f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86ec0d0c30 0x7f86ec15e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f295f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86ec0d1c00 0x7f86ec0b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f295f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86ec15d300 0x7f86ec0b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f295f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec1f2c10 con 0x7f86ec0d0c30 2024-05-08T23:33:28.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f295f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec15ee90 con 0x7f86ec15d300 2024-05-08T23:33:28.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f295f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec15f170 con 0x7f86ec0d1c00 2024-05-08T23:33:28.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f195d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86ec0d0c30 0x7f86ec15e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f195d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86ec0d0c30 0x7f86ec15e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34464/0 (socket says 172.21.15.81:34464) 2024-05-08T23:33:28.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f195d700 1 -- 172.21.15.81:0/4277315312 learned_addr learned my addr 172.21.15.81:0/4277315312 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:28.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f215e700 1 --2- 172.21.15.81:0/4277315312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86ec15d300 0x7f86ec0b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f115c700 1 --2- 172.21.15.81:0/4277315312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86ec0d1c00 0x7f86ec0b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f195d700 1 -- 172.21.15.81:0/4277315312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86ec0d1c00 msgr2=0x7f86ec0b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f195d700 1 --2- 172.21.15.81:0/4277315312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86ec0d1c00 0x7f86ec0b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f195d700 1 -- 172.21.15.81:0/4277315312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86ec15d300 msgr2=0x7f86ec0b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.616 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f195d700 1 --2- 172.21.15.81:0/4277315312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86ec15d300 0x7f86ec0b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f195d700 1 -- 172.21.15.81:0/4277315312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e4009b30 con 0x7f86ec0d0c30 2024-05-08T23:33:28.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.610+0000 7f86f215e700 1 --2- 172.21.15.81:0/4277315312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86ec15d300 0x7f86ec0b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:28.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.614+0000 7f86f195d700 1 --2- 172.21.15.81:0/4277315312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86ec0d0c30 0x7f86ec15e8b0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00c500 tx=0x7f86dc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.614+0000 7f86e27fc700 1 -- 172.21.15.81:0/4277315312 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc0052d0 con 0x7f86ec0d0c30 2024-05-08T23:33:28.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.614+0000 7f86e27fc700 1 -- 172.21.15.81:0/4277315312 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc005430 con 0x7f86ec0d0c30 2024-05-08T23:33:28.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.614+0000 7f86f295f700 1 -- 172.21.15.81:0/4277315312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86ec15f450 con 0x7f86ec0d0c30 2024-05-08T23:33:28.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.614+0000 7f86e27fc700 1 -- 172.21.15.81:0/4277315312 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc0055a0 con 0x7f86ec0d0c30 2024-05-08T23:33:28.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.614+0000 7f86f295f700 1 -- 172.21.15.81:0/4277315312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ec15fa10 con 0x7f86ec0d0c30 2024-05-08T23:33:28.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.614+0000 7f86f295f700 1 -- 172.21.15.81:0/4277315312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f86ec15d6e0 con 0x7f86ec0d0c30 2024-05-08T23:33:28.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.614+0000 7f86e27fc700 1 -- 172.21.15.81:0/4277315312 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f86dc0038e0 con 0x7f86ec0d0c30 2024-05-08T23:33:28.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.614+0000 7f86e27fc700 1 --2- 172.21.15.81:0/4277315312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f86d0066730 0x7f86d0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.614+0000 7f86e27fc700 1 -- 172.21.15.81:0/4277315312 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f86dc083a00 con 0x7f86ec0d0c30 2024-05-08T23:33:28.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.614+0000 7f86e27fc700 1 --2- 172.21.15.81:0/4277315312 >> [v2:172.21.15.117:6800/26389,v1:172.21.15.117:6801/26389] conn(0x7f86d006c590 0x7f86d006e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.614+0000 7f86e27fc700 1 -- 172.21.15.81:0/4277315312 --> [v2:172.21.15.117:6800/26389,v1:172.21.15.117:6801/26389] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86d006f060 con 0x7f86d006c590 2024-05-08T23:33:28.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.614+0000 7f86e27fc700 1 -- 172.21.15.81:0/4277315312 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86dc083e50 con 0x7f86ec0d0c30 2024-05-08T23:33:28.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.614+0000 7f86f115c700 1 --2- 172.21.15.81:0/4277315312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f86d0066730 0x7f86d0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.614+0000 7f86f215e700 1 --2- 172.21.15.81:0/4277315312 >> [v2:172.21.15.117:6800/26389,v1:172.21.15.117:6801/26389] conn(0x7f86d006c590 0x7f86d006e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.618+0000 7f86f215e700 1 --2- 172.21.15.81:0/4277315312 >> [v2:172.21.15.117:6800/26389,v1:172.21.15.117:6801/26389] conn(0x7f86d006c590 0x7f86d006e9b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.618+0000 7f86f115c700 1 --2- 172.21.15.81:0/4277315312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f86d0066730 0x7f86d0068bf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f86e8000f60 tx=0x7f86e8007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.618+0000 7f86e27fc700 1 -- 172.21.15.81:0/4277315312 <== osd.7 v2:172.21.15.117:6800/26389 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f86d006f060 con 0x7f86d006c590 2024-05-08T23:33:28.635 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.634+0000 7f86f295f700 1 -- 172.21.15.81:0/4277315312 --> [v2:172.21.15.117:6800/26389,v1:172.21.15.117:6801/26389] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f86ec0684b0 con 0x7f86d006c590 2024-05-08T23:33:28.636 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.634+0000 7f86e27fc700 1 -- 172.21.15.81:0/4277315312 <== osd.7 v2:172.21.15.117:6800/26389 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f86ec0684b0 con 0x7f86d006c590 2024-05-08T23:33:28.636 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-08T23:33:28.636 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:33:28.636 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-08T23:33:28.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.634+0000 7f86f295f700 1 -- 172.21.15.81:0/4277315312 >> [v2:172.21.15.117:6800/26389,v1:172.21.15.117:6801/26389] conn(0x7f86d006c590 msgr2=0x7f86d006e9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.634+0000 7f86f295f700 1 --2- 172.21.15.81:0/4277315312 >> [v2:172.21.15.117:6800/26389,v1:172.21.15.117:6801/26389] conn(0x7f86d006c590 0x7f86d006e9b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.634+0000 7f86f295f700 1 -- 172.21.15.81:0/4277315312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f86d0066730 msgr2=0x7f86d0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.634+0000 7f86f295f700 1 --2- 172.21.15.81:0/4277315312 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f86d0066730 0x7f86d0068bf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f86e8000f60 tx=0x7f86e8007d30 comp rx=0 tx=0).stop 2024-05-08T23:33:28.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.634+0000 7f86f295f700 1 -- 172.21.15.81:0/4277315312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86ec0d0c30 msgr2=0x7f86ec15e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.634+0000 7f86f295f700 1 --2- 172.21.15.81:0/4277315312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86ec0d0c30 0x7f86ec15e8b0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00c500 tx=0x7f86dc00c8c0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.634+0000 7f86f215e700 1 -- 172.21.15.81:0/4277315312 reap_dead start 2024-05-08T23:33:28.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.634+0000 7f86f295f700 1 -- 172.21.15.81:0/4277315312 shutdown_connections 2024-05-08T23:33:28.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.634+0000 7f86f295f700 1 -- 172.21.15.81:0/4277315312 >> 172.21.15.81:0/4277315312 conn(0x7f86ec09c3b0 msgr2=0x7f86ec0cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:28.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.634+0000 7f86f295f700 1 -- 172.21.15.81:0/4277315312 shutdown_connections 2024-05-08T23:33:28.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.634+0000 7f86f295f700 1 -- 172.21.15.81:0/4277315312 wait complete. 2024-05-08T23:33:28.649 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-08T23:33:28.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac06716700 1 Processor -- start 2024-05-08T23:33:28.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac06716700 1 -- start start 2024-05-08T23:33:28.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac06716700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac00159a80 0x7fac00159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac06716700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fac000709a0 0x7fac00070e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac06716700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac000b3590 con 0x7fac001596a0 2024-05-08T23:33:28.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac06716700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac000b5020 con 0x7fac000709a0 2024-05-08T23:33:28.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac06716700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac000b24f0 con 0x7fac00159a80 2024-05-08T23:33:28.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac05f15700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fac000709a0 0x7fac00070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac05f15700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fac000709a0 0x7fac00070e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34468/0 (socket says 172.21.15.81:34468) 2024-05-08T23:33:28.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac05714700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fac001596a0 0x7fac000b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49106/0 (socket says 172.21.15.81:49106) 2024-05-08T23:33:28.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac05f15700 1 -- 172.21.15.81:0/3016354773 learned_addr learned my addr 172.21.15.81:0/3016354773 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:28.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac04f13700 1 --2- 172.21.15.81:0/3016354773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac00159a80 0x7fac00159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac05f15700 1 -- 172.21.15.81:0/3016354773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac00159a80 msgr2=0x7fac00159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:28.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac05f15700 1 --2- 172.21.15.81:0/3016354773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac00159a80 0x7fac00159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac05f15700 1 -- 172.21.15.81:0/3016354773 >> v1:172.21.15.81:6789/0 conn(0x7fac001596a0 legacy=0x7fac000b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac05f15700 1 -- 172.21.15.81:0/3016354773 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac000b27d0 con 0x7fac000709a0 2024-05-08T23:33:28.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac05f15700 1 --2- 172.21.15.81:0/3016354773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac000709a0 0x7fac00070e50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009e50 tx=0x7fabf800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2979f4376539ac63 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fabf7fff700 1 -- 172.21.15.81:0/3016354773 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf800e070 con 0x7fac000709a0 2024-05-08T23:33:28.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fabf7fff700 1 -- 172.21.15.81:0/3016354773 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf800b9c0 con 0x7fac000709a0 2024-05-08T23:33:28.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fabf7fff700 1 -- 172.21.15.81:0/3016354773 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf800bd70 con 0x7fac000709a0 2024-05-08T23:33:28.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac06716700 1 -- 172.21.15.81:0/3016354773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac000709a0 msgr2=0x7fac00070e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac06716700 1 --2- 172.21.15.81:0/3016354773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac000709a0 0x7fac00070e50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009e50 tx=0x7fabf800b330 comp rx=0 tx=0).stop 2024-05-08T23:33:28.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac06716700 1 -- 172.21.15.81:0/3016354773 shutdown_connections 2024-05-08T23:33:28.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac06716700 1 --2- 172.21.15.81:0/3016354773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac000709a0 0x7fac00070e50 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac06716700 1 --2- 172.21.15.81:0/3016354773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac00159a80 0x7fac00159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac06716700 1 -- 172.21.15.81:0/3016354773 >> 172.21.15.81:0/3016354773 conn(0x7fac0006c330 msgr2=0x7fac0006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:28.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac06716700 1 -- 172.21.15.81:0/3016354773 shutdown_connections 2024-05-08T23:33:28.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.762+0000 7fac06716700 1 -- 172.21.15.81:0/3016354773 wait complete. 2024-05-08T23:33:28.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac06716700 1 Processor -- start 2024-05-08T23:33:28.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac06716700 1 -- start start 2024-05-08T23:33:28.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac06716700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac000709a0 0x7fac00141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac06716700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac001596a0 0x7fac0013ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac06716700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac00159a80 0x7fac0014d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac06716700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac00150360 con 0x7fac00159a80 2024-05-08T23:33:28.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac06716700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0015b2d0 con 0x7fac000709a0 2024-05-08T23:33:28.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac06716700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0015a9a0 con 0x7fac001596a0 2024-05-08T23:33:28.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac05f15700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac00159a80 0x7fac0014d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac05f15700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac00159a80 0x7fac0014d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34470/0 (socket says 172.21.15.81:34470) 2024-05-08T23:33:28.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac05f15700 1 -- 172.21.15.81:0/133892888 learned_addr learned my addr 172.21.15.81:0/133892888 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:28.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac05714700 1 --2- 172.21.15.81:0/133892888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac000709a0 0x7fac00141c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac04f13700 1 --2- 172.21.15.81:0/133892888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac001596a0 0x7fac0013ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac05f15700 1 -- 172.21.15.81:0/133892888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac001596a0 msgr2=0x7fac0013ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac05f15700 1 --2- 172.21.15.81:0/133892888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac001596a0 0x7fac0013ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac05f15700 1 -- 172.21.15.81:0/133892888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac000709a0 msgr2=0x7fac00141c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac05f15700 1 --2- 172.21.15.81:0/133892888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac000709a0 0x7fac00141c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac05f15700 1 -- 172.21.15.81:0/133892888 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf8009b30 con 0x7fac00159a80 2024-05-08T23:33:28.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac04f13700 1 --2- 172.21.15.81:0/133892888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac001596a0 0x7fac0013ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:28.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fac05f15700 1 --2- 172.21.15.81:0/133892888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac00159a80 0x7fac0014d700 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fac000725c0 tx=0x7fabf8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.766+0000 7fabf5ffb700 1 -- 172.21.15.81:0/133892888 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf80099b0 con 0x7fac00159a80 2024-05-08T23:33:28.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.770+0000 7fabf5ffb700 1 -- 172.21.15.81:0/133892888 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf8004970 con 0x7fac00159a80 2024-05-08T23:33:28.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.770+0000 7fabf5ffb700 1 -- 172.21.15.81:0/133892888 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf801c660 con 0x7fac00159a80 2024-05-08T23:33:28.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.770+0000 7fac06716700 1 -- 172.21.15.81:0/133892888 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac0015ac80 con 0x7fac00159a80 2024-05-08T23:33:28.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.770+0000 7fac06716700 1 -- 172.21.15.81:0/133892888 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0015c940 con 0x7fac00159a80 2024-05-08T23:33:28.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.770+0000 7fac06716700 1 -- 172.21.15.81:0/133892888 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac0015b440 con 0x7fac00159a80 2024-05-08T23:33:28.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.770+0000 7fabf5ffb700 1 -- 172.21.15.81:0/133892888 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fabf801c7c0 con 0x7fac00159a80 2024-05-08T23:33:28.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.770+0000 7fabf5ffb700 1 --2- 172.21.15.81:0/133892888 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fabe4066730 0x7fabe4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.770+0000 7fac05714700 1 --2- 172.21.15.81:0/133892888 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fabe4066730 0x7fabe4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.770+0000 7fabf5ffb700 1 -- 172.21.15.81:0/133892888 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fabf8085a70 con 0x7fac00159a80 2024-05-08T23:33:28.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.770+0000 7fac05714700 1 --2- 172.21.15.81:0/133892888 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fabe4066730 0x7fabe4068bf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fac001499f0 tx=0x7fabf0007f30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.782+0000 7fabf5ffb700 1 -- 172.21.15.81:0/133892888 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fabf80530f0 con 0x7fac00159a80 2024-05-08T23:33:28.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.950+0000 7fac06716700 1 -- 172.21.15.81:0/133892888 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fac0015b140 con 0x7fabe4066730 2024-05-08T23:33:28.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.950+0000 7fabf5ffb700 1 -- 172.21.15.81:0/133892888 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fac0015b140 con 0x7fabe4066730 2024-05-08T23:33:28.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.950+0000 7fac06716700 1 -- 172.21.15.81:0/133892888 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fabe4066730 msgr2=0x7fabe4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.950+0000 7fac06716700 1 --2- 172.21.15.81:0/133892888 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fabe4066730 0x7fabe4068bf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fac001499f0 tx=0x7fabf0007f30 comp rx=0 tx=0).stop 2024-05-08T23:33:28.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.950+0000 7fac06716700 1 -- 172.21.15.81:0/133892888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac00159a80 msgr2=0x7fac0014d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.950+0000 7fac06716700 1 --2- 172.21.15.81:0/133892888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac00159a80 0x7fac0014d700 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fac000725c0 tx=0x7fabf8004af0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.950+0000 7fac06716700 1 -- 172.21.15.81:0/133892888 shutdown_connections 2024-05-08T23:33:28.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.950+0000 7fac06716700 1 --2- 172.21.15.81:0/133892888 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7fabe4066730 0x7fabe4068bf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.950+0000 7fac06716700 1 --2- 172.21.15.81:0/133892888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac00159a80 0x7fac0014d700 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.950+0000 7fac06716700 1 --2- 172.21.15.81:0/133892888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac001596a0 0x7fac0013ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.950+0000 7fac06716700 1 --2- 172.21.15.81:0/133892888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac000709a0 0x7fac00141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.950+0000 7fac06716700 1 -- 172.21.15.81:0/133892888 >> 172.21.15.81:0/133892888 conn(0x7fac0006c330 msgr2=0x7fac000cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:28.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.950+0000 7fac06716700 1 -- 172.21.15.81:0/133892888 shutdown_connections 2024-05-08T23:33:28.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:28.950+0000 7fac06716700 1 -- 172.21.15.81:0/133892888 wait complete. 2024-05-08T23:33:28.956 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 7 to deep-scrub 2024-05-08T23:33:28.972 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:33:29.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.082+0000 7f8303fff700 1 Processor -- start 2024-05-08T23:33:29.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.082+0000 7f8303fff700 1 -- start start 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.086+0000 7f8303fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83040ce510 0x7f830415a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.086+0000 7f8303fff700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83040ce080 con 0x7f830415aaa0 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.086+0000 7f8303fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83040cd7c0 con 0x7f8304071ab0 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.086+0000 7f8303fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83040cda30 con 0x7f83040ce510 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.086+0000 7f83037fe700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f830415aaa0 0x7f830413d500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49114/0 (socket says 172.21.15.81:49114) 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.086+0000 7f83037fe700 1 -- 172.21.15.81:0/1640692557 learned_addr learned my addr 172.21.15.81:0/1640692557 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.086+0000 7f83027fc700 1 --2- 172.21.15.81:0/1640692557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83040ce510 0x7f830415a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.086+0000 7f83027fc700 1 -- 172.21.15.81:0/1640692557 >> v1:172.21.15.117:6789/0 conn(0x7f8304071ab0 legacy=0x7f8304071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.086+0000 7f83027fc700 1 -- 172.21.15.81:0/1640692557 >> v1:172.21.15.81:6789/0 conn(0x7f830415aaa0 legacy=0x7f830413d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.086+0000 7f83027fc700 1 -- 172.21.15.81:0/1640692557 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83040cdd10 con 0x7f83040ce510 2024-05-08T23:33:29.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.086+0000 7f83027fc700 1 --2- 172.21.15.81:0/1640692557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83040ce510 0x7f830415a560 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009880 tx=0x7f82f8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eff71acdd028f1d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:29.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.090+0000 7f8301ffb700 1 -- 172.21.15.81:0/1640692557 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8011830 con 0x7f83040ce510 2024-05-08T23:33:29.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.090+0000 7f8301ffb700 1 -- 172.21.15.81:0/1640692557 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f8011990 con 0x7f83040ce510 2024-05-08T23:33:29.093 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.090+0000 7f8301ffb700 1 -- 172.21.15.81:0/1640692557 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8011d20 con 0x7f83040ce510 2024-05-08T23:33:29.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.090+0000 7f8303fff700 1 -- 172.21.15.81:0/1640692557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83040ce510 msgr2=0x7f830415a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.090+0000 7f8303fff700 1 --2- 172.21.15.81:0/1640692557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83040ce510 0x7f830415a560 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009880 tx=0x7f82f8009b90 comp rx=0 tx=0).stop 2024-05-08T23:33:29.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.090+0000 7f8303fff700 1 -- 172.21.15.81:0/1640692557 shutdown_connections 2024-05-08T23:33:29.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.090+0000 7f8303fff700 1 --2- 172.21.15.81:0/1640692557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83040ce510 0x7f830415a560 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.090+0000 7f8303fff700 1 -- 172.21.15.81:0/1640692557 >> 172.21.15.81:0/1640692557 conn(0x7f830406c330 msgr2=0x7f830406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:29.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.090+0000 7f8303fff700 1 -- 172.21.15.81:0/1640692557 shutdown_connections 2024-05-08T23:33:29.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.090+0000 7f8303fff700 1 -- 172.21.15.81:0/1640692557 wait complete. 2024-05-08T23:33:29.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f8303fff700 1 Processor -- start 2024-05-08T23:33:29.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f8303fff700 1 -- start start 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f8303fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8304071ab0 0x7f830412f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f8303fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83040ce510 0x7f830413aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f8303fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f830415aaa0 0x7f8304134b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f8303fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830413ba70 con 0x7f83040ce510 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f8303fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830413bd50 con 0x7f830415aaa0 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f8303fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830413c030 con 0x7f8304071ab0 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f83027fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83040ce510 0x7f830413aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f83027fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83040ce510 0x7f830413aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34484/0 (socket says 172.21.15.81:34484) 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f83027fc700 1 -- 172.21.15.81:0/939086592 learned_addr learned my addr 172.21.15.81:0/939086592 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f83037fe700 1 --2- 172.21.15.81:0/939086592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f830415aaa0 0x7f8304134b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f8302ffd700 1 --2- 172.21.15.81:0/939086592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8304071ab0 0x7f830412f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f83027fc700 1 -- 172.21.15.81:0/939086592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8304071ab0 msgr2=0x7f830412f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f83027fc700 1 --2- 172.21.15.81:0/939086592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8304071ab0 0x7f830412f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f83027fc700 1 -- 172.21.15.81:0/939086592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f830415aaa0 msgr2=0x7f8304134b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f83027fc700 1 --2- 172.21.15.81:0/939086592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f830415aaa0 0x7f8304134b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f83027fc700 1 -- 172.21.15.81:0/939086592 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f80094b0 con 0x7f83040ce510 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f83027fc700 1 --2- 172.21.15.81:0/939086592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83040ce510 0x7f830413aff0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f82f8000c00 tx=0x7f82f8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f82eb7fe700 1 -- 172.21.15.81:0/939086592 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8004300 con 0x7f83040ce510 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f82eb7fe700 1 -- 172.21.15.81:0/939086592 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f801f070 con 0x7f83040ce510 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f8303fff700 1 -- 172.21.15.81:0/939086592 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830413c310 con 0x7f83040ce510 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f82eb7fe700 1 -- 172.21.15.81:0/939086592 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f80036c0 con 0x7f83040ce510 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.094+0000 7f8303fff700 1 -- 172.21.15.81:0/939086592 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830412fde0 con 0x7f83040ce510 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.098+0000 7f8303fff700 1 -- 172.21.15.81:0/939086592 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830413c830 con 0x7f83040ce510 2024-05-08T23:33:29.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.098+0000 7f82eb7fe700 1 -- 172.21.15.81:0/939086592 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f82f8021030 con 0x7f83040ce510 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.098+0000 7f82eb7fe700 1 --2- 172.21.15.81:0/939086592 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f82f0066730 0x7f82f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.098+0000 7f82eb7fe700 1 -- 172.21.15.81:0/939086592 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f82f8085140 con 0x7f83040ce510 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.098+0000 7f8302ffd700 1 --2- 172.21.15.81:0/939086592 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f82f0066730 0x7f82f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.102+0000 7f8302ffd700 1 --2- 172.21.15.81:0/939086592 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f82f0066730 0x7f82f0068bf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f82f4000f60 tx=0x7f82f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:29.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.106+0000 7f82eb7fe700 1 -- 172.21.15.81:0/939086592 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f830413c830 con 0x7f83040ce510 2024-05-08T23:33:29.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.274+0000 7f8303fff700 1 -- 172.21.15.81:0/939086592 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f830413bec0 con 0x7f82f0066730 2024-05-08T23:33:29.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.278+0000 7f82eb7fe700 1 -- 172.21.15.81:0/939086592 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49725 (secure 0 0 0) 0x7f830413bec0 con 0x7f82f0066730 2024-05-08T23:33:29.282 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:29.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.282+0000 7f8303fff700 1 -- 172.21.15.81:0/939086592 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f82f0066730 msgr2=0x7f82f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.282+0000 7f8303fff700 1 --2- 172.21.15.81:0/939086592 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f82f0066730 0x7f82f0068bf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f82f4000f60 tx=0x7f82f4009040 comp rx=0 tx=0).stop 2024-05-08T23:33:29.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.282+0000 7f8303fff700 1 -- 172.21.15.81:0/939086592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83040ce510 msgr2=0x7f830413aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.282+0000 7f8303fff700 1 --2- 172.21.15.81:0/939086592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83040ce510 0x7f830413aff0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f82f8000c00 tx=0x7f82f8004770 comp rx=0 tx=0).stop 2024-05-08T23:33:29.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.282+0000 7f8303fff700 1 -- 172.21.15.81:0/939086592 shutdown_connections 2024-05-08T23:33:29.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.282+0000 7f8303fff700 1 --2- 172.21.15.81:0/939086592 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f82f0066730 0x7f82f0068bf0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.282+0000 7f8303fff700 1 --2- 172.21.15.81:0/939086592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f830415aaa0 0x7f8304134b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.282+0000 7f8303fff700 1 --2- 172.21.15.81:0/939086592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83040ce510 0x7f830413aff0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.282+0000 7f8303fff700 1 --2- 172.21.15.81:0/939086592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8304071ab0 0x7f830412f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.282+0000 7f8303fff700 1 -- 172.21.15.81:0/939086592 >> 172.21.15.81:0/939086592 conn(0x7f830406c330 msgr2=0x7f830415b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:29.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.282+0000 7f8303fff700 1 -- 172.21.15.81:0/939086592 shutdown_connections 2024-05-08T23:33:29.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:29.282+0000 7f8303fff700 1 -- 172.21.15.81:0/939086592 wait complete. 2024-05-08T23:33:29.285 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:33:29.297 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-05-08T23:33:27.381206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":230640,"kb_used_data":6280,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":754744080,"statfs":{"total":773094113280,"available":772857937920,"internally_reserved":0,"allocated":6430720,"data_stored":3348264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":229625400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14891,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001838"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":193,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-08T23:33:20.047260+0000","last_change":"2024-05-08T23:30:27.226484+0000","last_active":"2024-05-08T23:33:20.047260+0000","last_peered":"2024-05-08T23:33:20.047260+0000","last_clean":"2024-05-08T23:33:20.047260+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:33:20.047260+0000","last_undegraded":"2024-05-08T23:33:20.047260+0000","last_fullsized":"2024-05-08T23:33:20.047260+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:30:26.204118+0000","last_clean_scrub_stamp":"2024-05-08T23:30:26.204118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:14.267287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":270941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.89400000000000002,"15min":0.90500000000000003},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1719999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.2350000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.96899999999999997,"15min":0.97899999999999998},"min":{"1min":0.58299999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1910000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.92500000000000004}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.59099999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6579999999999999,"5min":2.004,"15min":2.004},"last":1.125},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.92600000000000005,"15min":0.92400000000000004},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.327,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.0229999999999999}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88300000000000001,"15min":0.88500000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.196},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.95999999999999996,"15min":0.97099999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.218,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.3029999999999999}]},{"osd":3,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.89200000000000002,"15min":0.88400000000000001},"min":{"1min":0.65100000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.071},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96899999999999997,"15min":0.97799999999999998},"min":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2290000000000001,"5min":1.272,"15min":1.272},"last":0.97999999999999998}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.71499999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.1439999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0249999999999999,"15min":1.0269999999999999},"min":{"1min":0.67800000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.173}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0469999999999999,"15min":1.056},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4450000000000001,"5min":2.109,"15min":2.109},"last":1.095},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1240000000000001,"15min":1.1439999999999999},"min":{"1min":0.75800000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.046}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1970000000000001,"15min":1.2070000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5660000000000001,"5min":2.544,"15min":2.544},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.103,"15min":1.109},"min":{"1min":0.81200000000000006,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.155}]}]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27664,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":655360,"data_stored":270941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2969999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.97099999999999997,"15min":0.98299999999999998},"min":{"1min":0.63700000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1859999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.78100000000000003}]},{"osd":1,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.90900000000000003,"15min":0.90400000000000003},"min":{"1min":0.67300000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.88400000000000001,"15min":0.878},"min":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.82299999999999995}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.91800000000000004,"15min":0.91700000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.385,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.93700000000000006,"15min":0.92500000000000004},"min":{"1min":0.70299999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.454,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.86699999999999999}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.97199999999999998,"15min":0.98399999999999999},"min":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.252,"5min":1.395,"15min":1.395},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.004,"5min":0.96099999999999997,"15min":0.95399999999999996},"min":{"1min":0.63700000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.94499999999999995}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.995,"15min":1.0129999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.1339999999999999,"15min":1.1599999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.484,"5min":2.464,"15min":2.464},"last":0.998}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0800000000000001,"15min":1.0820000000000001},"min":{"1min":0.82099999999999995,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.2010000000000001}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1930000000000001,"15min":1.216},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.607,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.1759999999999999,"15min":1.1779999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.1639999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":270941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.84099999999999997,"15min":0.85099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2030000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77900000000000003,"15min":0.77300000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.91500000000000004}]},{"osd":1,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.70999999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1459999999999999,"5min":1.268,"15min":1.268},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.88800000000000001,"15min":0.90000000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.125,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.96899999999999997}]},{"osd":2,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.88800000000000001,"15min":0.89800000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.008},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89900000000000002,"15min":0.90700000000000003},"min":{"1min":0.65400000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0249999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.88600000000000001}]},{"osd":3,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.85299999999999998,"15min":0.84899999999999998},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1759999999999999,"5min":1.23,"15min":1.23},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2390000000000001,"5min":1.431,"15min":1.431},"last":1.085}]},{"osd":5,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3420000000000001,"5min":1.413,"15min":1.413},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95899999999999996,"15min":0.95099999999999996},"min":{"1min":0.73999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.278,"5min":1.504,"15min":1.504},"last":0.58199999999999996}]},{"osd":6,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0269999999999999,"15min":1.0309999999999999},"min":{"1min":0.73899999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1419999999999999,"15min":1.1559999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3859999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.0409999999999999}]},{"osd":7,"last update":"Wed May 8 23:32:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0800000000000001,"15min":1.0840000000000001},"min":{"1min":0.69899999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.4379999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.113,"5min":1.143,"15min":1.1479999999999999},"min":{"1min":0.94799999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4099999999999999,"5min":1.583,"15min":1.583},"last":1.1279999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27664,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":655360,"data_stored":270941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.49099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.46300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":1.032,"15min":1.032},"last":0.73699999999999999}]},{"osd":1,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70599999999999996,"15min":0.71099999999999997},"min":{"1min":0.50700000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.89300000000000002,"5min":1,"15min":1},"last":0.89600000000000002}]},{"osd":2,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0469999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80200000000000005,"15min":0.79900000000000004},"min":{"1min":0.56699999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0229999999999999,"5min":1.242,"15min":1.242},"last":0.67000000000000004}]},{"osd":4,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.65600000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.129,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.96699999999999997,"15min":0.97399999999999998},"min":{"1min":0.78000000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.244,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.85799999999999998}]},{"osd":5,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.014,"15min":1.022},"min":{"1min":0.80300000000000005,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.161,"5min":1.623,"15min":1.623},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.995,"15min":1.004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1359999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.99099999999999999}]},{"osd":6,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.997,"15min":1.0009999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.232,"5min":1.482,"15min":1.482},"last":1.173},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.093,"15min":1.111},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3480000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.0820000000000001}]},{"osd":7,"last update":"Wed May 8 23:32:29 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0800000000000001,"15min":1.0860000000000001},"min":{"1min":0.78400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.456,"5min":1.514,"15min":1.514},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0580000000000001,"15min":1.0600000000000001},"min":{"1min":0.75,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.2869999999999999,"5min":1.552,"15min":1.552},"last":1.2270000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":270941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80400000000000005,"5min":1.099,"15min":1.099},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0660000000000001,"5min":1.129,"15min":1.129},"last":0.628}]},{"osd":1,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70999999999999996,"15min":0.70899999999999996},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.72099999999999997}]},{"osd":3,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0509999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.96799999999999997,"15min":0.97799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.27,"5min":1.367,"15min":1.367},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.95599999999999996,"15min":0.96399999999999997},"min":{"1min":0.69699999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.006}]},{"osd":5,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.98999999999999999,"15min":0.98599999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0760000000000001,"15min":1.0840000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.91800000000000004}]},{"osd":6,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0620000000000001,"15min":1.0740000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4239999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.075,"15min":1.075},"min":{"1min":0.85699999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.94099999999999995}]},{"osd":7,"last update":"Wed May 8 23:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1220000000000001,"15min":1.1240000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1000000000000001,"15min":1.1020000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.492,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.89600000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":270941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.51300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":1.113},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.44,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65000000000000002}]},{"osd":2,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88400000000000001,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.68899999999999995,"15min":0.67000000000000004},"min":{"1min":0.53900000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.69999999999999996}]},{"osd":3,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73499999999999999,"15min":0.72599999999999998},"min":{"1min":0.57999999999999996,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.42799999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.038,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.83599999999999997}]},{"osd":4,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.95099999999999996,"15min":0.95199999999999996},"min":{"1min":0.67600000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.256,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.95499999999999996,"15min":0.94299999999999995},"min":{"1min":0.77300000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.2709999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.85899999999999999}]},{"osd":5,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0109999999999999,"15min":1.0089999999999999},"min":{"1min":0.72299999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.403,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.077,"15min":1.079},"min":{"1min":0.85499999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3879999999999999,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.034}]},{"osd":6,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0369999999999999,"15min":1.034},"min":{"1min":0.80000000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.4330000000000001,"5min":1.633,"15min":1.633},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.095,"5min":1.135,"15min":1.1419999999999999},"min":{"1min":0.80500000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.4750000000000001,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.1459999999999999}]},{"osd":7,"last update":"Wed May 8 23:32:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.0209999999999999,"15min":1.0009999999999999},"min":{"1min":0.83099999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.371,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.004},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.081,"15min":1.0820000000000001},"min":{"1min":0.84999999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.306,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.073}]}]},{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1249280,"data_stored":861309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.88300000000000001,"15min":0.89700000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.625},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.145}]},{"osd":1,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.86399999999999999,"15min":0.86699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1379999999999999,"5min":1.359,"15min":1.359},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.91200000000000003,"15min":0.91600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.23,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.0620000000000001}]},{"osd":2,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88,"15min":0.873},"min":{"1min":0.57499999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88,"15min":0.88500000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.133,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.68100000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.91300000000000003,"15min":0.91100000000000003},"min":{"1min":0.60699999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.99299999999999999,"15min":0.99099999999999999},"min":{"1min":0.80100000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2470000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.96199999999999997}]},{"osd":4,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.90400000000000003,"15min":0.91200000000000003},"min":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.73999999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.1899999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.1000000000000001}]},{"osd":5,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.028,"15min":1.028},"min":{"1min":0.64100000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.405,"5min":1.714,"15min":1.714},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.1950000000000001,"5min":1.325,"15min":1.325},"last":0.94399999999999995}]},{"osd":6,"last update":"Wed May 8 23:32:30 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.048,"15min":1.048},"min":{"1min":0.86599999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.4430000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.032},{"interface":"front","average":{"1min":1.091,"5min":1.105,"15min":1.107},"min":{"1min":0.83999999999999997,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.359,"5min":1.496,"15min":1.496},"last":1.1679999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542183,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1249280,"data_stored":861309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.47499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96399999999999997,"5min":1.351,"15min":1.351},"last":0.68899999999999995}]},{"osd":2,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.51500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.238,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.546,"5min":1.869,"15min":1.869},"last":0.55800000000000005}]},{"osd":3,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.84199999999999997,"15min":0.85599999999999998},"min":{"1min":0.55200000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.94199999999999995,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.80800000000000005}]},{"osd":4,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97199999999999998,"15min":0.97299999999999998},"min":{"1min":0.71999999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.357,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.04},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.90800000000000003,"15min":0.89600000000000002},"min":{"1min":0.69899999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.2490000000000001}]},{"osd":5,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.052,"15min":1.0600000000000001},"min":{"1min":0.76800000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3819999999999999,"5min":1.474,"15min":1.474},"last":1.173},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.92500000000000004,"15min":0.91400000000000003},"min":{"1min":0.72699999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.0580000000000001}]},{"osd":6,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.056,"15min":1.0720000000000001},"min":{"1min":0.76900000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.171,"5min":1.736,"15min":1.736},"last":1.095},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1180000000000001,"15min":1.121},"min":{"1min":0.89000000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4830000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.0009999999999999}]},{"osd":7,"last update":"Wed May 8 23:32:28 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.113,"15min":1.117},"min":{"1min":0.66600000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.397,"5min":1.72,"15min":1.72},"last":1.117},{"interface":"front","average":{"1min":1.097,"5min":1.109,"15min":1.111},"min":{"1min":0.86399999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.0169999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:33:29.301 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-08T23:30:26.204118+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=30, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=33, tm_sec=24, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:33:29.301 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T23:33:36.402 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:33:36.405 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:33:49.302 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:33:49.424 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f901ef96700 1 Processor -- start 2024-05-08T23:33:49.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f901ef96700 1 -- start start 2024-05-08T23:33:49.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f901ef96700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9018161bd0 0x7f901806e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f901ef96700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90180b35f0 0x7f90180ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f901ef96700 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90180cc7c0 con 0x7f90180c3fb0 2024-05-08T23:33:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f901ef96700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901806e9c0 con 0x7f9018161bd0 2024-05-08T23:33:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f901ef96700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90180cc170 con 0x7f90180b35f0 2024-05-08T23:33:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f9017fff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9018161bd0 0x7f901806e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f901cd32700 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f90180c3fb0 0x7f90180c4390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50964/0 (socket says 172.21.15.81:50964) 2024-05-08T23:33:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f901cd32700 1 -- 172.21.15.81:0/3536846807 learned_addr learned my addr 172.21.15.81:0/3536846807 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T23:33:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f901d533700 1 --2- 172.21.15.81:0/3536846807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90180b35f0 0x7f90180ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f90177fe700 1 -- 172.21.15.81:0/3536846807 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614670386 0 0) 0x7f90180cc7c0 con 0x7f90180c3fb0 2024-05-08T23:33:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f90177fe700 1 -- 172.21.15.81:0/3536846807 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90040035d0 con 0x7f90180c3fb0 2024-05-08T23:33:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f9017fff700 1 -- 172.21.15.81:0/3536846807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90180b35f0 msgr2=0x7f90180ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f9017fff700 1 --2- 172.21.15.81:0/3536846807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90180b35f0 0x7f90180ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f9017fff700 1 -- 172.21.15.81:0/3536846807 >> v1:172.21.15.81:6789/0 conn(0x7f90180c3fb0 legacy=0x7f90180c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:49.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f9017fff700 1 -- 172.21.15.81:0/3536846807 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90180cc450 con 0x7f9018161bd0 2024-05-08T23:33:49.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f9017fff700 1 --2- 172.21.15.81:0/3536846807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9018161bd0 0x7f901806e3e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9010009d90 tx=0x7f901000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69561240fe3a3e15 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:49.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f90177fe700 1 -- 172.21.15.81:0/3536846807 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901000e070 con 0x7f9018161bd0 2024-05-08T23:33:49.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f90177fe700 1 -- 172.21.15.81:0/3536846807 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901000bb20 con 0x7f9018161bd0 2024-05-08T23:33:49.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.421+0000 7f90177fe700 1 -- 172.21.15.81:0/3536846807 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901000be90 con 0x7f9018161bd0 2024-05-08T23:33:49.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.425+0000 7f901ef96700 1 -- 172.21.15.81:0/3536846807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9018161bd0 msgr2=0x7f901806e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:49.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.425+0000 7f901ef96700 1 --2- 172.21.15.81:0/3536846807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9018161bd0 0x7f901806e3e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9010009d90 tx=0x7f901000b330 comp rx=0 tx=0).stop 2024-05-08T23:33:49.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.425+0000 7f901ef96700 1 -- 172.21.15.81:0/3536846807 shutdown_connections 2024-05-08T23:33:49.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.425+0000 7f901ef96700 1 --2- 172.21.15.81:0/3536846807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90180b35f0 0x7f90180ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.425+0000 7f901ef96700 1 --2- 172.21.15.81:0/3536846807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9018161bd0 0x7f901806e3e0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.425+0000 7f901ef96700 1 -- 172.21.15.81:0/3536846807 >> 172.21.15.81:0/3536846807 conn(0x7f90180b2ad0 msgr2=0x7f90180b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:49.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.425+0000 7f901ef96700 1 -- 172.21.15.81:0/3536846807 shutdown_connections 2024-05-08T23:33:49.429 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.425+0000 7f901ef96700 1 -- 172.21.15.81:0/3536846807 wait complete. 2024-05-08T23:33:49.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901ef96700 1 Processor -- start 2024-05-08T23:33:49.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901ef96700 1 -- start start 2024-05-08T23:33:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901ef96700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90180b35f0 0x7f9018160370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901ef96700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90180c3fb0 0x7f901815f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901ef96700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9018161bd0 0x7f90180c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901ef96700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901815eea0 con 0x7f90180b35f0 2024-05-08T23:33:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901ef96700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9018160bc0 con 0x7f9018161bd0 2024-05-08T23:33:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901ef96700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9018160ea0 con 0x7f90180c3fb0 2024-05-08T23:33:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901cd32700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90180b35f0 0x7f9018160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901cd32700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90180b35f0 0x7f9018160370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57790/0 (socket says 172.21.15.81:57790) 2024-05-08T23:33:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901cd32700 1 -- 172.21.15.81:0/3916214902 learned_addr learned my addr 172.21.15.81:0/3916214902 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901d533700 1 --2- 172.21.15.81:0/3916214902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9018161bd0 0x7f90180c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f9017fff700 1 --2- 172.21.15.81:0/3916214902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90180c3fb0 0x7f901815f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901cd32700 1 -- 172.21.15.81:0/3916214902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90180c3fb0 msgr2=0x7f901815f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901cd32700 1 --2- 172.21.15.81:0/3916214902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90180c3fb0 0x7f901815f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901cd32700 1 -- 172.21.15.81:0/3916214902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9018161bd0 msgr2=0x7f90180c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901cd32700 1 --2- 172.21.15.81:0/3916214902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9018161bd0 0x7f90180c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901cd32700 1 -- 172.21.15.81:0/3916214902 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9010009a70 con 0x7f90180b35f0 2024-05-08T23:33:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901d533700 1 --2- 172.21.15.81:0/3916214902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9018161bd0 0x7f90180c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901cd32700 1 --2- 172.21.15.81:0/3916214902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90180b35f0 0x7f9018160370 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f900000c4f0 tx=0x7f900000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f90157fa700 1 -- 172.21.15.81:0/3916214902 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000005d90 con 0x7f90180b35f0 2024-05-08T23:33:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f90157fa700 1 -- 172.21.15.81:0/3916214902 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90000070d0 con 0x7f90180b35f0 2024-05-08T23:33:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901ef96700 1 -- 172.21.15.81:0/3916214902 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90180cd490 con 0x7f90180b35f0 2024-05-08T23:33:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f90157fa700 1 -- 172.21.15.81:0/3916214902 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000007440 con 0x7f90180b35f0 2024-05-08T23:33:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.429+0000 7f901ef96700 1 -- 172.21.15.81:0/3916214902 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90181608b0 con 0x7f90180b35f0 2024-05-08T23:33:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.433+0000 7f901ef96700 1 -- 172.21.15.81:0/3916214902 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901815f350 con 0x7f90180b35f0 2024-05-08T23:33:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.433+0000 7f90157fa700 1 -- 172.21.15.81:0/3916214902 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f90000075a0 con 0x7f90180b35f0 2024-05-08T23:33:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.433+0000 7f90157fa700 1 --2- 172.21.15.81:0/3916214902 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8ff8066730 0x7f8ff8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.433+0000 7f90157fa700 1 -- 172.21.15.81:0/3916214902 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9000083200 con 0x7f90180b35f0 2024-05-08T23:33:49.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.433+0000 7f9017fff700 1 --2- 172.21.15.81:0/3916214902 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8ff8066730 0x7f8ff8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:49.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.437+0000 7f9017fff700 1 --2- 172.21.15.81:0/3916214902 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8ff8066730 0x7f8ff8068bf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f901000b370 tx=0x7f9010011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:49.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.445+0000 7f90157fa700 1 -- 172.21.15.81:0/3916214902 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f901815f350 con 0x7f90180b35f0 2024-05-08T23:33:49.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.617+0000 7f901ef96700 1 -- 172.21.15.81:0/3916214902 --> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f901815f350 con 0x7f8ff8066730 2024-05-08T23:33:49.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.617+0000 7f90157fa700 1 -- 172.21.15.81:0/3916214902 <== mgr.4107 v2:172.21.15.81:6832/25965 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49441 (secure 0 0 0) 0x7f901815f350 con 0x7f8ff8066730 2024-05-08T23:33:49.624 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.621+0000 7f901ef96700 1 -- 172.21.15.81:0/3916214902 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8ff8066730 msgr2=0x7f8ff8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.621+0000 7f901ef96700 1 --2- 172.21.15.81:0/3916214902 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8ff8066730 0x7f8ff8068bf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f901000b370 tx=0x7f9010011000 comp rx=0 tx=0).stop 2024-05-08T23:33:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.621+0000 7f901ef96700 1 -- 172.21.15.81:0/3916214902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90180b35f0 msgr2=0x7f9018160370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.621+0000 7f901ef96700 1 --2- 172.21.15.81:0/3916214902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90180b35f0 0x7f9018160370 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f900000c4f0 tx=0x7f900000c8b0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.621+0000 7f901ef96700 1 -- 172.21.15.81:0/3916214902 shutdown_connections 2024-05-08T23:33:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.621+0000 7f901ef96700 1 --2- 172.21.15.81:0/3916214902 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f8ff8066730 0x7f8ff8068bf0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.621+0000 7f901ef96700 1 --2- 172.21.15.81:0/3916214902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9018161bd0 0x7f90180c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.621+0000 7f901ef96700 1 --2- 172.21.15.81:0/3916214902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90180c3fb0 0x7f901815f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.621+0000 7f901ef96700 1 --2- 172.21.15.81:0/3916214902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90180b35f0 0x7f9018160370 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.621+0000 7f901ef96700 1 -- 172.21.15.81:0/3916214902 >> 172.21.15.81:0/3916214902 conn(0x7f90180b2ad0 msgr2=0x7f90180b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.621+0000 7f901ef96700 1 -- 172.21.15.81:0/3916214902 shutdown_connections 2024-05-08T23:33:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.621+0000 7f901ef96700 1 -- 172.21.15.81:0/3916214902 wait complete. 2024-05-08T23:33:49.627 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-05-08T23:33:49.638 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-05-08T23:33:49.386922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":230632,"kb_used_data":6280,"kb_used_omap":12,"kb_used_meta":224243,"kb_avail":754744088,"statfs":{"total":773094113280,"available":772857946112,"internally_reserved":0,"allocated":6430720,"data_stored":3348264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":229625400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003057"},"pg_stats":[{"pgid":"1.0","version":"55'119","reported_seq":207,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-08T23:33:30.627138+0000","last_change":"2024-05-08T23:33:30.627471+0000","last_active":"2024-05-08T23:33:30.627138+0000","last_peered":"2024-05-08T23:33:30.627138+0000","last_clean":"2024-05-08T23:33:30.627138+0000","last_became_active":"2024-05-08T23:30:27.225978+0000","last_became_peered":"2024-05-08T23:30:27.225978+0000","last_unstale":"2024-05-08T23:33:30.627138+0000","last_undegraded":"2024-05-08T23:33:30.627138+0000","last_fullsized":"2024-05-08T23:33:30.627138+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'119","last_scrub_stamp":"2024-05-08T23:33:30.627066+0000","last_deep_scrub":"55'119","last_deep_scrub_stamp":"2024-05-08T23:33:30.627066+0000","last_clean_scrub_stamp":"2024-05-08T23:33:30.627066+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:52.091015+0000","scrub_duration":0.010018842,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":270941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.87,"15min":0.89700000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2350000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.94099999999999995,"15min":0.96899999999999997},"min":{"1min":0.628,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.179,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.82899999999999996}]},{"osd":1,"last update":"Wed May 8 23:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.86799999999999999,"15min":0.86499999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1479999999999999,"5min":2.004,"15min":2.004},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.63900000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.0589999999999999,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.66900000000000004}]},{"osd":2,"last update":"Wed May 8 23:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.63400000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.196,"5min":1.71,"15min":1.71},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.96399999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3029999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.92200000000000004}]},{"osd":3,"last update":"Wed May 8 23:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.89300000000000002,"15min":0.88400000000000001},"min":{"1min":0.67000000000000004,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1890000000000001,"5min":1.482,"15min":1.482},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.93899999999999995,"15min":0.96799999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.0489999999999999,"5min":1.272,"15min":1.272},"last":0.94999999999999996}]},{"osd":4,"last update":"Wed May 8 23:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":1.002,"15min":1.036},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.2509999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.0289999999999999,"15min":1.028},"min":{"1min":0.749,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.204,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.90200000000000002}]},{"osd":5,"last update":"Wed May 8 23:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.022,"15min":1.048},"min":{"1min":0.71099999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.246,"5min":2.109,"15min":2.109},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.105,"15min":1.1379999999999999},"min":{"1min":0.85399999999999998,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3460000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.0740000000000001}]},{"osd":7,"last update":"Wed May 8 23:33:31 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.175,"15min":1.1990000000000001},"min":{"1min":0.872,"5min":0.752,"15min":0.752},"max":{"1min":1.276,"5min":2.544,"15min":2.544},"last":1.01},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.1060000000000001},"min":{"1min":0.81100000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.409,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.048}]}]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27664,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":655360,"data_stored":270941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:33:27 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.878,"15min":0.88700000000000001},"min":{"1min":0.61899999999999999,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.3,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.94299999999999995,"15min":0.97299999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1910000000000001,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.90600000000000003}]},{"osd":1,"last update":"Wed May 8 23:33:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.89400000000000002,"15min":0.89900000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3400000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.879},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.117,"5min":1.177,"15min":1.177},"last":0.87}]},{"osd":2,"last update":"Wed May 8 23:33:27 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.90900000000000003,"15min":0.91400000000000003},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.329,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93999999999999995,"15min":0.92600000000000005},"min":{"1min":0.69099999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.258,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.63}]},{"osd":3,"last update":"Wed May 8 23:33:27 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.93999999999999995,"15min":0.97299999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0760000000000001,"5min":1.395,"15min":1.395},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.96599999999999997,"15min":0.95599999999999996},"min":{"1min":0.63700000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.401,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.98999999999999999}]},{"osd":4,"last update":"Wed May 8 23:33:27 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.97599999999999998,"15min":1.0069999999999999},"min":{"1min":0.67300000000000004,"5min":0.624,"15min":0.624},"max":{"1min":1.242,"5min":1.361,"15min":1.361},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.997,"5min":1.099,"15min":1.1479999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4019999999999999,"5min":2.464,"15min":2.464},"last":0.96799999999999997}]},{"osd":6,"last update":"Wed May 8 23:33:27 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0620000000000001,"15min":1.0649999999999999},"min":{"1min":0.74399999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.4410000000000001,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0740000000000001,"15min":1.0800000000000001},"min":{"1min":0.72199999999999998,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.3979999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.3069999999999999}]},{"osd":7,"last update":"Wed May 8 23:33:27 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.1659999999999999,"15min":1.2070000000000001},"min":{"1min":0.78700000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.456,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.014},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1759999999999999,"15min":1.1779999999999999},"min":{"1min":0.90200000000000002,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.5660000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.0580000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":270941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.84399999999999997,"15min":0.85199999999999998},"min":{"1min":0.54000000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.79600000000000004,"15min":0.77900000000000003},"min":{"1min":0.623,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.67700000000000005}]},{"osd":1,"last update":"Wed May 8 23:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.874,"15min":0.85499999999999998},"min":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.87,"15min":0.89400000000000002},"min":{"1min":0.57899999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2769999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.83899999999999997}]},{"osd":2,"last update":"Wed May 8 23:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.88600000000000001,"15min":0.89700000000000002},"min":{"1min":0.623,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2050000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.877},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.89800000000000002,"15min":0.90600000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.92300000000000004}]},{"osd":3,"last update":"Wed May 8 23:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85799999999999998,"15min":0.85099999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.93700000000000006,"15min":0.94099999999999995},"min":{"1min":0.68100000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.357,"5min":1.431,"15min":1.431},"last":0.97199999999999998}]},{"osd":5,"last update":"Wed May 8 23:33:31 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95699999999999996,"15min":0.96699999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.996},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.95899999999999996,"15min":0.95099999999999996},"min":{"1min":0.58199999999999996,"5min":0.441,"15min":0.441},"max":{"1min":1.409,"5min":1.504,"15min":1.504},"last":1.2010000000000001}]},{"osd":6,"last update":"Wed May 8 23:33:31 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.03,"15min":1.032},"min":{"1min":0.71699999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.135,"15min":1.153},"min":{"1min":0.80800000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.03}]},{"osd":7,"last update":"Wed May 8 23:33:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.073,"15min":1.081},"min":{"1min":0.75600000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.653,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1439999999999999,"15min":1.1479999999999999},"min":{"1min":0.93899999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.1379999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27664,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":655360,"data_stored":270941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.625,"15min":0.627},"min":{"1min":0.51400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.91900000000000004,"5min":1.032,"15min":1.032},"last":0.69299999999999995}]},{"osd":1,"last update":"Wed May 8 23:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64100000000000001,"15min":0.64300000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85299999999999998,"5min":0.871,"15min":0.871},"last":1.097},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.48799999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.95899999999999996}]},{"osd":2,"last update":"Wed May 8 23:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.74399999999999999,"15min":0.74199999999999999},"min":{"1min":0.48899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92600000000000005,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0960000000000001,"5min":1.242,"15min":1.242},"last":1.022}]},{"osd":4,"last update":"Wed May 8 23:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.91700000000000004,"15min":0.93500000000000005},"min":{"1min":0.67000000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.238,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.95899999999999996,"15min":0.97099999999999997},"min":{"1min":0.69399999999999995,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.361,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":1.2969999999999999}]},{"osd":5,"last update":"Wed May 8 23:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.004,"15min":1.018},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.25,"5min":1.623,"15min":1.623},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.997,"15min":1.0049999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.464,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.133}]},{"osd":6,"last update":"Wed May 8 23:33:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1,"15min":1.002},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3080000000000001,"5min":1.482,"15min":1.482},"last":1.25},{"interface":"front","average":{"1min":1.026,"5min":1.0740000000000001,"15min":1.1040000000000001},"min":{"1min":0.78300000000000003,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.46,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.3779999999999999}]},{"osd":7,"last update":"Wed May 8 23:33:30 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0760000000000001,"15min":1.085},"min":{"1min":0.76600000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0580000000000001,"15min":1.0600000000000001},"min":{"1min":0.80900000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.395,"5min":1.552,"15min":1.552},"last":1.353}]}]},{"osd":2,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":270941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63700000000000001,"15min":0.64900000000000002},"min":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0169999999999999,"5min":1.099,"15min":1.099},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.44900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80300000000000005,"5min":1.129,"15min":1.129},"last":0.66900000000000004}]},{"osd":1,"last update":"Wed May 8 23:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71299999999999997,"15min":0.70999999999999996},"min":{"1min":0.51300000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.626},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78800000000000003,"15min":0.79100000000000004},"min":{"1min":0.52500000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0049999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.80600000000000005}]},{"osd":3,"last update":"Wed May 8 23:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.66400000000000003,"15min":0.67700000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82399999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.79300000000000004,"15min":0.79800000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97299999999999998,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.88400000000000001}]},{"osd":4,"last update":"Wed May 8 23:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.94799999999999995,"15min":0.97199999999999998},"min":{"1min":0.69299999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0780000000000001,"5min":1.367,"15min":1.367},"last":1.077},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.94199999999999995,"15min":0.95899999999999996},"min":{"1min":0.70299999999999996,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.0649999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.92100000000000004}]},{"osd":5,"last update":"Wed May 8 23:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99099999999999999,"15min":0.98599999999999999},"min":{"1min":0.78500000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.747},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.056,"15min":1.077},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.97499999999999998}]},{"osd":6,"last update":"Wed May 8 23:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.0369999999999999,"15min":1.0660000000000001},"min":{"1min":0.66700000000000004,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.22,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.004},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.073,"15min":1.0740000000000001},"min":{"1min":0.93799999999999994,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.381,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.95299999999999996}]},{"osd":7,"last update":"Wed May 8 23:33:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.109,"15min":1.1200000000000001},"min":{"1min":0.82099999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.4890000000000001,"5min":1.758,"15min":1.758},"last":1.04},{"interface":"front","average":{"1min":0.999,"5min":1.0800000000000001,"15min":1.095},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.2070000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.103}]}]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":270941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.57099999999999995},"min":{"1min":0.443,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.629,"15min":0.61899999999999999},"min":{"1min":0.442,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.66200000000000003}]},{"osd":2,"last update":"Wed May 8 23:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65900000000000003,"15min":0.66700000000000004},"min":{"1min":0.47999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69399999999999995,"15min":0.67200000000000004},"min":{"1min":0.39000000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.62}]},{"osd":3,"last update":"Wed May 8 23:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.74199999999999999,"15min":0.72799999999999998},"min":{"1min":0.47199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.012,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.72099999999999997,"15min":0.71499999999999997},"min":{"1min":0.52100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.169,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.997}]},{"osd":4,"last update":"Wed May 8 23:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.93200000000000005,"15min":0.94599999999999995},"min":{"1min":0.64500000000000002,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1859999999999999,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":0.871},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.96299999999999997,"15min":0.94599999999999995},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.2509999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.82699999999999996}]},{"osd":5,"last update":"Wed May 8 23:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0029999999999999,"15min":1.006},"min":{"1min":0.75,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.216,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0569999999999999,"15min":1.0720000000000001},"min":{"1min":0.77200000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3859999999999999,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.0289999999999999}]},{"osd":6,"last update":"Wed May 8 23:33:28 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0389999999999999,"15min":1.0349999999999999},"min":{"1min":0.73399999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.276,"5min":1.633,"15min":1.633},"last":1.139},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.1140000000000001,"15min":1.135},"min":{"1min":0.78200000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.51,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.2150000000000001}]},{"osd":7,"last update":"Wed May 8 23:33:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0329999999999999,"15min":1.0049999999999999},"min":{"1min":0.78300000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.2330000000000001,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.083,"15min":1.083},"min":{"1min":0.80100000000000005,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.349,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.3300000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1249280,"data_stored":861309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.85499999999999998,"15min":0.88800000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.123,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.90900000000000003,"15min":0.91400000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.264,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.0449999999999999}]},{"osd":1,"last update":"Wed May 8 23:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.502,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.097,"5min":1.359,"15min":1.359},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.90800000000000003,"15min":0.91400000000000003},"min":{"1min":0.60999999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.171,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.1220000000000001}]},{"osd":2,"last update":"Wed May 8 23:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88300000000000001,"15min":0.874},"min":{"1min":0.64800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1339999999999999,"5min":1.331,"15min":1.331},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.86599999999999999,"15min":0.88},"min":{"1min":0.63500000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.083,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.76700000000000002}]},{"osd":3,"last update":"Wed May 8 23:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.90900000000000003,"15min":0.91000000000000003},"min":{"1min":0.71899999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.98499999999999999,"15min":0.98899999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1779999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":1.2829999999999999}]},{"osd":4,"last update":"Wed May 8 23:33:34 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.89100000000000001,"15min":0.90700000000000003},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.0960000000000001,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97799999999999998,"15min":0.97899999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2290000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.99199999999999999}]},{"osd":5,"last update":"Wed May 8 23:33:34 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0329999999999999,"15min":1.03},"min":{"1min":0.90400000000000003,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.218,"5min":1.714,"15min":1.714},"last":1.2330000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.98399999999999999,"15min":0.98999999999999999},"min":{"1min":0.77500000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.1910000000000001,"5min":1.325,"15min":1.325},"last":0.88800000000000001}]},{"osd":6,"last update":"Wed May 8 23:33:34 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0429999999999999,"15min":1.0469999999999999},"min":{"1min":0.82799999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.2589999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.095,"5min":1.1020000000000001,"15min":1.1060000000000001},"min":{"1min":0.82399999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.2749999999999999,"5min":1.496,"15min":1.496},"last":1.2090000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1249280,"data_stored":861309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.63700000000000001,"15min":0.64800000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.82699999999999996,"5min":2.008,"15min":2.008},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.73099999999999998,"15min":0.76600000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.89900000000000002,"5min":1.351,"15min":1.351},"last":0.64300000000000002}]},{"osd":2,"last update":"Wed May 8 23:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.73599999999999999,"15min":0.746},"min":{"1min":0.34999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.95699999999999996,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.73299999999999998,"15min":0.76100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80100000000000005,"5min":1.869,"15min":1.869},"last":0.60299999999999998}]},{"osd":3,"last update":"Wed May 8 23:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67700000000000005,"15min":0.68799999999999994},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79900000000000004,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.81799999999999995,"15min":0.84799999999999998},"min":{"1min":0.46500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":1.0169999999999999,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.79300000000000004}]},{"osd":4,"last update":"Wed May 8 23:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.94799999999999995,"15min":0.96499999999999997},"min":{"1min":0.61699999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.1439999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.90300000000000002,"15min":0.89400000000000002},"min":{"1min":0.64600000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.2490000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.69799999999999995}]},{"osd":5,"last update":"Wed May 8 23:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.016,"15min":1.048},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3440000000000001,"5min":1.474,"15min":1.474},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.92200000000000004,"15min":0.91300000000000003},"min":{"1min":0.67200000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.113,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.99099999999999999}]},{"osd":6,"last update":"Wed May 8 23:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":1.0289999999999999,"15min":1.0629999999999999},"min":{"1min":0.64300000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.095,"5min":1.736,"15min":1.736},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.087,"15min":1.111},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3919999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.103}]},{"osd":7,"last update":"Wed May 8 23:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0900000000000001,"15min":1.1100000000000001},"min":{"1min":0.66900000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.26,"5min":1.72,"15min":1.72},"last":1.046},{"interface":"front","average":{"1min":1.01,"5min":1.089,"15min":1.105},"min":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.1950000000000001,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.96699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:33:49.640 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-08T23:33:49.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.753+0000 7f33e5f71700 1 Processor -- start 2024-05-08T23:33:49.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.753+0000 7f33e5f71700 1 -- start start 2024-05-08T23:33:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33e5f71700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80e13c0 0x7f33d803a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33e5f71700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f33d800abf0 0x7f33d8009e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33e5f71700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d80559d0 con 0x7f33d8051790 2024-05-08T23:33:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33e5f71700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d80e17a0 con 0x7f33d800abf0 2024-05-08T23:33:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33e5f71700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d803a760 con 0x7f33d80e13c0 2024-05-08T23:33:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33dffff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f33d800abf0 0x7f33d8009e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33dffff700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f33d800abf0 0x7f33d8009e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57792/0 (socket says 172.21.15.81:57792) 2024-05-08T23:33:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33dffff700 1 -- 172.21.15.81:0/873921477 learned_addr learned my addr 172.21.15.81:0/873921477 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33e4f6f700 1 --2- 172.21.15.81:0/873921477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80e13c0 0x7f33d803a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33dffff700 1 -- 172.21.15.81:0/873921477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80e13c0 msgr2=0x7f33d803a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33dffff700 1 --2- 172.21.15.81:0/873921477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80e13c0 0x7f33d803a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33dffff700 1 -- 172.21.15.81:0/873921477 >> v1:172.21.15.117:6789/0 conn(0x7f33d8051790 legacy=0x7f33d8055e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33dffff700 1 -- 172.21.15.81:0/873921477 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d8054dc0 con 0x7f33d800abf0 2024-05-08T23:33:49.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33dffff700 1 --2- 172.21.15.81:0/873921477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d800abf0 0x7f33d8009e00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009910 tx=0x7f33d4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c36c1ceed01b3e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:49.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33df7fe700 1 -- 172.21.15.81:0/873921477 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4011830 con 0x7f33d800abf0 2024-05-08T23:33:49.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33df7fe700 1 -- 172.21.15.81:0/873921477 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d4011990 con 0x7f33d800abf0 2024-05-08T23:33:49.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33df7fe700 1 -- 172.21.15.81:0/873921477 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4011d40 con 0x7f33d800abf0 2024-05-08T23:33:49.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33e5f71700 1 -- 172.21.15.81:0/873921477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d800abf0 msgr2=0x7f33d8009e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:49.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33e5f71700 1 --2- 172.21.15.81:0/873921477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d800abf0 0x7f33d8009e00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009910 tx=0x7f33d4009c20 comp rx=0 tx=0).stop 2024-05-08T23:33:49.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33e5f71700 1 -- 172.21.15.81:0/873921477 shutdown_connections 2024-05-08T23:33:49.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33e5f71700 1 --2- 172.21.15.81:0/873921477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d800abf0 0x7f33d8009e00 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33e5f71700 1 --2- 172.21.15.81:0/873921477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80e13c0 0x7f33d803a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.757+0000 7f33e5f71700 1 -- 172.21.15.81:0/873921477 >> 172.21.15.81:0/873921477 conn(0x7f33d8056d80 msgr2=0x7f33d8057190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:49.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.761+0000 7f33e5f71700 1 -- 172.21.15.81:0/873921477 shutdown_connections 2024-05-08T23:33:49.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.761+0000 7f33e5f71700 1 -- 172.21.15.81:0/873921477 wait complete. 2024-05-08T23:33:49.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.761+0000 7f33e5f71700 1 Processor -- start 2024-05-08T23:33:49.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.761+0000 7f33e5f71700 1 -- start start 2024-05-08T23:33:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.761+0000 7f33e5f71700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d800abf0 0x7f33d804cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.761+0000 7f33e5f71700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d8051790 0x7f33d80373f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.761+0000 7f33e5f71700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80e13c0 0x7f33d8050040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.761+0000 7f33e5f71700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d817a0a0 con 0x7f33d8051790 2024-05-08T23:33:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.761+0000 7f33e5f71700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d804d4b0 con 0x7f33d800abf0 2024-05-08T23:33:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.761+0000 7f33e5f71700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d817a360 con 0x7f33d80e13c0 2024-05-08T23:33:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.761+0000 7f33dffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d8051790 0x7f33d80373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33dffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d8051790 0x7f33d80373f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57794/0 (socket says 172.21.15.81:57794) 2024-05-08T23:33:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33dffff700 1 -- 172.21.15.81:0/3534094733 learned_addr learned my addr 172.21.15.81:0/3534094733 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33e4f6f700 1 --2- 172.21.15.81:0/3534094733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d800abf0 0x7f33d804cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33e5770700 1 --2- 172.21.15.81:0/3534094733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80e13c0 0x7f33d8050040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33dffff700 1 -- 172.21.15.81:0/3534094733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80e13c0 msgr2=0x7f33d8050040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:49.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33dffff700 1 --2- 172.21.15.81:0/3534094733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80e13c0 0x7f33d8050040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33dffff700 1 -- 172.21.15.81:0/3534094733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d800abf0 msgr2=0x7f33d804cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:49.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33dffff700 1 --2- 172.21.15.81:0/3534094733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d800abf0 0x7f33d804cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33dffff700 1 -- 172.21.15.81:0/3534094733 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d4009570 con 0x7f33d8051790 2024-05-08T23:33:49.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33e5770700 1 --2- 172.21.15.81:0/3534094733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80e13c0 0x7f33d8050040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:49.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33dffff700 1 --2- 172.21.15.81:0/3534094733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d8051790 0x7f33d80373f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f33d4000c00 tx=0x7f33d4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33dd7fa700 1 -- 172.21.15.81:0/3534094733 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4004590 con 0x7f33d8051790 2024-05-08T23:33:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33dd7fa700 1 -- 172.21.15.81:0/3534094733 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d401e070 con 0x7f33d8051790 2024-05-08T23:33:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33e5f71700 1 -- 172.21.15.81:0/3534094733 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d817a640 con 0x7f33d8051790 2024-05-08T23:33:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33dd7fa700 1 -- 172.21.15.81:0/3534094733 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4003900 con 0x7f33d8051790 2024-05-08T23:33:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33e5f71700 1 -- 172.21.15.81:0/3534094733 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d817ac00 con 0x7f33d8051790 2024-05-08T23:33:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.765+0000 7f33e5f71700 1 -- 172.21.15.81:0/3534094733 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d817adf0 con 0x7f33d8051790 2024-05-08T23:33:49.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.769+0000 7f33dd7fa700 1 -- 172.21.15.81:0/3534094733 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f33d4003a60 con 0x7f33d8051790 2024-05-08T23:33:49.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.769+0000 7f33dd7fa700 1 --2- 172.21.15.81:0/3534094733 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f33cc066760 0x7f33cc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:49.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.769+0000 7f33e4f6f700 1 --2- 172.21.15.81:0/3534094733 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f33cc066760 0x7f33cc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:49.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.769+0000 7f33dd7fa700 1 -- 172.21.15.81:0/3534094733 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f33d4084c90 con 0x7f33d8051790 2024-05-08T23:33:49.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.769+0000 7f33e4f6f700 1 --2- 172.21.15.81:0/3534094733 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f33cc066760 0x7f33cc068c20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009bf0 tx=0x7f33d0009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:49.776 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.773+0000 7f33dd7fa700 1 -- 172.21.15.81:0/3534094733 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f33d4052390 con 0x7f33d8051790 2024-05-08T23:33:49.948 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.945+0000 7f33e5f71700 1 -- 172.21.15.81:0/3534094733 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f33d8005890 con 0x7f33d8051790 2024-05-08T23:33:49.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.949+0000 7f33dd7fa700 1 -- 172.21.15.81:0/3534094733 <== mon.0 v2:172.21.15.81:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f33d4052390 con 0x7f33d8051790 2024-05-08T23:33:49.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.949+0000 7f33dd7fa700 1 -- 172.21.15.81:0/3534094733 <== mon.0 v2:172.21.15.81:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7f33d40559b0 con 0x7f33d8051790 2024-05-08T23:33:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.957+0000 7f33e5f71700 1 -- 172.21.15.81:0/3534094733 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f33cc066760 msgr2=0x7f33cc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.957+0000 7f33e5f71700 1 --2- 172.21.15.81:0/3534094733 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f33cc066760 0x7f33cc068c20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009bf0 tx=0x7f33d0009240 comp rx=0 tx=0).stop 2024-05-08T23:33:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.957+0000 7f33e5f71700 1 -- 172.21.15.81:0/3534094733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d8051790 msgr2=0x7f33d80373f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.957+0000 7f33e5f71700 1 --2- 172.21.15.81:0/3534094733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d8051790 0x7f33d80373f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f33d4000c00 tx=0x7f33d4004770 comp rx=0 tx=0).stop 2024-05-08T23:33:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.957+0000 7f33e5f71700 1 -- 172.21.15.81:0/3534094733 shutdown_connections 2024-05-08T23:33:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.957+0000 7f33e5f71700 1 --2- 172.21.15.81:0/3534094733 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f33cc066760 0x7f33cc068c20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.957+0000 7f33e5f71700 1 --2- 172.21.15.81:0/3534094733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80e13c0 0x7f33d8050040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.957+0000 7f33e5f71700 1 --2- 172.21.15.81:0/3534094733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d8051790 0x7f33d80373f0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.957+0000 7f33e5f71700 1 --2- 172.21.15.81:0/3534094733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d800abf0 0x7f33d804cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.957+0000 7f33e5f71700 1 -- 172.21.15.81:0/3534094733 >> 172.21.15.81:0/3534094733 conn(0x7f33d8056d80 msgr2=0x7f33d803a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.957+0000 7f33e5f71700 1 -- 172.21.15.81:0/3534094733 shutdown_connections 2024-05-08T23:33:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:49.957+0000 7f33e5f71700 1 -- 172.21.15.81:0/3534094733 wait complete. 2024-05-08T23:33:49.972 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T23:33:49.972 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:33:50.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e527a700 1 Processor -- start 2024-05-08T23:33:50.078 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e527a700 1 -- start start 2024-05-08T23:33:50.079 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e527a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e006b370 0x7f59e015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e527a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f59e00b35c0 0x7f59e006b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e527a700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f59e0160d10 0x7f59e01611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e527a700 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e0071ce0 con 0x7f59e0160d10 2024-05-08T23:33:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e527a700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e00ad980 con 0x7f59e00b35c0 2024-05-08T23:33:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e527a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e015e6e0 con 0x7f59e006b370 2024-05-08T23:33:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e4a79700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f59e0160d10 0x7f59e01611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e4a79700 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f59e0160d10 0x7f59e01611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57804/0 (socket says 172.21.15.81:57804) 2024-05-08T23:33:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e4a79700 1 -- 172.21.15.81:0/3564045560 learned_addr learned my addr 172.21.15.81:0/3564045560 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59df7fe700 1 --2- 172.21.15.81:0/3564045560 >> v2:172.21.15.117:3300/0 conn(0x7f59e00b35c0 0x7f59e006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59dffff700 1 --2- 172.21.15.81:0/3564045560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e006b370 0x7f59e015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e4a79700 1 -- 172.21.15.81:0/3564045560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e006b370 msgr2=0x7f59e015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e4a79700 1 --2- 172.21.15.81:0/3564045560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e006b370 0x7f59e015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e4a79700 1 -- 172.21.15.81:0/3564045560 >> v2:172.21.15.117:3300/0 conn(0x7f59e00b35c0 msgr2=0x7f59e006b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e4a79700 1 --2- 172.21.15.81:0/3564045560 >> v2:172.21.15.117:3300/0 conn(0x7f59e00b35c0 0x7f59e006b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.073+0000 7f59e4a79700 1 -- 172.21.15.81:0/3564045560 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e006bc90 con 0x7f59e0160d10 2024-05-08T23:33:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59e4a79700 1 --2- 172.21.15.81:0/3564045560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e0160d10 0x7f59e01611a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f59e0066a10 tx=0x7f59d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33668cfbd5ca6cde server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59deffd700 1 -- 172.21.15.81:0/3564045560 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800e070 con 0x7f59e0160d10 2024-05-08T23:33:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59deffd700 1 -- 172.21.15.81:0/3564045560 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f59d800b910 con 0x7f59e0160d10 2024-05-08T23:33:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59deffd700 1 -- 172.21.15.81:0/3564045560 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800bce0 con 0x7f59e0160d10 2024-05-08T23:33:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59e527a700 1 -- 172.21.15.81:0/3564045560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e0160d10 msgr2=0x7f59e01611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59e527a700 1 --2- 172.21.15.81:0/3564045560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e0160d10 0x7f59e01611a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f59e0066a10 tx=0x7f59d800b330 comp rx=0 tx=0).stop 2024-05-08T23:33:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59e527a700 1 -- 172.21.15.81:0/3564045560 shutdown_connections 2024-05-08T23:33:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59e527a700 1 --2- 172.21.15.81:0/3564045560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e0160d10 0x7f59e01611a0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59e527a700 1 --2- 172.21.15.81:0/3564045560 >> v2:172.21.15.117:3300/0 conn(0x7f59e00b35c0 0x7f59e006b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59e527a700 1 --2- 172.21.15.81:0/3564045560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e006b370 0x7f59e015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59e527a700 1 -- 172.21.15.81:0/3564045560 >> 172.21.15.81:0/3564045560 conn(0x7f59e0070960 msgr2=0x7f59e0070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59e527a700 1 -- 172.21.15.81:0/3564045560 shutdown_connections 2024-05-08T23:33:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59e527a700 1 -- 172.21.15.81:0/3564045560 wait complete. 2024-05-08T23:33:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59e527a700 1 Processor -- start 2024-05-08T23:33:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59e527a700 1 -- start start 2024-05-08T23:33:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59e527a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e006b370 0x7f59e015e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.077+0000 7f59e527a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e00b35c0 0x7f59e013d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e527a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e0160d10 0x7f59e0151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e527a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e015fc60 con 0x7f59e0160d10 2024-05-08T23:33:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e527a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e015efe0 con 0x7f59e006b370 2024-05-08T23:33:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e527a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e015f2c0 con 0x7f59e00b35c0 2024-05-08T23:33:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e4a79700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e0160d10 0x7f59e0151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e4a79700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e0160d10 0x7f59e0151ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57820/0 (socket says 172.21.15.81:57820) 2024-05-08T23:33:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59dffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e006b370 0x7f59e015e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e4a79700 1 -- 172.21.15.81:0/3514861758 learned_addr learned my addr 172.21.15.81:0/3514861758 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-08T23:33:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59df7fe700 1 --2- 172.21.15.81:0/3514861758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e00b35c0 0x7f59e013d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e4a79700 1 -- 172.21.15.81:0/3514861758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e00b35c0 msgr2=0x7f59e013d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e4a79700 1 --2- 172.21.15.81:0/3514861758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e00b35c0 0x7f59e013d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e4a79700 1 -- 172.21.15.81:0/3514861758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e006b370 msgr2=0x7f59e015e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e4a79700 1 --2- 172.21.15.81:0/3514861758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e006b370 0x7f59e015e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e4a79700 1 -- 172.21.15.81:0/3514861758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d8009b30 con 0x7f59e0160d10 2024-05-08T23:33:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59dffff700 1 --2- 172.21.15.81:0/3514861758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e006b370 0x7f59e015e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e4a79700 1 --2- 172.21.15.81:0/3514861758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e0160d10 0x7f59e0151ca0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f59d800b370 tx=0x7f59d8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59dcff9700 1 -- 172.21.15.81:0/3514861758 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800be70 con 0x7f59e0160d10 2024-05-08T23:33:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59dcff9700 1 -- 172.21.15.81:0/3514861758 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f59d80092e0 con 0x7f59e0160d10 2024-05-08T23:33:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59dcff9700 1 -- 172.21.15.81:0/3514861758 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d80256a0 con 0x7f59e0160d10 2024-05-08T23:33:50.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e527a700 1 -- 172.21.15.81:0/3514861758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e015f5a0 con 0x7f59e0160d10 2024-05-08T23:33:50.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e527a700 1 -- 172.21.15.81:0/3514861758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e013df20 con 0x7f59e0160d10 2024-05-08T23:33:50.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.081+0000 7f59e527a700 1 -- 172.21.15.81:0/3514861758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e015fda0 con 0x7f59e0160d10 2024-05-08T23:33:50.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.085+0000 7f59dcff9700 1 -- 172.21.15.81:0/3514861758 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59d8025800 con 0x7f59e0160d10 2024-05-08T23:33:50.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.085+0000 7f59dcff9700 1 --2- 172.21.15.81:0/3514861758 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f59cc066730 0x7f59cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:50.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.085+0000 7f59dffff700 1 --2- 172.21.15.81:0/3514861758 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f59cc066730 0x7f59cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:50.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.085+0000 7f59dcff9700 1 -- 172.21.15.81:0/3514861758 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f59d8085910 con 0x7f59e0160d10 2024-05-08T23:33:50.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.085+0000 7f59dffff700 1 --2- 172.21.15.81:0/3514861758 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f59cc066730 0x7f59cc068bf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009ec0 tx=0x7f59d0009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:50.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.089+0000 7f59dcff9700 1 -- 172.21.15.81:0/3514861758 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f59d8052f80 con 0x7f59e0160d10 2024-05-08T23:33:50.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.301+0000 7f59e527a700 1 -- 172.21.15.81:0/3514861758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59e00672a0 con 0x7f59e0160d10 2024-05-08T23:33:50.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.301+0000 7f59dcff9700 1 -- 172.21.15.81:0/3514861758 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f59d8020090 con 0x7f59e0160d10 2024-05-08T23:33:50.305 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T23:33:50.305 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/2105335102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":2105335102},{"type":"v1","addr":"172.21.15.117:6834","nonce":2105335102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2900127785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2900127785},{"type":"v1","addr":"172.21.15.81:6835","nonce":2900127785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1899281779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1899281779},{"type":"v1","addr":"172.21.15.81:6837","nonce":1899281779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":6298,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/958886953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":958886953},{"type":"v1","addr":"172.21.15.117:6835","nonce":958886953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[]} 2024-05-08T23:33:50.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.305+0000 7f59e527a700 1 -- 172.21.15.81:0/3514861758 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f59cc066730 msgr2=0x7f59cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:50.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.305+0000 7f59e527a700 1 --2- 172.21.15.81:0/3514861758 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f59cc066730 0x7f59cc068bf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009ec0 tx=0x7f59d0009240 comp rx=0 tx=0).stop 2024-05-08T23:33:50.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.305+0000 7f59e527a700 1 -- 172.21.15.81:0/3514861758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e0160d10 msgr2=0x7f59e0151ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:50.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.305+0000 7f59e527a700 1 --2- 172.21.15.81:0/3514861758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e0160d10 0x7f59e0151ca0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f59d800b370 tx=0x7f59d8004af0 comp rx=0 tx=0).stop 2024-05-08T23:33:50.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.305+0000 7f59e527a700 1 -- 172.21.15.81:0/3514861758 shutdown_connections 2024-05-08T23:33:50.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.305+0000 7f59e527a700 1 --2- 172.21.15.81:0/3514861758 >> [v2:172.21.15.81:6832/25965,v1:172.21.15.81:6833/25965] conn(0x7f59cc066730 0x7f59cc068bf0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:50.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.305+0000 7f59e527a700 1 --2- 172.21.15.81:0/3514861758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59e0160d10 0x7f59e0151ca0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:50.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.305+0000 7f59e527a700 1 --2- 172.21.15.81:0/3514861758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e00b35c0 0x7f59e013d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:50.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.305+0000 7f59e527a700 1 --2- 172.21.15.81:0/3514861758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e006b370 0x7f59e015e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:50.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.305+0000 7f59e527a700 1 -- 172.21.15.81:0/3514861758 >> 172.21.15.81:0/3514861758 conn(0x7f59e0070960 msgr2=0x7f59e0161120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:50.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.305+0000 7f59e527a700 1 -- 172.21.15.81:0/3514861758 shutdown_connections 2024-05-08T23:33:50.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T23:33:50.305+0000 7f59e527a700 1 -- 172.21.15.81:0/3514861758 wait complete. 2024-05-08T23:33:50.311 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 25 2024-05-08T23:33:50.323 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-08T23:33:50.324 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-08T23:33:50.324 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T23:33:50.324 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:33:50.333 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T23:33:50.333 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T23:33:50.333 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:33:50.348 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T23:33:50.348 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T23:33:50.348 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:33:50.366 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T23:33:50.366 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T23:33:50.366 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:33:50.425 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T23:33:50.426 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-08T23:33:50.426 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-08T23:33:50.426 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:33:50.521 INFO:tasks.ceph.osd.0:Stopped 2024-05-08T23:33:50.521 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-08T23:33:50.521 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:33:50.607 INFO:tasks.ceph.osd.1:Stopped 2024-05-08T23:33:50.607 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-08T23:33:50.607 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:33:50.701 INFO:tasks.ceph.osd.2:Stopped 2024-05-08T23:33:50.701 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-08T23:33:50.701 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:33:51.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-05-08T23:33:56.803 INFO:tasks.ceph.osd.3:Stopped 2024-05-08T23:33:56.803 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-08T23:33:56.803 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:33:56.895 INFO:tasks.ceph.osd.4:Stopped 2024-05-08T23:33:56.895 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-08T23:33:56.895 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:33:57.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-05-08T23:33:57.708 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-05-08T23:34:02.997 INFO:tasks.ceph.osd.5:Stopped 2024-05-08T23:34:02.998 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-08T23:34:02.998 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:34:03.083 INFO:tasks.ceph.osd.6:Stopped 2024-05-08T23:34:03.083 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-08T23:34:03.083 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:34:03.414 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~6s 2024-05-08T23:34:06.458 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:34:06.460 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:34:08.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-05-08T23:34:08.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-05-08T23:34:09.185 INFO:tasks.ceph.osd.7:Stopped 2024-05-08T23:34:09.185 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-08T23:34:09.185 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-08T23:34:09.185 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:34:09.214 INFO:tasks.ceph.mgr.y:Stopped 2024-05-08T23:34:09.215 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-08T23:34:09.215 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:34:09.246 INFO:tasks.ceph.mgr.x:Stopped 2024-05-08T23:34:09.246 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-08T23:34:09.247 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T23:34:09.247 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:34:09.306 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T23:34:09.306 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T23:34:09.306 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:34:09.355 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T23:34:09.355 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T23:34:09.356 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:34:09.403 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T23:34:09.403 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-08T23:34:09.403 DEBUG:teuthology.orchestra.run.smithi081:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-05-08T23:34:09.425 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:34:09.425 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-08T23:34:09.601 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:34:09.602 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-08T23:34:09.843 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:34:09.843 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-08T23:34:10.051 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:34:10.051 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-08T23:34:10.241 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:34:10.241 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-08T23:34:10.545 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:34:10.545 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-08T23:34:10.735 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:34:10.736 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-08T23:34:10.926 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:34:10.927 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-08T23:34:11.102 INFO:tasks.ceph:Archiving mon data... 2024-05-08T23:34:11.103 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698173/data/mon.a.tgz 2024-05-08T23:34:11.104 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-05-08T23:34:11.111 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.vohd9KeGKR 2024-05-08T23:34:11.111 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.vohd9KeGKR 2024-05-08T23:34:11.241 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.vohd9KeGKR 2024-05-08T23:34:11.268 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.vohd9KeGKR is 348KB 2024-05-08T23:34:11.329 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.vohd9KeGKR 2024-05-08T23:34:11.336 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698173/data/mon.b.tgz 2024-05-08T23:34:11.336 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-05-08T23:34:11.343 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.lZlJ2OoCi5 2024-05-08T23:34:11.343 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.lZlJ2OoCi5 2024-05-08T23:34:11.497 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.lZlJ2OoCi5 2024-05-08T23:34:11.519 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.lZlJ2OoCi5 is 459KB 2024-05-08T23:34:11.580 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.lZlJ2OoCi5 2024-05-08T23:34:11.586 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698173/data/mon.c.tgz 2024-05-08T23:34:11.586 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-05-08T23:34:11.593 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.KDmP57IR17 2024-05-08T23:34:11.593 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.KDmP57IR17 2024-05-08T23:34:11.706 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.KDmP57IR17 2024-05-08T23:34:11.723 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.KDmP57IR17 is 458KB 2024-05-08T23:34:11.745 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.KDmP57IR17 2024-05-08T23:34:11.750 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-08T23:34:11.751 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T23:34:11.753 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T23:34:11.808 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-05-08T23:34:11.809 DEBUG:teuthology.orchestra.run.smithi081:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-05-08T23:34:11.812 DEBUG:teuthology.orchestra.run.smithi117:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-05-08T23:34:11.839 INFO:teuthology.orchestra.run.smithi117.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-05-08T23:34:11.843 INFO:teuthology.orchestra.run.smithi081.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-05-08T23:34:11.849 INFO:tasks.ceph:Archiving crash dumps... 2024-05-08T23:34:11.851 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698173/remote/smithi081/crash 2024-05-08T23:34:11.852 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T23:34:11.868 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698173/remote/smithi117/crash 2024-05-08T23:34:11.869 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T23:34:11.886 INFO:tasks.ceph:Shutting down logrotate 2024-05-08T23:34:11.886 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:34:11.912 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:34:11.979 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-08T23:34:11.995 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-08T23:34:12.010 INFO:tasks.ceph:Compressing logs... 2024-05-08T23:34:12.010 DEBUG:teuthology.orchestra.run.smithi081:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:34:12.013 DEBUG:teuthology.orchestra.run.smithi117:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:34:13.086 INFO:tasks.ceph:Archiving logs... 2024-05-08T23:34:13.086 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698173/remote/smithi081/log 2024-05-08T23:34:13.087 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T23:34:13.515 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698173/remote/smithi117/log 2024-05-08T23:34:13.516 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T23:34:13.677 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T23:34:13.686 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T23:34:13.687 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T23:34:13.689 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T23:34:13.768 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T23:34:13.777 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T23:34:13.777 DEBUG:teuthology.orchestra.run.smithi081:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:34:13.780 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:34:13.803 INFO:teuthology.orchestra.run.smithi117.stdout: remote refid st t when poll reach delay offset jitter 2024-05-08T23:34:13.803 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================== 2024-05-08T23:34:13.803 INFO:teuthology.orchestra.run.smithi117.stdout:*hv01.front.sepi 67.205.162.81 3 u 22 64 377 0.082 -0.012 1.576 2024-05-08T23:34:13.803 INFO:teuthology.orchestra.run.smithi117.stdout:+hv02.front.sepi 63.231.80.2 3 u 18 64 377 0.087 2.363 1.525 2024-05-08T23:34:13.803 INFO:teuthology.orchestra.run.smithi117.stdout:+hv03.front.sepi 74.6.168.72 3 u 17 64 377 0.090 0.267 0.790 2024-05-08T23:34:13.803 INFO:teuthology.orchestra.run.smithi117.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T23:34:13.805 INFO:teuthology.orchestra.run.smithi081.stdout: remote refid st t when poll reach delay offset jitter 2024-05-08T23:34:13.805 INFO:teuthology.orchestra.run.smithi081.stdout:============================================================================== 2024-05-08T23:34:13.805 INFO:teuthology.orchestra.run.smithi081.stdout:+hv01.front.sepi 67.205.162.81 3 u 24 64 377 0.090 -0.859 0.432 2024-05-08T23:34:13.805 INFO:teuthology.orchestra.run.smithi081.stdout:*hv02.front.sepi 63.231.80.2 3 u 24 64 377 0.086 1.720 0.434 2024-05-08T23:34:13.805 INFO:teuthology.orchestra.run.smithi081.stdout:+hv03.front.sepi 74.6.168.72 3 u 17 64 377 0.059 1.053 0.554 2024-05-08T23:34:13.805 INFO:teuthology.orchestra.run.smithi081.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-05-08T23:34:13.805 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T23:34:13.815 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T23:34:13.815 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T23:34:13.841 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T23:34:13.875 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T23:34:13.908 INFO:teuthology.task.internal:Duration was 820.735436 seconds 2024-05-08T23:34:13.909 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T23:34:13.941 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T23:34:13.942 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T23:34:13.942 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-05-08T23:34:13.944 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T23:34:13.998 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T23:34:13.999 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T23:34:13.999 DEBUG:teuthology.orchestra.run.smithi081:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T23:34:14.010 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-05-08T23:34:14.010 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T23:34:14.021 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T23:34:14.021 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:34:14.052 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:34:14.092 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T23:34:14.092 DEBUG:teuthology.orchestra.run.smithi081:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T23:34:14.094 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T23:34:14.171 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T23:34:14.182 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T23:34:14.182 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T23:34:14.196 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T23:34:14.213 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T23:34:14.224 DEBUG:teuthology.orchestra.run.smithi081:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:34:14.241 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:34:14.253 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = core 2024-05-08T23:34:14.256 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-05-08T23:34:14.272 DEBUG:teuthology.orchestra.run.smithi081:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:34:14.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:34:14.278 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:34:14.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:34:14.283 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T23:34:14.293 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T23:34:14.294 DEBUG:teuthology.misc:Transferring archived files from smithi081:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698173/remote/smithi081 2024-05-08T23:34:14.294 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T23:34:14.349 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698173/remote/smithi117 2024-05-08T23:34:14.349 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T23:34:14.375 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T23:34:14.375 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T23:34:14.378 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T23:34:14.383 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T23:34:14.395 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T23:34:14.395 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T23:34:14.418 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T23:34:14.418 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T23:34:14.428 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T23:34:14.434 INFO:teuthology.orchestra.run.smithi081.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 23:34 /home/ubuntu/cephtest 2024-05-08T23:34:14.436 INFO:teuthology.orchestra.run.smithi117.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 23:34 /home/ubuntu/cephtest 2024-05-08T23:34:14.438 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T23:34:14.450 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T23:34:14.520 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/pool-perm} duration: 820.7354357242584 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-08T23:34:14.521 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T23:34:14.599 INFO:teuthology.run:pass